DevCon and an Arduino IoT house, the search engine moving to HiddenWeb.co.uk, hoverboard motors and LiDAR.
Read more of December 2023: DevCon, HiddenWeb and the electric motorbike →
Burf.co search
The answer is 2,828,752,948
Common Crawl's text corpus, around 13 TB and 2.8 billion records, is now loaded into MongoDB.
April 2020: InMoov and Burf.co in lockdown
Lockdown progress: a rebuilt InMoov head and neck, and the Burf.co search project.
My servers shit the bed!
The HP C7000 blade server failed, so I moved to an HP DL580 with 16 TB of storage, which then had its own problems.
Crazy times with Innovation and Ninja’s
A plan to learn how to run a limited company, starting with a keyword research tool.
Elasticsearch : I did it, I used an existing framework, not recreate the wheel :)
Switching Burf.co search to Elasticsearch, fed from Common Crawl data filtered through MongoDB, all in Java.
Read more of Elasticsearch : I did it, I used an existing framework, not recreate the wheel :) →
This weeks update: Just keep spinner!
Burf.co needs to be fast, useful, self-updating and niche, because MongoDB can't serve 100 million pages quickly enough.
Let’s kick it all off!
Training for a bodybuilding contest, and plans to finish Hack24, fix Burf.co and turn the garage into a robotics lab.
ALL THE SERVERS (Sponsored by SUGAR)
Rebuilding the Burf.co server after MongoDB full-text search proved too slow, even on RAID 0.
Tumbleweed and that damn CommonCrawl
Common Crawl's open web data changes the plan for the Burf.co search engine.
