Well I never thought I would get there and it took a few attempts but I managed to stick the CommonCrawls text corpses of the Internet into MongoDB. This is around 13TB of text, which is definitely more than I can read in a day!
The next task is to work out the language of each page, I have a few ways to do that, then ignore anything that is not English. I am not really sure how I would analyise text data in a language I do not understand hence why I plan to skip it.
After that I need to try and work out what each page’s context is, then chuck it in ElasticSearch.
CommonCrawl also contains millions (63,270,007) of links to files like PDFs, Docs, and images. I have started processing this data to see what useful information I can extra.
Shocking, I know, I think I need to hire someone to do a good job of it! Watch this space.
Wow, it appears that I have not updated my blog since Feb 2019. At that time, life was great, brainpower was at a high, ideas were flowing and world domination (Burf Style) was near (Nope this isn’t a Brexit rant). Then life got turned upside down, and I had to deal (well try and deal) with a great crisis in my life, my wife sadly no longer wanted to be with me. 2019 was a horrible horrible year and so I didn’t really build anything, I shut down my sites, projects, and hobbies and just tried to deal with what was going on at home.
Long road ahead
So, I would love to write there was a happy ending to 2019 but these things take years especially when kids are involved and the process has just started however you just can’t keep my crazy brain down for too long! So while I was aimlessly surfing eBay for random things, I came across this for sale!
Seeing an Inmoov robot for sell on eBay is rare as rocking horse poo, it takes people years to print these (4 years for this one) and are usually attached to a college, university or something. This one needed some work but I thought what a perfect project for me to sink my efforts in to! My dad agreed and purchased it as my Christmas present (Thanks dad).
The Inmoov robot project is one of the most amazing 3D projects out there, it allows someone with a lot of time (and some money to buy servos), to print a fully functional top half of a robot, to buy the same sort of thing would cost you hundreds of thousands of pounds. I have printed a few parts of it before to merge with educational products like LEGO and VEX however, I always run out of steam (Time) while actually attempting to print the whole thing. Buying a mostly built one means I can fix a bit at a time.
So this year’s focus is to complete this while learning more about robotics, engineering, and electronics:) I will try and do a YouTube series on my progress and I will still try to think of crazy ways to try and make Burf.co a better search engine.
Watch this space 🙂
Oh boy, these updates are going to get crazier!
So first up I sold 2 copies of the Keyword Research Software which is on offer at KeywordResearchNinja.co.uk The odd thing about this is that I hadn’t even finished the site for people to even access the software because I assumed no one would ever find it until I wanted them to! I hadn’t even finished setting up my business PayPal account which is needed to make a payment link 🙁 This leads me on to…
So, my plan is to learn how to run a business through Vibe Innovation. For me to do this, I need a business bank account, which I also need for my PayPal Business Account. I looked online for good deals on business bank accounts (e.g ones with little or no monthly cost) and went with Tide. Now for them to accept me, I need to have an updated Confirmation Statement logged with Companies House as the current one reflects the previous owners/shareholders. It was fairly easy to do but (I say that, it’s not gone through yet) this whole process is to set up an account, to finish setting up my PayPal account.
My Garage on Google Maps
While trying to achieve something, (which I have completely forgotten what it was) I got sidetracked on to listing Vibe Innovation on Google Maps. Google makes it super easy to list your business however, it picks a picture of where it thinks that address is. The current picture is of some white garages which is a great introduction to new customers 🙂
FOCUS TIME BABY
I am really thinking about the concept of focusing! I really want to focus on the concept of focusing on something. At the moment I seem to be doing far too many small things. I do think I am making some sort of progress but I think my New Years Resolutions will be around doing less but doing the things I do better.
Early morning coding (5 am) is going rather well. Still waiting for design assets from the team but getting there.
Making a little bit of progress, however I have a plan and it’s in this video: