As mentioned in my previous post, I have started looking at Server Side Swift with the aim to build a search engine (Burf.co). To store my crawled data I decided to try and use MongoDB as it supports full-text search out of the box. The original Burf.com used Equinox (made by Compsoft) and then later used Microsoft Indexing Service. This time round I wanted to be a little more scalable. Now there are probably better DB solutions for what I plan to do, but MongoDB seemed really simple to get up and running with. Later on, I should be able to switch out the database layer if needed.

MongoKitten

Now that I had decided to use Swift, and MongoDB, I needed to find a framework that connects them, my friend (who knows his stuff) recommended MongoKitten! I got up and running with it fairly quickly even though I don’t know MongoDB too well. Life was good, however, there were a few things I did struggle with:

Contains

So, search a field for a partial string requires you to use Regex (it seems).

Mongo:

db.users.findOne({"username" : {$regex : ".*eBay.*"}});

MongoKitten:

let query: Query = [

"url": RegularExpression(pattern: ".\(eBay).")

]

let matchingEntities: CollectionSlice<Document> = try pages.find(query)

Sorting results on $meta textScore

MongoDB allows you to setup full text searching across your data, it can be across an entire record, or just certain fields (name, address etc). When you perform a full-text search, MongoDB returns the relevant records with an accuracy score ($meta.textScore). MongoDB lets you change how it creates these scores by allowing you to adjust the weights each field receives e.g name is more important than address.

Mongo:

db.pages.find( {$text: {$search: "ebay"}},{score: {$meta: "textScore" }}).sort({score: {$meta:"textScore"}})

MongoKitten:

let query: Query = [

"$text": ["$search": str ],

"lang" : ["$eq": "en"],

]

let projection: Projection = [

“_id”: .excluded,

“url”: “url”,

“title”: “title”,

"score": ["$meta" : "textScore"]

]

let sort: Sort = [

"score": .custom([

“$meta”: “textScore”

])

]

let matchingEntities: CollectionSlice<Document> = try pages.find(query, sortedBy: sort, projecting: projection, readConcern:nil,  collation:nil, skipping: 0, limitedTo: Settings.searchResultLimit )

Getting Help

I found the best way to get help was to contact the creator(Joannis) of MongoKitten via Slack, he is pretty busy but super helpful!