Geeking with Greg: Findory's got a new search engine

Tuesday, June 15, 2004

A new keyword search engine under the hood for both Findory News and Findory Blogory. Should provide substantially faster keyword searches.

Seun Osewa said...: I didn't notice that the keyword search engine was slow! Can you tell us abstract facts about the engine, like:
* is it a third-party engine?
* If built in-house, what algorithmic changes were made? (it's unlikely that you needed to use any patentable methods, but it would be interesting to know about some of the trade-offs involved in this sort of project).; June 16, 2004 at 4:12 AM
Greg Linden said...: Hi, Seun. Sure, I can tell you a little more about the keyword search engine. It's built in house. Pretty standard implementation; our needs are simple.

Most of the complexity of a search engine comes from the relevance rank, but our application only requires sorting in reverse chronological order, so this really wasn't an issue.

At some point, we may start personalizing the search results, which will make the relevance ranking substantially more complicated. This was our primary reason for building the engine in house. We wanted to be able to extend it in atypical ways in the future.

If you want to build something like this yourself, I can recommend an excellent book on the subject, Managing Gigabytes. A truly incredible work, worth its weight in gold.

However, I wouldn't necessarily recommend that path. There's many open source libraries available for search engines. Building one yourself really isn't necessary (or perhaps even wise).; June 16, 2004 at 8:19 AM

Geeking with Greg