0

I'm creating a full text search engine for a series of approximately 10 public domain books. The corpus will rarely be updated. Proximity search will be the core feature, e.g. locating "hello" within 5 words of "world".

I'm looking at both Haystack for Django/Python and Ruby solutions. I'm aware of: Solr, Elasticsearch, Whoosh, Xapian, and Sphinx.

With such a small corpus I'm focused on ease of implementation, and right now I'm leaning toward Whoosh (pure Python) or Sphinx (with Haystack or maybe Ruby on Rails). Thoughts?

  • Do you have a definition for "word A near word B"? Does "near" mean within 3 words before or after word B? Within 5 words? Is the "nearness" definable by the user? – Bulrush May 24 '16 at 18:19
  • It would be ideal if it were definable by the user. Even just "in the same chapter" would be sufficient though. Same paragraph or a specific # of words apart would be even better but not strictly required. – Christopher Galpin May 24 '16 at 19:47
  • 1
    I think the feature you mean is called proximity search. – unor May 25 '16 at 2:27
  • 1
    @unor that has narrowed me down quite a bit. I might try out Whoosh after noting in its documentation that it supports proximity search, and I already have a Django setup. Thanks! – Christopher Galpin May 25 '16 at 22:36
  • Actually realized Ruby might be better suited for my needs, and noted Sphinx has paragraph, sentence, and just "nearby" proximity searches, awesome! sphinxsearch.com/docs/current/extended-syntax.html Might use that. I'll answer my own question here once I really start implementing. – Christopher Galpin May 27 '16 at 14:59
1

I went with Python, Flask, Whoosh, and gunicorn. It was an absolute cinch, I had it running almost immediately, and the Python nativity of Whoosh has been quite nice. Very advanced search, easy additions. Perfect for my small corpus.

I've completed and published the code for my engine. Here is a live demonstration. Read my introductory article for more details, and if I write any further articles for this project they will be listed on my site.

Here's a reference to proximity search with Whoosh's query language.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.