What is a good open source information retrieval library (search engine)?

Question

So I have a Java application running on a websphere server. The server has an oracle database of boxes. Each box has a list of things in it. Each thing has a main description (input as text from humans), and a list of label/value pairs (ie, color:black, quantity:3, etc) that describe the thing.

I'd like to allow a user to search this database so I need a library to search the database and rank the results. The user will be allowed to input the same kind of information described above. I want to be able to list the boxes that most closely match what the user input.

I've looked into some search libraries such as Lucene and Xapian, but I'm not sure about the exact capabilities of various libraries. I'd like to be able to see the pros and cons and a little bit about how they work, so I've started doing some research.

But before that, I'd like to hear your suggestions for this particular project.

Answer 1

You can also try ElasticSearch.

ElasticSearch it's a search server on top of Lucene. It provides a Json API for performing the search queries and it's really handy when it comes to scalability.

In order to to index an existing database you should continually poll it's content.
Here is a tutorial for that.

On the downside, you should be familiar with Lucene to go beyond the basics(e.g. custom Analyzers, Tokenizers etc), those things are required for an effective search.

Last but not least ElasticSearch provides AND a Java API which is a wrapper of the Json API. So far I've only worked with Json but I have made some research on this and it's really simple to use if you already have a Java application.

Open Source (Apache 2 License)

Answer 2

Solr might be a good fit for your choice.

As ElasticSearch, Solr is based on Lucene and provides the same functionalities like full-text search, hit highlighting and easy-scalability among others.

Generally when searching for those 2 solutions you will find many resources. I leave it to you to decide which one to use ;) Solr has definitely the advantage when it comes to maturity and number of users.

To index your existing database you can use a DataImportRequestHandler which uses JdbcDataSources to connect to one or many databases and import their content.

Open Source (Apache 2 License)

Answer 3

Well, you said the 3 magic words: database, text-search and Java.
I would strongly suggest using Hibernate-Search because it's made for this purpose.

To be more precise, Hibernate-Search has the ability to:

• Add text-search in your existing database by annotations in your existing entities.
• find by approximation (fuzzy search) and rank results.
• It's Lucene basically but very easy to use in a database especially if you already use Hibernate. You could try Lucene but the overhead of storing the index, manually updating it and simultaneously keep in mind changes in weights of fields or structure of your database, yeah not really worth the trouble...
• Extremely easy to start with: books, excellent documentation, many examples online and a quickstart I've created if you want to check how it works.
• The motto of Hibernate-Search is: "Google your database"

P.S.
I've been using professionally Lucene for over a year and had worked with and in Hibernate-Search in the past.

Open source (LGPL v2.1)