Greetings!
First off, I realize Lucene is a search engine and therefore does not
possess many of the features of a database. That being said, I have
encountered a particular use case where I need to lookup potentially
thousands of records in a Lucene index based upon an ID (a String field
in the index). This data also needs be sorted based upon any chosen
field in the index. In pseudo code, this is how its currently done:
String[] ids = { "123aeeff", "34eacc", ...}
results.clear();
StringBuffer lookupQuery = new StringBuffer()
for (int i=0; i<ids.size();i++) {
lookupQuery.append(ids.get(i))
lookupQuery.append(" ")
if ((i+1) % 1024 == 0) {
search(lookupQuery.toString())
lookupQuery = new StringBuffer()
}
}
if (lookupQuery.length()>0) {
search(lookupQuery.toString())
}
As you can see, in a loop, Lucene queries are constructed into a maximum
of 1024 terms, for example, consisting of IDS "123aeeff 34eacc ..".
After each query in the loop is constructed, a search is executed and
then the results are combined into a single linkedlist (this is done in
the search function). This works well aside from two outstanding questions:
1. Is executing separate search queries, the best way to lookup
thousands of records in an index? Is there a more efficient way to
lookup thousands of records based upon ID?
2. The results are unsorted after they are combined into a single
linkedlist. What is the best way to sort the combined results based
upon any chosen field in the lucene index? Is there a way to do that
would leverage Lucene's inbuilt sort abilities?
Many thanks for your consideration
Jamie
