RE: BitSet in Filters

2014-08-12 Thread Uwe Schindler
Hi, in general you cannot cache Filter, you can cache their DocIdSets (CachingWrapperFilter is for example doing this). Lucene Queries are executed per segment, that means when you index new documents or update new documents, lucene creates new index segments. Older ones *never* change, so a Do

Re: BitSet in Filters

2014-08-12 Thread Sandeep Khanzode
Hi Erick, I have mentioned everything that is relevant, I believe :). However, just to give more background: Assume documents of the order of more than 300 million, and multiple concurrent users running search. I may front Lucene with ElasticSearch, and ES basically calls Lucene TermFilters. My

Re: BitSet in Filters

2014-08-12 Thread Erick Erickson
bq: Unless, I can cache these filters in memory, the cost of constructing this filter at run time per query is not practical Why do you say that? Do you have evidence? Because lots and lots of Solr installations do exactly this and they run fine. So I suspect there's something you're not telling