Detect index changes

2013-10-16 Thread Alice Wong
Hello, I am quite confused about the Lucene NRT feature. And there are not many examples out there. My understanding is we can create an DirectoryReader from a IndexWriter. Whenever IndexWriter changes the index, we can use DirectoryReader to detect the changes and recreate DirectoryReader if nec

Re: prepareCommit vs Commit

2013-10-16 Thread Michael McCandless
prepareCommit flushes any buffered docs (in RAM) to a new segment, applies all deletes, fsync's the new files, writes all but the last 4 checksum bytes of the segments file. Ie, it tries to do everything possible short of finishing the commit up front, so that if there will be a problem, e.g. disk

Extending Query class for custom query evaluation.

2013-10-16 Thread Marcos Juarez Lopez
Posted something similar some time ago, but didn't get any responses, so I thought I'd try again with more details. We allow end-user queries that have our own proprietary query language, which we then translate to a Lucene Query* AST. This has worked well for us. However, a few of the operators

prepareCommit vs Commit

2013-10-16 Thread Phani Chaitanya
Hi all, I'd like to know a bit more in detail about what is happening behind the scenes in case of prepareCommit vs Commit. Also, I read that if a indexing request comes while commit is requested, it opens a new segment for the indexing and commit happens as usual. So, the question is when wo

Re: Optimizing Filters

2013-10-16 Thread James Clarke
Filters are created programmatically per request (and customized for the request) thus in order to benefit from CachingWrapperFilter we require a mechanism for looking up CachingWrapperFilters based on the request. But this is certainly an area worth trying (we could probably reuse each filter 10 t