Re: scanning whole index stored fields while using best compression mode

2015-06-03 Thread Robert Muir
On Wed, Jun 3, 2015 at 4:59 PM, Anton Zenkov wrote: > Reindexing. If I want to add new fields or change existing fields in the > index I need to go through all documents of the index. > if your reindexing process needs all the docs, i dont think i can really recommend a better way. a merge reader

Re: scanning whole index stored fields while using best compression mode

2015-06-03 Thread Anton Zenkov
Reindexing. If I want to add new fields or change existing fields in the index I need to go through all documents of the index. On Wed, Jun 3, 2015 at 4:46 PM, Robert Muir wrote: > On Wed, Jun 3, 2015 at 4:00 PM, Anton Zenkov > wrote: > > > > > for (int i = 0; i < leafReader.maxDoc(); i++) { >

Re: scanning whole index stored fields while using best compression mode

2015-06-03 Thread Robert Muir
On Wed, Jun 3, 2015 at 4:00 PM, Anton Zenkov wrote: > > for (int i = 0; i < leafReader.maxDoc(); i++) { > DocumentStoredFieldVisitor visitor = new DocumentStoredFieldVisitor(); > fieldsReader.visitDocument(i, visitor); > visitor.getDocument(); > > } > > } > > I was wondering if there

scanning whole index stored fields while using best compression mode

2015-06-03 Thread Anton Zenkov
Hello, I ran into a problem while trying to make a utility which loads all documents in the index one by one. Loading was super slow. Orders of magnitude slower then it is supposed to be. After some debugging and looking at the code I figured that the culprit was the index compression which I set