Re: Document updates work as delete/add under the hood

2015-07-10 Thread Erick Erickson
Well, if it's a docValues field you can do update in place at the Lucene level for certain types of simple values (numerics, strings, but not text types) see: https://issues.apache.org/jira/browse/LUCENE-5189 In essence the reason it's a delete/re-add is that the structure of the postings list and

Re: Delete Parents without any children...

2015-07-10 Thread Ravikumar Govindarajan
Yes, as you suggested simply wrapping up postings with LZ4 could not be best-fit for all cases. Byte-Pair Encoding looks very promising I accidentally stumbled upon this JIRA and found it was abandoned mid-way. Thanks for sharing the details -- Ravi On Fri, Jul 3, 2015 at 5:46 PM, Adrien Grand

Re: DocValues API and behaviour when no values for a field in an segment

2015-07-10 Thread Varun Thacker
Thanks Adrien! That makes sense now. DocValues.getSortedSet is pretty cool since the app doesn't need to worry about if the field is single valued or multi valued. On Thu, Jul 9, 2015 at 5:16 PM, Adrien Grand wrote: > LeafReader.get*Values is a low-level API whose return value depends on > what

Re: Document updates work as delete/add under the hood

2015-07-10 Thread Gimantha Bandara
ah.. I misread the thread,I thought you were using two APIs to acheive the same done by updateDocument. Yes it is an overhead and harder for user to keep track of the fields that he doesn't need to update. Already there is a Jira opened for this[1]. [1] https://issues.apache.org/jira/browse/LUCENE

Re: Document updates work as delete/add under the hood

2015-07-10 Thread chalitha udara Perera
Hi Gimatha, Yes. It is possible to use IndexWriter updateDocument() to update document. But with that method what happens under the hood is it deletes matching documents and re-index new document. I need to update only a single field. Re-indexing a new document with updated field + other fields se

Re: Document updates work as delete/add under the hood

2015-07-10 Thread Gimantha Bandara
Hi Chalitha, You can simply use indexWriter.updateDocument to update the existing index documents On Fri, Jul 10, 2015 at 11:38 AM, chalitha udara Perera < chalithaud...@gmail.com> wrote: > Hi All, > > I have a requirement for updating lucene index (add single field for > existing docs and modif