Hmm, that should be "OK" from Lucene's standpoint. I mean, it should not result in strange merge exceptions later on.
I think there's a bug somewhere in Lucene's efforts to pretend it's fully schema-less ... I'll try to reproduce this. Mike McCandless http://blog.mikemccandless.com On Tue, Oct 11, 2016 at 4:38 AM, Hans Lund <ha.l...@gmail.com> wrote: > Turned out to be must much simpler - we had added a new 'dynamic' field to > a stats doc a count on articles based on identified language code. Having a > set of test documents in German, English, Swedish - no one had suspected > the obvious that the language detection categorized a single document as > being Indonesian, making the stats count id:1. > > I realized that the debug output I added - made output of everything else > that the interesting field (iterating over already added fields - not the > field causing the error later on ;-) > > > > > > On Mon, Oct 10, 2016 at 4:32 PM, Adrien Grand <jpou...@gmail.com> wrote: > >> It looks like the field infos of your index went out of sync with data >> stored in the files about points. >> >> Can you run CheckIndex on your index (potentially with the `-fast` option >> so that it only verifies checksums)? It could be that one of these two >> parts of the index got corrupted. >> >> Since you were able to modify the way add(IndexableField) is implemented, >> I'm wondering if you are running a fork of Lucene? If yes, maybe you did >> some changes that triggered this bug? >> >> Otherwise is your application: >> - using IndexWriter.addIndexes? >> - customizing merging in some way, eg. by wrapping the merge readers? >> >> Le mar. 4 oct. 2016 à 16:40, Hans Lund <ha.l...@gmail.com> a écrit : >> >> > After upgrading to 6.2 we are having problems during merges (after >> running >> > for a while). >> > >> > When the problem occurs its always complaining about the same field - and >> > throws: >> > >> > java.lang.IllegalArgumentException: field="id" did not index point >> values >> > at >> > >> > org.apache.lucene.codecs.lucene60.Lucene60PointsReader.getBKDReader( >> Lucene60PointsReader.java:126) >> > at >> > >> > org.apache.lucene.codecs.lucene60.Lucene60PointsReader. >> size(Lucene60PointsReader.java:224) >> > at >> > >> > org.apache.lucene.codecs.lucene60.Lucene60PointsWriter. >> merge(Lucene60PointsWriter.java:169) >> > at >> > org.apache.lucene.index.SegmentMerger.mergePoints( >> SegmentMerger.java:173) >> > at org.apache.lucene.index.SegmentMerger.merge( >> SegmentMerger.java:122) >> > at >> > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4312) >> > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3889) >> > >> > >> > To figure out where we messed up - I have added some ugly logging to >> > Document: >> > >> > public final void add(IndexableField field) { >> > if ("id".equals(field.name()) && >> > field.fieldType().pointDimensionCount() >> > != 0) { >> > System.err.println("Point value detected"); >> > for (IndexableField i : fields) { >> > System.err.println(i); >> > } >> > } >> > fields.add(field); >> > } >> > >> > In hope to intercept the document we messed up. >> > >> > But to my surprise toString on the suspected field just says (contains a >> > URN): >> > >> > indexed,omitNorms,indexOptions=DOCS<id:urn:wiki:doc:YEL:57028#1-1> >> > >> > So any hints as to why field.fieldType().pointDimensionCount() != 0 >> > >> > and any suggestions what might cause this? >> > >> > Regards >> > Hans Lund >> > >> --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
