It happens because you get PathTokenizer + WDF etc. WDF is still not correct if you use it after a "complicated" chain. It is only really safe after somethign like WhitespaceTokenizer.
On Mon, May 5, 2014 at 5:30 AM, Uwe Schindler <[email protected]> wrote: > The same happened to me locally while running tests, with the other type of > PathTokenizers, too! > > ----- > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: [email protected] > > >> -----Original Message----- >> From: Policeman Jenkins Server [mailto:[email protected]] >> Sent: Monday, May 05, 2014 9:58 AM >> To: [email protected] >> Subject: [JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.7.0_60-ea-b15) - Build # >> 10125 - Failure! >> >> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/10125/ >> Java: 64bit/jdk1.7.0_60-ea-b15 -XX:+UseCompressedOops -XX:+UseG1GC >> >> 1 tests failed. >> REGRESSION: >> org.apache.lucene.analysis.core.TestRandomChains.testRandomChains >> >> Error Message: >> startOffset must be non-negative, and endOffset must be >= startOffset, >> startOffset=12,endOffset=11 >> >> Stack Trace: >> java.lang.IllegalArgumentException: startOffset must be non-negative, and >> endOffset must be >= startOffset, startOffset=12,endOffset=11 >> at >> __randomizedtesting.SeedInfo.seed([4C26A4A98D15A0DC:71C78DC8CA07B >> D1C]:0) >> at org.apache.lucene.analysis.Token.checkOffsets(Token.java:621) >> at org.apache.lucene.analysis.Token.setOffset(Token.java:304) >> at >> org.apache.lucene.analysis.shingle.ShingleFilter.incrementToken(ShingleFilte >> r.java:345) >> at >> org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(Validating >> TokenFilter.java:68) >> at >> org.apache.lucene.analysis.synonym.SynonymFilter.parse(SynonymFilter.jav >> a:358) >> at >> org.apache.lucene.analysis.synonym.SynonymFilter.incrementToken(Synon >> ymFilter.java:624) >> at >> org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(Validating >> TokenFilter.java:68) >> at >> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsist >> ency(BaseTokenStreamTestCase.java:702) >> at >> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(B >> aseTokenStreamTestCase.java:613) >> at >> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(B >> aseTokenStreamTestCase.java:512) >> at >> org.apache.lucene.analysis.core.TestRandomChains.testRandomChains(Test >> RandomChains.java:927) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j >> ava:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces >> sorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(Randomize >> dRunner.java:1618) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(Rando >> mizedRunner.java:827) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(Rando >> mizedRunner.java:863) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(Rando >> mizedRunner.java:877) >> at >> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRule >> SetupTeardownChained.java:50) >> at >> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCa >> cheSanity.java:51) >> at >> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeA >> fterRule.java:46) >> at >> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1 >> .evaluate(SystemPropertiesInvariantRule.java:55) >> at >> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleTh >> readAndTestName.java:49) >> at >> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRule >> IgnoreAfterMaxFailures.java:65) >> at >> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure >> .java:48) >> at >> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat >> ementAdapter.java:36) >> at >> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner. >> run(ThreadLeakControl.java:360) >> at >> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask >> (ThreadLeakControl.java:793) >> at >> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadL >> eakControl.java:453) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(Ran >> domizedRunner.java:836) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(Rando >> mizedRunner.java:738) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(Rando >> mizedRunner.java:772) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(Rando >> mizedRunner.java:783) >> at >> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeA >> fterRule.java:46) >> at >> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreCl >> assName.java:42) >> at >> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1 >> .evaluate(SystemPropertiesInvariantRule.java:55) >> at >> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet >> hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) >> at >> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet >> hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) >> at >> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat >> ementAdapter.java:36) >> at >> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat >> ementAdapter.java:36) >> at >> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat >> ementAdapter.java:36) >> at >> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAss >> ertionsRequired.java:43) >> at >> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure >> .java:48) >> at >> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRule >> IgnoreAfterMaxFailures.java:65) >> at >> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnore >> TestSuites.java:55) >> at >> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat >> ementAdapter.java:36) >> at >> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner. >> run(ThreadLeakControl.java:360) >> at java.lang.Thread.run(Thread.java:745) >> >> >> >> >> Build Log: >> [...truncated 5650 lines...] >> [junit4] Suite: org.apache.lucene.analysis.core.TestRandomChains >> [junit4] 2> TEST FAIL: useCharFilter=false >> text='\u0591\u0007\u0b66\u29f9 >> &#x</p tayz' >> [junit4] 2> Exception from random analyzer: >> [junit4] 2> charfilters= >> [junit4] 2> tokenizer= >> [junit4] 2> >> org.apache.lucene.analysis.path.ReversePathHierarchyTokenizer(org.apache >> .lucene.analysis.core.TestRandomChains$CheckThatYouDidntReadAnythingR >> eaderWrapper@224ae57f) >> [junit4] 2> filters= >> [junit4] 2> >> org.apache.lucene.analysis.in.IndicNormalizationFilter(ValidatingTokenFilter >> @d2d9441 >> term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLen >> gth=1,payload=null,flags=0,type=word) >> [junit4] 2> >> org.apache.lucene.analysis.miscellaneous.WordDelimiterFilter(LUCENE_4_9, >> ValidatingTokenFilter@3b910bfe >> term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLen >> gth=1,payload=null,flags=0,type=word, [63, -18, 0, -18, 30, -76, -106, 84, >> -59, - >> 104, 66, 91, 75, 99, 45, 123, 87, 121, -7, 111, -16, 86, 23, 116, -84, -61, >> 125, -81, >> 43, -27, -78, 112, 64, -13, -126, -20, 66, 22, 48, -43, -26, 62, 86, -65, >> 78, -17, -85, - >> 14, -66, -107, 43, 110, -15], -39, [swm, q, zkgooaz, cttg, trvwhcdye, >> rbxmobel]) >> [junit4] 2> >> org.apache.lucene.analysis.shingle.ShingleFilter(ValidatingTokenFilter@5e6d >> a98b >> term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLen >> gth=1,payload=null,flags=0,type=word) >> [junit4] 2> >> org.apache.lucene.analysis.synonym.SynonymFilter(ValidatingTokenFilter@3 >> 443d9df >> term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLen >> gth=1,payload=null,flags=0,type=word, >> org.apache.lucene.analysis.synonym.SynonymMap@5294da47, true) >> [junit4] 2> offsetsAreCorrect=false >> [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestRandomChains >> -Dtests.method=testRandomChains -Dtests.seed=4C26A4A98D15A0DC - >> Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=no - >> Dtests.timezone=America/Asuncion -Dtests.file.encoding=UTF-8 >> [junit4] ERROR 12.9s J1 | TestRandomChains.testRandomChains <<< >> [junit4] > Throwable #1: java.lang.IllegalArgumentException: >> startOffset >> must be non-negative, and endOffset must be >= startOffset, >> startOffset=12,endOffset=11 >> [junit4] > at >> __randomizedtesting.SeedInfo.seed([4C26A4A98D15A0DC:71C78DC8CA07B >> D1C]:0) >> [junit4] > at >> org.apache.lucene.analysis.Token.checkOffsets(Token.java:621) >> [junit4] > at >> org.apache.lucene.analysis.Token.setOffset(Token.java:304) >> [junit4] > at >> org.apache.lucene.analysis.shingle.ShingleFilter.incrementToken(ShingleFilte >> r.java:345) >> [junit4] > at >> org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(Validating >> TokenFilter.java:68) >> [junit4] > at >> org.apache.lucene.analysis.synonym.SynonymFilter.parse(SynonymFilter.jav >> a:358) >> [junit4] > at >> org.apache.lucene.analysis.synonym.SynonymFilter.incrementToken(Synon >> ymFilter.java:624) >> [junit4] > at >> org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(Validating >> TokenFilter.java:68) >> [junit4] > at >> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsist >> ency(BaseTokenStreamTestCase.java:702) >> [junit4] > at >> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(B >> aseTokenStreamTestCase.java:613) >> [junit4] > at >> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(B >> aseTokenStreamTestCase.java:512) >> [junit4] > at >> org.apache.lucene.analysis.core.TestRandomChains.testRandomChains(Test >> RandomChains.java:927) >> [junit4] > at java.lang.Thread.run(Thread.java:745) >> [junit4] 2> NOTE: leaving temporary files on disk at: >> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x- >> Linux/lucene/build/analysis/common/test/J1/./lucene.analysis.core.TestRan >> domChains-4C26A4A98D15A0DC-001 >> [junit4] 2> NOTE: test params are: codec=Lucene42, >> sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): >> {dummy=DFR I(n)2}, locale=no, timezone=America/Asuncion >> [junit4] 2> NOTE: Linux 3.8.0-39-generic amd64/Oracle Corporation >> 1.7.0_60-ea (64-bit)/cpus=8,threads=1,free=198461320,total=455081984 >> [junit4] 2> NOTE: All tests run in this JVM: [TestCodepointCountFilter, >> TestKeepFilterFactory, TestCapitalizationFilter, TestCollationKeyAnalyzer, >> TestNorwegianMinimalStemFilter, TestAnalyzers, TestSolrSynonymParser, >> TestSegmentingTokenizerBase, NGramTokenFilterTest, >> TestPortugueseStemFilterFactory, TestCaseInsensitive, TestCharacterUtils, >> TestCzechStemmer, TestDanishAnalyzer, TestPorterStemFilterFactory, >> TestPatternReplaceFilter, TestRussianLetterTokenizer, >> TestGreekLowerCaseFilterFactory, TestHunspellStemFilterFactory, >> TestTrimFilterFactory, TestApostropheFilter, TestCJKWidthFilter, >> TestGermanMinimalStemFilterFactory, TestRussianAnalyzer, >> TestCommonGramsFilterFactory, TestSnowballPorterFilterFactory, >> TestOptionalCondition, TestCharArrayMap, TestCharTokenizers, >> TestFrenchAnalyzer, TestReverseStringFilter, TestTypeTokenFilterFactory, >> TestPortugueseMinimalStemFilter, TestSpanishLightStemFilter, >> TestArmenianAnalyzer, TestLimitTokenCountFilter, >> DateRecognizerSinkTokenizerTest, TestGreekStemFilterFactory, >> TestKeepWordFilter, TestGalicianStemFilter, TestPortugueseStemFilter, >> TestPersianCharFilter, TestBulgarianStemmer, TestPortugueseAnalyzer, >> TestHunspellStemFilter, TypeAsPayloadTokenFilterTest, >> TestSwedishLightStemFilter, TestStandardFactories, TestSnowball, >> TestAllAnalyzersHaveFactories, TestKeywordMarkerFilterFactory, >> TestEnglishAnalyzer, TestDictionaryCompoundWordTokenFilterFactory, >> TestStandardAnalyzer, TestSlowSynonymFilter, TestHindiFilters, >> TestGermanStemFilterFactory, TestSpanishLightStemFilterFactory, >> TestAnalysisSPILoader, TestPortugueseLightStemFilter, >> DelimitedPayloadTokenFilterTest, TestHindiAnalyzer, >> TestLimitTokenPositionFilter, TestRussianFilters, TestFrenchLightStemFilter, >> TestFinnishLightStemFilterFactory, TestHungarianLightStemFilterFactory, >> TestRandomChains] >> [junit4] Completed on J1 in 32.22s, 2 tests, 1 error <<< FAILURES! >> >> [...truncated 475 lines...] >> BUILD FAILED >> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:467: The >> following error occurred while executing this line: >> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:447: The >> following error occurred while executing this line: >> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:45: The >> following error occurred while executing this line: >> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: >> The following error occurred while executing this line: >> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build.xml:539: >> The following error occurred while executing this line: >> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common- >> build.xml:2017: The following error occurred while executing this line: >> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x- >> Linux/lucene/analysis/build.xml:106: The following error occurred while >> executing this line: >> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x- >> Linux/lucene/analysis/build.xml:38: The following error occurred while >> executing this line: >> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/module- >> build.xml:60: The following error occurred while executing this line: >> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common- >> build.xml:1297: The following error occurred while executing this line: >> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common- >> build.xml:921: There were test failures: 266 suites, 1450 tests, 1 error, 1 >> ignored >> >> Total time: 21 minutes 13 seconds >> Build step 'Invoke Ant' marked build as failure Description set: Java: >> 64bit/jdk1.7.0_60-ea-b15 -XX:+UseCompressedOops -XX:+UseG1GC >> Archiving artifacts Recording test results Email was triggered for: Failure >> Sending email for trigger: Failure >> > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
