The problem here is incomplete cleanup: This test parses the whole algorithm, it then creates an Algorithm instance containing all Tasks to execute. The "WriteLineDocsTask" is instantiated, which in fact opens the output stream for the line docs file in its ctor. But as the tasks are never executed, so the Task's close() is never called.
I fixed the whole thing by making Algorithm implement AutoCloseable (not Closeable, because it throws not only IOException). I also backported the whole stuff to Lucene 4, because this is unrelated to Robert's changes. Roberts changes just triggered other code execution paths, so GC did not clean up algorithm. TODO: verify other uses of Algorithm and check that they are correctly closed. In fact I have seen this bug sometimes when running tests locally, so this was existent since long time, just in most cases GC was faster! Uwe ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: [email protected] > -----Original Message----- > From: Uwe Schindler [mailto:[email protected]] > Sent: Sunday, September 14, 2014 12:21 PM > To: [email protected] > Subject: RE: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_67) - > Build # 4310 - Still Failing! > > This fails reproducible. > > I will dig! > > ----- > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: [email protected] > > > -----Original Message----- > > From: Policeman Jenkins Server [mailto:[email protected]] > > Sent: Sunday, September 14, 2014 2:48 AM > > To: [email protected]; [email protected]; [email protected]; > > [email protected]; [email protected]; [email protected] > > Subject: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_67) - > > Build # > > 4310 - Still Failing! > > > > Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4310/ > > Java: 64bit/jdk1.7.0_67 -XX:-UseCompressedOops - > > XX:+UseConcMarkSweepGC > > > > 1 tests failed. > > FAILED: > > > junit.framework.TestSuite.org.apache.lucene.benchmark.byTask.TestPerfT > > a > > sksParse > > > > Error Message: > > Could not remove the following files (in the order of attempts): > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt: > > java.nio.file.AccessDeniedException: > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt: > > java.nio.file.AccessDeniedException: > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > .TestPerfTasksParse-2D34950549928D3F-001: > > java.nio.file.DirectoryNotEmptyException: > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > .TestPerfTasksParse-2D34950549928D3F-001 > > > > Stack Trace: > > java.io.IOException: Could not remove the following files (in the > > order of > > attempts): > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt: > > java.nio.file.AccessDeniedException: > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt: > > java.nio.file.AccessDeniedException: > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > .TestPerfTasksParse-2D34950549928D3F-001: > > java.nio.file.DirectoryNotEmptyException: > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > .TestPerfTasksParse-2D34950549928D3F-001 > > > > at __randomizedtesting.SeedInfo.seed([2D34950549928D3F]:0) > > at org.apache.lucene.util.IOUtils.rm(IOUtils.java:288) > > at > > org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestR > > ul > > eTemporaryFilesCleanup.java:126) > > at > > com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways > > (T > > estRuleAdapter.java:31) > > at > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Sta > > t > > ementAdapter.java:43) > > at > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Sta > > t > > ementAdapter.java:36) > > at > > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleA > > ss > > ertionsRequired.java:43) > > at > > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFail > > ure > > .java:48) > > at > > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestR > > ule > > IgnoreAfterMaxFailures.java:65) > > at > > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgn > > ore > > TestSuites.java:55) > > at > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Sta > > t > > ementAdapter.java:36) > > at > > > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner. > > run(ThreadLeakControl.java:365) > > at java.lang.Thread.run(Thread.java:745) > > > > > > > > > > Build Log: > > [...truncated 7622 lines...] > > [junit4] Suite: org.apache.lucene.benchmark.byTask.TestPerfTasksParse > > [junit4] 1> ------------> queries: > > [junit4] 1> > > [junit4] 1> ------------> queries: > > [junit4] 1> > > [junit4] 1> ------------> queries: > > [junit4] 1> > > [junit4] 1> ------------> queries: > > [junit4] 1> > > [junit4] 1> ------------> queries: > > [junit4] 1> > > [junit4] 1> ------------> queries: > > [junit4] 1> > > [junit4] 1> ------------> queries: > > [junit4] 1> > > [junit4] 1> ------------> queries: > > [junit4] 1> > > [junit4] 1> ------------> queries: > > [junit4] 1> > > [junit4] 1> ------------> queries: > > [junit4] 1> > > [junit4] 1> ------------> queries: > > [junit4] 1> > > [junit4] 1> ------------> queries: > > [junit4] 1> > > [junit4] 1> ------------> queries: > > [junit4] 1> > > [junit4] 1> Spatial Strategy: > > RecursivePrefixTreeStrategy(SPG:(QuadPrefixTree(maxLevels:26,ctx:Spati > > al Context{geo=true, calculator=Haversine, worldBounds=Rect(minX=- > > 180.0,maxX=180.0,minY=- > > 90.0,maxY=90.0)})),pointsOnly,pruneLeafyBranches) > > [junit4] 1> ------------> queries: > > [junit4] 1> > > [junit4] 1> ------------> queries: > > [junit4] 1> > > [junit4] 1> ------------> queries: > > [junit4] 1> > > [junit4] 1> ------------> queries: > > [junit4] 1> > > [junit4] 1> ------------> queries: > > [junit4] 1> > > [junit4] 2> NOTE: test params are: codec=Lucene410: {}, docValues:{}, > > sim=DefaultSimilarity, locale=sq_AL, timezone=America/Yakutat > > [junit4] 2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.7.0_67 (64- > > bit)/cpus=2,threads=1,free=120281832,total=140083200 > > [junit4] 2> NOTE: All tests run in this JVM: [DocMakerTest, > > WriteLineDocTaskTest, AddIndexesTaskTest, StreamUtilsTest, > > TrecContentSourceTest, TestQualityRun, PerfTaskTest, > > LineDocSourceTest, TestPerfTasksLogic, TestPerfTasksParse] > > [junit4] 2> NOTE: reproduce with: ant test - > Dtestcase=TestPerfTasksParse > > -Dtests.seed=2D34950549928D3F -Dtests.slow=true -Dtests.locale=sq_AL - > > Dtests.timezone=America/Yakutat -Dtests.file.encoding=ISO-8859-1 > > [junit4] ERROR 0.00s | TestPerfTasksParse (suite) <<< > > [junit4] > Throwable #1: java.io.IOException: Could not remove the > > following files (in the order of attempts): > > [junit4] > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt: > > java.nio.file.AccessDeniedException: > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt > > [junit4] > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt: > > java.nio.file.AccessDeniedException: > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt > > [junit4] > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > .TestPerfTasksParse-2D34950549928D3F-001: > > java.nio.file.DirectoryNotEmptyException: > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > .TestPerfTasksParse-2D34950549928D3F-001 > > [junit4] > at > > __randomizedtesting.SeedInfo.seed([2D34950549928D3F]:0) > > [junit4] > at org.apache.lucene.util.IOUtils.rm(IOUtils.java:288) > > [junit4] > at java.lang.Thread.run(Thread.java:745) > > [junit4] Completed in 3.00s, 3 tests, 1 error <<< FAILURES! > > > > [...truncated 33 lines...] > > BUILD FAILED > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > Windows\build.xml:491: The following error occurred while executing > > this > > line: > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > Windows\build.xml:471: The following error occurred while executing > > this > > line: > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > Windows\build.xml:60: > > The following error occurred while executing this line: > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra- > > targets.xml:39: The following error occurred while executing this line: > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > Windows\lucene\build.xml:462: The following error occurred while > > executing this line: > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > Windows\lucene\common-build.xml:2140: The following error occurred > > while executing this line: > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > Windows\lucene\module-build.xml:58: The following error occurred while > > executing this line: > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > Windows\lucene\common-build.xml:1358: The following error occurred > > while executing this line: > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > Windows\lucene\common-build.xml:961: There were test failures: 18 > > suites, > > 93 tests, 1 suite-level error > > > > Total time: 31 minutes 50 seconds > > Build step 'Invoke Ant' marked build as failure [description-setter] > > Description set: Java: 64bit/jdk1.7.0_67 -XX:-UseCompressedOops - > > XX:+UseConcMarkSweepGC Archiving artifacts Recording test results > > Email was triggered for: Failure - Any Sending email for trigger: > > Failure - Any > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] For additional > commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
