Hi, I know why it fails consequently with the new Java 7 APIs: FileOutputStream since version 1.0 of Java has a custom finalizer() method, so the file gets closed on GC, kept until now for backwards compatibility. But if you the stream with Java 7's NIO.2, the created OutputStream instance (some non-public internal platform specific impl class) no longer has a finalizer(), so the file handle is never closed when the OutputStream is GCed. This is great (f*ck finalizers)!
So by using NIO.2 we have a very nice "detector" for unclosed file, if people do broken stuff like "new Properties(new FileInputStream(...))" as seen in Solr quite often! Uwe ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: [email protected] > -----Original Message----- > From: Uwe Schindler [mailto:[email protected]] > Sent: Sunday, September 14, 2014 1:16 PM > To: [email protected] > Subject: RE: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_67) - > Build # 4310 - Still Failing! > > The problem here is incomplete cleanup: > > This test parses the whole algorithm, it then creates an Algorithm instance > containing all Tasks to execute. The "WriteLineDocsTask" is instantiated, > which in fact opens the output stream for the line docs file in its ctor. But > as > the tasks are never executed, so the Task's close() is never called. > > I fixed the whole thing by making Algorithm implement AutoCloseable (not > Closeable, because it throws not only IOException). I also backported the > whole stuff to Lucene 4, because this is unrelated to Robert's changes. > Roberts changes just triggered other code execution paths, so GC did not > clean up algorithm. > > TODO: verify other uses of Algorithm and check that they are correctly > closed. In fact I have seen this bug sometimes when running tests locally, so > this was existent since long time, just in most cases GC was faster! > > Uwe > > ----- > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: [email protected] > > > > -----Original Message----- > > From: Uwe Schindler [mailto:[email protected]] > > Sent: Sunday, September 14, 2014 12:21 PM > > To: [email protected] > > Subject: RE: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_67) - > > Build # 4310 - Still Failing! > > > > This fails reproducible. > > > > I will dig! > > > > ----- > > Uwe Schindler > > H.-H.-Meier-Allee 63, D-28213 Bremen > > http://www.thetaphi.de > > eMail: [email protected] > > > > > -----Original Message----- > > > From: Policeman Jenkins Server [mailto:[email protected]] > > > Sent: Sunday, September 14, 2014 2:48 AM > > > To: [email protected]; [email protected]; [email protected]; > > > [email protected]; [email protected]; [email protected] > > > Subject: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_67) - > > > Build # > > > 4310 - Still Failing! > > > > > > Build: > > > http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4310/ > > > Java: 64bit/jdk1.7.0_67 -XX:-UseCompressedOops - > > > XX:+UseConcMarkSweepGC > > > > > > 1 tests failed. > > > FAILED: > > > > > > junit.framework.TestSuite.org.apache.lucene.benchmark.byTask.TestPerfT > > > a > > > sksParse > > > > > > Error Message: > > > Could not remove the following files (in the order of attempts): > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt: > > > java.nio.file.AccessDeniedException: > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt: > > > java.nio.file.AccessDeniedException: > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > > .TestPerfTasksParse-2D34950549928D3F-001: > > > java.nio.file.DirectoryNotEmptyException: > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > > .TestPerfTasksParse-2D34950549928D3F-001 > > > > > > Stack Trace: > > > java.io.IOException: Could not remove the following files (in the > > > order of > > > attempts): > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt: > > > java.nio.file.AccessDeniedException: > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt: > > > java.nio.file.AccessDeniedException: > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > > .TestPerfTasksParse-2D34950549928D3F-001: > > > java.nio.file.DirectoryNotEmptyException: > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > > .TestPerfTasksParse-2D34950549928D3F-001 > > > > > > at __randomizedtesting.SeedInfo.seed([2D34950549928D3F]:0) > > > at org.apache.lucene.util.IOUtils.rm(IOUtils.java:288) > > > at > > > org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(Tes > > > tR > > > ul > > > eTemporaryFilesCleanup.java:126) > > > at > > > com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlwa > > > ys > > > (T > > > estRuleAdapter.java:31) > > > at > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(S > > > ta > > > t > > > ementAdapter.java:43) > > > at > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(S > > > ta > > > t > > > ementAdapter.java:36) > > > at > > > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRul > > > eA > > > ss > > > ertionsRequired.java:43) > > > at > > > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFa > > > il > > > ure > > > .java:48) > > > at > > > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(Tes > > > tR > > > ule > > > IgnoreAfterMaxFailures.java:65) > > > at > > > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleI > > > gn > > > ore > > > TestSuites.java:55) > > > at > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(S > > > ta > > > t > > > ementAdapter.java:36) > > > at > > > > > > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner. > > > run(ThreadLeakControl.java:365) > > > at java.lang.Thread.run(Thread.java:745) > > > > > > > > > > > > > > > Build Log: > > > [...truncated 7622 lines...] > > > [junit4] Suite: > org.apache.lucene.benchmark.byTask.TestPerfTasksParse > > > [junit4] 1> ------------> queries: > > > [junit4] 1> > > > [junit4] 1> ------------> queries: > > > [junit4] 1> > > > [junit4] 1> ------------> queries: > > > [junit4] 1> > > > [junit4] 1> ------------> queries: > > > [junit4] 1> > > > [junit4] 1> ------------> queries: > > > [junit4] 1> > > > [junit4] 1> ------------> queries: > > > [junit4] 1> > > > [junit4] 1> ------------> queries: > > > [junit4] 1> > > > [junit4] 1> ------------> queries: > > > [junit4] 1> > > > [junit4] 1> ------------> queries: > > > [junit4] 1> > > > [junit4] 1> ------------> queries: > > > [junit4] 1> > > > [junit4] 1> ------------> queries: > > > [junit4] 1> > > > [junit4] 1> ------------> queries: > > > [junit4] 1> > > > [junit4] 1> ------------> queries: > > > [junit4] 1> > > > [junit4] 1> Spatial Strategy: > > > RecursivePrefixTreeStrategy(SPG:(QuadPrefixTree(maxLevels:26,ctx:Spa > > > ti al Context{geo=true, calculator=Haversine, > > > worldBounds=Rect(minX=- > > > 180.0,maxX=180.0,minY=- > > > 90.0,maxY=90.0)})),pointsOnly,pruneLeafyBranches) > > > [junit4] 1> ------------> queries: > > > [junit4] 1> > > > [junit4] 1> ------------> queries: > > > [junit4] 1> > > > [junit4] 1> ------------> queries: > > > [junit4] 1> > > > [junit4] 1> ------------> queries: > > > [junit4] 1> > > > [junit4] 1> ------------> queries: > > > [junit4] 1> > > > [junit4] 2> NOTE: test params are: codec=Lucene410: {}, docValues:{}, > > > sim=DefaultSimilarity, locale=sq_AL, timezone=America/Yakutat > > > [junit4] 2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.7.0_67 > (64- > > > bit)/cpus=2,threads=1,free=120281832,total=140083200 > > > [junit4] 2> NOTE: All tests run in this JVM: [DocMakerTest, > > > WriteLineDocTaskTest, AddIndexesTaskTest, StreamUtilsTest, > > > TrecContentSourceTest, TestQualityRun, PerfTaskTest, > > > LineDocSourceTest, TestPerfTasksLogic, TestPerfTasksParse] > > > [junit4] 2> NOTE: reproduce with: ant test - > > Dtestcase=TestPerfTasksParse > > > -Dtests.seed=2D34950549928D3F -Dtests.slow=true -Dtests.locale=sq_AL > > > - Dtests.timezone=America/Yakutat -Dtests.file.encoding=ISO-8859-1 > > > [junit4] ERROR 0.00s | TestPerfTasksParse (suite) <<< > > > [junit4] > Throwable #1: java.io.IOException: Could not remove the > > > following files (in the order of attempts): > > > [junit4] > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt: > > > java.nio.file.AccessDeniedException: > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt > > > [junit4] > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt: > > > java.nio.file.AccessDeniedException: > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt > > > [junit4] > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > > .TestPerfTasksParse-2D34950549928D3F-001: > > > java.nio.file.DirectoryNotEmptyException: > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > > > > Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask > > > .TestPerfTasksParse-2D34950549928D3F-001 > > > [junit4] > at > > > __randomizedtesting.SeedInfo.seed([2D34950549928D3F]:0) > > > [junit4] > at org.apache.lucene.util.IOUtils.rm(IOUtils.java:288) > > > [junit4] > at java.lang.Thread.run(Thread.java:745) > > > [junit4] Completed in 3.00s, 3 tests, 1 error <<< FAILURES! > > > > > > [...truncated 33 lines...] > > > BUILD FAILED > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\build.xml:491: The following error occurred while executing > > > this > > > line: > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\build.xml:471: The following error occurred while executing > > > this > > > line: > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > Windows\build.xml:60: > > > The following error occurred while executing this line: > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra- > > > targets.xml:39: The following error occurred while executing this line: > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\build.xml:462: The following error occurred while > > > executing this line: > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\common-build.xml:2140: The following error occurred > > > while executing this line: > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\module-build.xml:58: The following error occurred > > > while executing this line: > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\common-build.xml:1358: The following error occurred > > > while executing this line: > > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk- > > > Windows\lucene\common-build.xml:961: There were test failures: 18 > > > suites, > > > 93 tests, 1 suite-level error > > > > > > Total time: 31 minutes 50 seconds > > > Build step 'Invoke Ant' marked build as failure [description-setter] > > > Description set: Java: 64bit/jdk1.7.0_67 -XX:-UseCompressedOops - > > > XX:+UseConcMarkSweepGC Archiving artifacts Recording test results > > > Email was triggered for: Failure - Any Sending email for trigger: > > > Failure - Any > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] For > > additional commands, e-mail: [email protected] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] For additional > commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
