On Mon, 11 Jul 2022 15:31:46 GMT, Roger Riggs <rri...@openjdk.org> wrote:
>> The `ProcessBuilder.pipelineStart()` implementation does not close all of
>> the file descriptors it uses to create the pipeline of processes.
>>
>> The process calling `pipelineStart()` is creating the pipes between the
>> stages.
>> As each process is launched, the file descriptor is inherited by the child
>> process and
>> the child process `dup`s them to the respective stdin/stdout/stderr fd.
>> These copies of inherited file descriptors are handled correctly.
>>
>> Between the launching of each Process, the file descriptor for the read-side
>> of the pipe for the output of the previous process is kept open (in the
>> parent process invoking `pipelineStart`). The file descriptor is correctly
>> passed to the child and is dup'd to the stdin of the next process.
>>
>> However, the open file descriptor in the parent is not closed after it has
>> been used as the input for the next Process.
>> The fix is to close the fd after it has been used as the input of the next
>> process.
>>
>> A new test verifies that after `pipelineStart` is complete, the same file
>> descriptors are open for Unix Pipes as before the test.
>> The test only runs on Linux using the /proc/<pid>/fd filesystem to identify
>> open file descriptors.
>>
>> The bug fix is in `ProcessBuilder.pipelineStart` and is applicable to all
>> platforms.
>
> Roger Riggs has updated the pull request incrementally with one additional
> commit since the last revision:
>
> remove debug output
Hello Roger, I just noticed that the failures reported in the GitHub actions
job are genuine and related to this new test. 2 separate failures:
test
PipelineLeaksFD.checkForLeaks(java.util.ImmutableCollections$List12@73c167a8):
failure
java.lang.AssertionError: More or fewer pipes than expected
at org.testng.Assert.fail(Assert.java:99)
at PipelineLeaksFD.checkForLeaks(PipelineLeaksFD.java:94)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
at java.base/java.lang.reflect.Method.invoke(Method.java:578)
at
org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
at org.testng.internal.TestInvoker.invokeMethod(TestInvoker.java:599)
at
org.testng.internal.TestInvoker.invokeTestMethod(TestInvoker.java:174)
at org.testng.internal.MethodRunner.runInSequence(MethodRunner.java:46)
at
org.testng.internal.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:822)
at
org.testng.internal.TestInvoker.invokeTestMethods(TestInvoker.java:147)
at
org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:128)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at org.testng.TestRunner.privateRun(TestRunner.java:764)
at org.testng.TestRunner.run(TestRunner.java:585)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:384)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:378)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:337)
at org.testng.SuiteRunner.run(SuiteRunner.java:286)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:96)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1218)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1140)
at org.testng.TestNG.runSuites(TestNG.java:1069)
at org.testng.TestNG.run(TestNG.java:1037)
at
com.sun.javatest.regtest.agent.TestNGRunner.main(TestNGRunner.java:94)
at
com.sun.javatest.regtest.agent.TestNGRunner.main(TestNGRunner.java:54)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
at java.base/java.lang.reflect.Method.invoke(Method.java:578)
at
com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
at java.base/java.lang.Thread.run(Thread.java:1589)
and
test
PipelineLeaksFD.checkForLeaks(java.util.ImmutableCollections$ListN@13c9b949):
failure
java.io.UncheckedIOException: java.nio.file.NoSuchFileException:
/proc/18459/fd/20
at
java.base/java.nio.file.FileTreeIterator.fetchNextIfNeeded(FileTreeIterator.java:87)
at
java.base/java.nio.file.FileTreeIterator.hasNext(FileTreeIterator.java:103)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:132)
at
java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1921)
at
java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at
java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at
java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
at
java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
at PipelineLeaksFD.myPipes(PipelineLeaksFD.java:130)
at PipelineLeaksFD.checkForLeaks(PipelineLeaksFD.java:86)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
at java.base/java.lang.reflect.Method.invoke(Method.java:578)
at
org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
at org.testng.internal.TestInvoker.invokeMethod(TestInvoker.java:599)
at
org.testng.internal.TestInvoker.invokeTestMethod(TestInvoker.java:174)
at org.testng.internal.MethodRunner.runInSequence(MethodRunner.java:46)
at
org.testng.internal.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:822)
at
org.testng.internal.TestInvoker.invokeTestMethods(TestInvoker.java:147)
at
org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:128)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at org.testng.TestRunner.privateRun(TestRunner.java:764)
at org.testng.TestRunner.run(TestRunner.java:585)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:384)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:378)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:337)
at org.testng.SuiteRunner.run(SuiteRunner.java:286)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:96)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1218)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1140)
at org.testng.TestNG.runSuites(TestNG.java:1069)
at org.testng.TestNG.run(TestNG.java:1037)
at
com.sun.javatest.regtest.agent.TestNGRunner.main(TestNGRunner.java:94)
at
com.sun.javatest.regtest.agent.TestNGRunner.main(TestNGRunner.java:54)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
at java.base/java.lang.reflect.Method.invoke(Method.java:578)
at
com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
at java.base/java.lang.Thread.run(Thread.java:1589)
Caused by: java.nio.file.NoSuchFileException: /proc/18459/fd/20
at
java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
at
java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
at
java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at
java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
at
java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:148)
at
java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
at java.base/java.nio.file.Files.readAttributes(Files.java:1848)
at
java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:220)
at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:277)
at java.base/java.nio.file.FileTreeWalker.next(FileTreeWalker.java:374)
at
java.base/java.nio.file.FileTreeIterator.fetchNextIfNeeded(FileTreeIterator.java:83)
... 39 more
-------------
PR: https://git.openjdk.org/jdk/pull/9414
