Branch: refs/heads/cli-hang
Home: https://github.com/jenkinsci/remoting
Commit: d6bacfda1a836696adf9a3c1a24fb1f6349b9841
https://github.com/jenkinsci/remoting/commit/d6bacfda1a836696adf9a3c1a24fb1f6349b9841
Author: Kohsuke Kawaguchi <[email protected]>
Date: 2015-03-27 (Fri, 27 Mar 2015)
Changed paths:
M src/main/java/hudson/remoting/Channel.java
M src/main/java/hudson/remoting/ExportTable.java
Log Message:
-----------
When a channel goes down, kill off all Pipes.
I came across a hanging CLI client that was at the following point:
"main" #1 prio=5 os_prio=0 tid=0x00c2b800 nid=0xf60 in Object.wait()
[0x00bbf000 ]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at hudson.remoting.FastPipedInputStream.read(FastPipedInputStream.java:175)
- locked <0x03656f68> (a [B)
at hudson.remoting.FastPipedInputStream.read(FastPipedInputStream.java:146)
at java.io.DataInputStream.readBoolean(Unknown Source)
at hudson.cli.Connection.readBoolean(Connection.java:93)
at hudson.cli.CLI.authenticate(CLI.java:560)
at hudson.cli.CLI._main(CLI.java:471)
at hudson.cli.CLI.main(CLI.java:382)
The thread is stuck waiting for data to arrive from FastPipedOuputStream, which
is exported and in turn waiting for the remote end to send more data. However,
there's no Channel ReaderThread, and indeed the channel has already failed:
Mar 08, 2015 11:10:50 PM
hudson.remoting.SynchronousCommandTransport$ReaderThread run
SEVERE: I/O error in channel CLI connection to
https://cloudbees.ci.cloudbees.com/
java.io.IOException: Unexpected termination of the channel
at
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
Caused by: java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(Unknown Source)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(Unknown Source)
at java.io.ObjectInputStream.readStreamHeader(Unknown Source)
at java.io.ObjectInputStream.<init>(Unknown Source)
at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:40)
at
hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
at
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)
The problem here is that the exported object (FastPipedOutputStream) doesn't
notice that the actual writer (the other side of the channel) has disappeared,
hence the reader blocks forever. Had FastPipedOutputStream got GC-ed, its
finalizer had closed the stream and allowed the reader to receive an exception,
but Channel was kept in the stack frame of the reader thread, so it is getting
strongly referenced.
In this commit, I'm fixing this problem twice.
First, the change allows exported objects to notice a failure through
ErrorPropagatingOutputStream, which FastPipedOutputStream and Pipe aleady
implements. It seems reasonable for a pipe to be disrupted if the channel it's
on is disrupted, and this would allow the "main" thread above to fail.
Second, the change clears strong referenced to exported objects, allowing them
to get GCed. FastPipedOutputStream would get GCed and it would have closed the
writer end, resulting in the reader end getting EOF.
--
You received this message because you are subscribed to the Google Groups
"Jenkins Commits" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
