[
https://issues.apache.org/jira/browse/CAMEL-12830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laurent Chiarello updated CAMEL-12830:
--------------------------------------
Description:
In our production systems, we had several threads stuck indefinitely while
trying to send a file to an FtpEndpoint. We had both _connectTimeout_ and
_soTimeout_ properties set so it surprised us a little bit.
After digging a bit, we found that the scenario is quite simple to reproduce:
this happens every time the {{FTPClient}} establishes the TCP connection with a
server that does not respond anything.
Here is a simplified view of what happens when establishing a connection using
a {{FTPClient}}:
{code:java}
// within Socket Client
public void connect(InetAddress host, int port) throws
SocketException,IOException {
_socket_.connect(new InetSocketAddress(host, port), connectTimeout);
_connectAction_();
}
protected void _connectAction_() throws IOException {
_socket_.setSoTimeout(_timeout_); // _timeout_ is the default timeout of
the socket
}
// overridden within FTP
protected void _connectAction_() {
super._connectAction_();
if (connectTimeout > 0) {
int original = _socket_.getSoTimeout();
_socket_.setSoTimeout(connectTimeout);
try {
__getReply();
} finally {
_socket_.setSoTimeout(original);
}
}{code}
A {{SocketTimeoutException}} can be thrown either during the initial socket
_connect_ action, either during the ___getReply()_ where the FTPClient waits
for the hello message from the server. Both are using _connectTimeout_, after
which the original (default) timeout is restored. The _soTimeout_ we specified
in the URI is configured by FTPOperations only when the connection is
successful. In this case, the Socket is connected, but an exception is thrown
afterwards and the _soTimeout_ is left at 0.
Within Camel, when the RemoteFileProducer encounters an exception while
processing an Exchange, it tries to disconnect the endpoint properly with a
_logout_ followed by a _disconnect_.
{code:java}
// RemoteFileProducer
public void handleFailedWrite(Exchange exchange, Exception exception) throws
Exception {
try {
if (getOperations().isConnected()) { // <== in our case, this returns
true because the socket is actually connected
getOperations().disconnect();
}
} catch (...) {
...
}
}
// FTPOperations
protected void doDisconnect() throws GenericFileOperationFailedException {
try {
client.logout();
} catch (IOException e) {
throw new GenericFileOperationFailedException
} finally {
try {
client.disconnect();
} catch (IOException e) {
throw new GenericFileOperationFailedException
}
}
}{code}
Unfortunately, at this point, the {{client.logout()}} sends the FTP {{QUIT}}
command, then waits for the response still using the default timeout of the
Socket. Since the misbehaving server/firewall never sent any form of response,
the thread is left waiting forever.
I attached a simple test case to illustrate the scenario, simply using a
ServerSocket that never accepts any connection. Also included is an easy
workaround that uses a custom FTPClient on which the _default timeout_ is set.
A possible fix would be to always set the _default timeout_ on the socket,
before connecting it.
was:
In our production systems, we had several threads stuck indefinitely while
trying to send a file to an FtpEndpoint. We had both _connectTimeout_ and
_soTimeout_ properties set so it surprised us a little bit.
After digging a bit, we found that the scenario is quite simple to reproduce:
this happens every time the {{FTPClient}} establishes the TCP connection with a
server that does not respond anything.
Here is a simplified view of what happens when establishing a connection using
a {{FTPClient}}:
{code:java}
// within Socket Client
public void connect(InetAddress host, int port) throws
SocketException,IOException {
_socket_.connect(new InetSocketAddress(host, port), connectTimeout);
_connectAction_();
}
protected void _connectAction_() throws IOException {
_socket_.setSoTimeout(_timeout_); // _timeout_ is the default timeout of
the socket
}
// overridden within FTP
protected void _connectAction_() {
super._connectAction_();
if (connectTimeout > 0) {
int original = _socket_.getSoTimeout();
_socket_.setSoTimeout(connectTimeout);
try {
__getReply();
} finally {
_socket_.setSoTimeout(original);
}
}{code}
A {{SocketTimeoutException}} can be thrown either during the initial socket
_connect_ action, either during the __getReply() where the FTPClient waits for
the hello message from the server. Both are using _connectTimeout_, after
which the original (default) timeout is restored. The _soTimeout_ we specified
in the URI is configured by FTPOperations only when the connection is
successful. In this case, the Socket is connected, but an exception is thrown
afterwards and the _soTimeout_ is left at 0.
Within Camel, when the RemoteFileProducer encounters an exception while
processing an Exchange, it tries to disconnect the endpoint properly with a
_logout_ followed by a _disconnect_.
{code:java}
// RemoteFileProducer
public void handleFailedWrite(Exchange exchange, Exception exception) throws
Exception {
try {
if (getOperations().isConnected()) { // <== in our case, this returns
true because the socket is actually connected
getOperations().disconnect();
}
} catch (...) {
...
}
}
// FTPOperations
protected void doDisconnect() throws GenericFileOperationFailedException {
try {
client.logout();
} catch (IOException e) {
throw new GenericFileOperationFailedException
} finally {
try {
client.disconnect();
} catch (IOException e) {
throw new GenericFileOperationFailedException
}
}
}{code}
Unfortunately, at this point, the {{client.logout()}} sends the FTP {{QUIT}}
command, then waits for the response still using the default timeout of the
Socket. Since the misbehaving server/firewall never sent any form of response,
the thread is left waiting forever.
I attached a simple test case to illustrate the scenario, simply using a
ServerSocket that never accepts any connection. Also included is an easy
workaround that uses a custom FTPClient on which the _default timeout_ is set.
A possible fix would be to always set the _default timeout_ on the socket,
before connecting it.
> FTP producer stuck if timeout occurs just after connect
> -------------------------------------------------------
>
> Key: CAMEL-12830
> URL: https://issues.apache.org/jira/browse/CAMEL-12830
> Project: Camel
> Issue Type: Bug
> Components: camel-ftp
> Affects Versions: 2.22.1
> Reporter: Laurent Chiarello
> Priority: Major
> Attachments: FtpSoTimeoutTest.java
>
>
> In our production systems, we had several threads stuck indefinitely while
> trying to send a file to an FtpEndpoint. We had both _connectTimeout_ and
> _soTimeout_ properties set so it surprised us a little bit.
> After digging a bit, we found that the scenario is quite simple to reproduce:
> this happens every time the {{FTPClient}} establishes the TCP connection with
> a server that does not respond anything.
> Here is a simplified view of what happens when establishing a connection
> using a {{FTPClient}}:
> {code:java}
> // within Socket Client
> public void connect(InetAddress host, int port) throws
> SocketException,IOException {
> _socket_.connect(new InetSocketAddress(host, port), connectTimeout);
> _connectAction_();
> }
> protected void _connectAction_() throws IOException {
> _socket_.setSoTimeout(_timeout_); // _timeout_ is the default timeout of
> the socket
> }
> // overridden within FTP
> protected void _connectAction_() {
> super._connectAction_();
> if (connectTimeout > 0) {
> int original = _socket_.getSoTimeout();
> _socket_.setSoTimeout(connectTimeout);
> try {
> __getReply();
> } finally {
> _socket_.setSoTimeout(original);
> }
> }{code}
> A {{SocketTimeoutException}} can be thrown either during the initial socket
> _connect_ action, either during the ___getReply()_ where the FTPClient waits
> for the hello message from the server. Both are using _connectTimeout_,
> after which the original (default) timeout is restored. The _soTimeout_ we
> specified in the URI is configured by FTPOperations only when the connection
> is successful. In this case, the Socket is connected, but an exception is
> thrown afterwards and the _soTimeout_ is left at 0.
> Within Camel, when the RemoteFileProducer encounters an exception while
> processing an Exchange, it tries to disconnect the endpoint properly with a
> _logout_ followed by a _disconnect_.
> {code:java}
> // RemoteFileProducer
> public void handleFailedWrite(Exchange exchange, Exception exception) throws
> Exception {
> try {
> if (getOperations().isConnected()) { // <== in our case, this returns
> true because the socket is actually connected
> getOperations().disconnect();
> }
> } catch (...) {
> ...
> }
> }
> // FTPOperations
> protected void doDisconnect() throws GenericFileOperationFailedException {
> try {
> client.logout();
> } catch (IOException e) {
> throw new GenericFileOperationFailedException
> } finally {
> try {
> client.disconnect();
> } catch (IOException e) {
> throw new GenericFileOperationFailedException
> }
> }
> }{code}
> Unfortunately, at this point, the {{client.logout()}} sends the FTP {{QUIT}}
> command, then waits for the response still using the default timeout of the
> Socket. Since the misbehaving server/firewall never sent any form of
> response, the thread is left waiting forever.
> I attached a simple test case to illustrate the scenario, simply using a
> ServerSocket that never accepts any connection. Also included is an easy
> workaround that uses a custom FTPClient on which the _default timeout_ is
> set.
> A possible fix would be to always set the _default timeout_ on the socket,
> before connecting it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
