Curl under cygwin - TCP keepalive I need to download a file from a remote server via a https request, and the remote server takes several minutes to prepare the file.
Without any keepalives, the remote end will terminate the socket, but it doesn't close it (which I know is weird). Running curl with --keepalive 45 on a linux centos7 machine, enables the transfer to work. With tcpdump, I see the nop packet every 45 seconds. But on cygwin, the --keepalive parameter doesn't seem to have effect. Running wireshark, I don't see any keepalives. Running the request in a browser, chrome or firefox, works fine, and I can see keepalives in wireshark, so clearly they are either setting the socket options correctly, or they are sending their own nop. Running on a vm, vmware workstation / centos8 , on the same machine where I am running curl, I can see the keepalives. I have downloaded and rebuilt curl and that did not resolve the issue. Can someone shed some light on this? On Sat, Sep 7, 2019 at 9:39 PM ALbert via cygwin <cygwin@cygwin.com> wrote: > I upgraded to the latest version curl 7.65.3 with libcurl 7.65.3, same > issue. > curl 7.65.3 (i686-pc-cygwin) libcurl/7.65.3 OpenSSL/1.1.1c zlib/1.2.11 > brotli/1.0.7 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.0.4) > libssh/0.8.7/openssl/zlib nghttp2/1.37.0 > Thanks. > > On Friday, September 6, 2019, 3:44:13 PM EDT, ALbert < > albert...@yahoo.com> wrote: > > We recently upgraded cURL from version:curl 7.50.3 (i686-pc-cygwin) > libcurl/7.50.3 OpenSSL/1.0.2j zlib/1.2.8 libidn/1.29 libpsl/0.14.0 > (+libidn/1.29) libssh2/1.7.0 nghttp2/1.14.0tocurl 7.64.1 (i686-pc-cygwin) > libcurl/7.65.3 OpenSSL/1.1.1c zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 > libpsl/0.21.0 (+libidn2/2.0.4) libssh/0.8.7/openssl/zlib nghttp2/1.37.0 > Since the upgrade, we've seen problems uploading text files to sftp > servers. Here's the call that we make: > curl--trace-ascii trace.txt -k sftp://<user>:<pass>@<server>/file.txt-T > file.txt > > On initial call, a 0-length file is uploaded, with successful return code. > On immediate rerun, a full file is uploaded (if we wait few minutes, > 0-length file will get uploaded). Below are the traces from initial and > subsequent calls. We also tried using same version curl on a linux box, > without any issues. We also downloaded latest version of curl from curl for > Windows, "curl 7.65.3 (i386-pc-win32) libcurl/7.65.3 OpenSSL/1.1.1c > (Schannel) zlib/1.2.11 brotli/1.0.7 WinIDN libssh2/1.9.0 nghttp2/1.39.2", > and its also working correctly. > Unfortunately, the server we're uploading to belongs to a client, and we > can't see if this is an issue on a server side. However, even if it was, > why would linux curl and windows curl (from curl.haxx.se) would work > correctly? > Any help or pointers would be appreciated. Thank you. > > Here's the trace files: > From initial call:== Info: STATE: INIT => CONNECT handle 0x80068340; line > 1356 (connection #-5000) > == Info: Added connection 0. The cache now contains 1 members > == Info: STATE: CONNECT => WAITRESOLVE handle 0x80068340; line 1397 > (connection #0) > == Info: Trying XXX.XX.XX.XX...== Info: TCP_NODELAY set > == Info: STATE: WAITRESOLVE => WAITCONNECT handle 0x80068340; line 1476 > (connection #0) > == Info: Connected to XXXXXXX ( XXX.XX.XX.XX) port 22 (#0)== Info: STATE: > WAITCONNECT => SENDPROTOCONNECT handle 0x80068340; line 1532 (connection #0) > == Info: Marked for [keep alive]: SSH default > == Info: User: Opus_efg > == Info: SSH 0x80048f90 state change from SSH_STOP to SSH_INIT (line 2115) > == Info: SSH 0x80048f90 state change from SSH_INIT to SSH_S_STARTUP (line > 582) > == Info: STATE: SENDPROTOCONNECT => PROTOCONNECT handle 0x80068340; line > 1547 (connection #0) > == Info: SSH 0x80048f90 state change from SSH_S_STARTUP to SSH_HOSTKEY > (line 595) > == Info: SSH 0x80048f90 state change from SSH_HOSTKEY to SSH_AUTHLIST > (line 605) > == Info: SSH 0x80048f90 state change from SSH_AUTHLIST to > SSH_AUTH_PKEY_INIT (line 628) > == Info: Authentication using SSH public key file > == Info: SSH 0x80048f90 state change from SSH_AUTH_PKEY_INIT to > SSH_AUTH_KEY_INIT (line 697) > == Info: SSH 0x80048f90 state change from SSH_AUTH_KEY_INIT to > SSH_AUTH_KEY (line 743) > == Info: completed keyboard interactive authentication > == Info: SSH 0x80048f90 state change from SSH_AUTH_KEY to SSH_AUTH_DONE > (line 761) > == Info: Authentication complete > == Info: SSH 0x80048f90 state change from SSH_AUTH_DONE to SSH_SFTP_INIT > (line 807) > == Info: SSH 0x80048f90 state change from SSH_SFTP_INIT to > SSH_SFTP_REALPATH (line 833) > == Info: SSH CONNECT phase done > == Info: SSH 0x80048f90 state change from SSH_SFTP_REALPATH to SSH_STOP > (line 850) > == Info: STATE: PROTOCONNECT => DO handle 0x80068340; line 1566 > (connection #0) > == Info: DO phase starts > == Info: SSH 0x80048f90 state change from SSH_STOP to SSH_SFTP_QUOTE_INIT > (line 2332) > == Info: SSH 0x80048f90 state change from SSH_SFTP_QUOTE_INIT to > SSH_SFTP_GETINFO (line 868) > == Info: SSH 0x80048f90 state change from SSH_SFTP_GETINFO to > SSH_SFTP_TRANS_INIT (line 1060) > == Info: SSH 0x80048f90 state change from SSH_SFTP_TRANS_INIT to > SSH_SFTP_UPLOAD_INIT (line 1080) > == Info: SSH 0x80048f90 state change from SSH_SFTP_UPLOAD_INIT to > SSH_SFTP_CLOSE (line 1143) > == Info: STATE: DO => DOING handle 0x80068340; line 1608 (connection #0) > == Info: SFTP DONE done > == Info: SSH 0x80048f90 state change from SSH_SFTP_CLOSE to SSH_STOP (line > 1651) > == Info: DO phase is complete > == Info: STATE: DOING => DO_DONE handle 0x80068340; line 1693 (connection > #0) > == Info: STATE: DO_DONE => PERFORM handle 0x80068340; line 1743 > (connection #0) > == Info: STATE: PERFORM => DONE handle 0x80068340; line 1933 (connection > #0) > == Info: multi_done > == Info: SSH 0x80048f90 state change from SSH_STOP to SSH_SFTP_CLOSE (line > 2390) > == Info: SFTP DONE done > == Info: SSH 0x80048f90 state change from SSH_SFTP_CLOSE to SSH_STOP (line > 1651) > == Info: Connection #0 to host XXXXXXX left intact > > > Trace from subsequent call:== Info: STATE: INIT => CONNECT handle > 0x80068340; line 1356 (connection #-5000) > == Info: Added connection 0. The cache now contains 1 members > == Info: STATE: CONNECT => WAITRESOLVE handle 0x80068340; line 1397 > (connection #0) > == Info: Trying XXX.XX.XX.XX...== Info: TCP_NODELAY set > == Info: STATE: WAITRESOLVE => WAITCONNECT handle 0x80068340; line 1476 > (connection #0) > == Info: Connected to XXXXXXX (XXX.XX.XX.XX) port 22 (#0) > == Info: STATE: WAITCONNECT => SENDPROTOCONNECT handle 0x80068340; line > 1532 (connection #0) > == Info: Marked for [keep alive]: SSH default > == Info: User: Opus_efg > == Info: SSH 0x80048f90 state change from SSH_STOP to SSH_INIT (line 2115) > == Info: SSH 0x80048f90 state change from SSH_INIT to SSH_S_STARTUP (line > 582) > == Info: STATE: SENDPROTOCONNECT => PROTOCONNECT handle 0x80068340; line > 1547 (connection #0) > == Info: SSH 0x80048f90 state change from SSH_S_STARTUP to SSH_HOSTKEY > (line 595) > == Info: SSH 0x80048f90 state change from SSH_HOSTKEY to SSH_AUTHLIST > (line 605) > == Info: SSH 0x80048f90 state change from SSH_AUTHLIST to > SSH_AUTH_PKEY_INIT (line 628) > == Info: Authentication using SSH public key file > == Info: SSH 0x80048f90 state change from SSH_AUTH_PKEY_INIT to > SSH_AUTH_KEY_INIT (line 697) > == Info: SSH 0x80048f90 state change from SSH_AUTH_KEY_INIT to > SSH_AUTH_KEY (line 743) > == Info: completed keyboard interactive authentication > == Info: SSH 0x80048f90 state change from SSH_AUTH_KEY to SSH_AUTH_DONE > (line 761) > == Info: Authentication complete > == Info: SSH 0x80048f90 state change from SSH_AUTH_DONE to SSH_SFTP_INIT > (line 807) > == Info: SSH 0x80048f90 state change from SSH_SFTP_INIT to > SSH_SFTP_REALPATH (line 833) > == Info: SSH CONNECT phase done > == Info: SSH 0x80048f90 state change from SSH_SFTP_REALPATH to SSH_STOP > (line 850) > == Info: STATE: PROTOCONNECT => DO handle 0x80068340; line 1566 > (connection #0) > == Info: DO phase starts > == Info: SSH 0x80048f90 state change from SSH_STOP to SSH_SFTP_QUOTE_INIT > (line 2332) > == Info: SSH 0x80048f90 state change from SSH_SFTP_QUOTE_INIT to > SSH_SFTP_GETINFO (line 868) > == Info: SSH 0x80048f90 state change from SSH_SFTP_GETINFO to > SSH_SFTP_TRANS_INIT (line 1060) > == Info: SSH 0x80048f90 state change from SSH_SFTP_TRANS_INIT to > SSH_SFTP_UPLOAD_INIT (line 1080) > == Info: SSH 0x80048f90 state change from SSH_SFTP_UPLOAD_INIT to SSH_STOP > (line 1222) > == Info: DO phase is complete > == Info: STATE: DO => DO_DONE handle 0x80068340; line 1621 (connection #0) > == Info: STATE: DO_DONE => PERFORM handle 0x80068340; line 1743 > (connection #0) > => Send data, 10688 bytes (0x29c0) > SENSITIVE DATA DELETED > == Info: We are completely uploaded and fine > == Info: STATE: PERFORM => DONE handle 0x80068340; line 1933 (connection > #0) > == Info: multi_done > == Info: SSH 0x80048f90 state change from SSH_STOP to SSH_SFTP_CLOSE (line > 2390) > == Info: SFTP DONE done > == Info: SSH 0x80048f90 state change from SSH_SFTP_CLOSE to SSH_STOP (line > 1651) > == Info: Connection #0 to host XXXXXXX left intact > == Info: Expire cleared (transfer 0x80068340) > > > > > > > > > -- > Problem reports: http://cygwin.com/problems.html > FAQ: http://cygwin.com/faq/ > Documentation: http://cygwin.com/docs.html > Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple > > -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation: https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
