Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-25 Thread Martin Pihlak
On 07/24/2011 11:33 PM, Tom Lane wrote: > I've applied the simplified fix (just set SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER) > as well as a patch to improve the error reporting situation. > Cool that this turned out to be a one-line fix. Thanks! regards, Martin -- Sent via pgsql-hackers mailing li

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-24 Thread Andrew Dunstan
On 07/24/2011 07:49 PM, Peter Geoghegan wrote: On 24 July 2011 21:33, Tom Lane wrote: I've applied the simplified fix (just set SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER) as well as a patch to improve the error reporting situation. I'm not exactly sure why, and don't have time to investigate right

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-24 Thread Peter Geoghegan
On 24 July 2011 21:33, Tom Lane wrote: > I've applied the simplified fix (just set SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER) > as well as a patch to improve the error reporting situation. I'm not exactly sure why, and don't have time to investigate right now, but this commit (fee476da952a1f02f7ccf6e23

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-24 Thread Tom Lane
Martin Pihlak writes: > If possible I would like the fix to be backported as well. This is > quite a nasty bug and difficult to track down. Especially as the > actual SSL error messages are masked by the "server closed the > connection unexpectedly" message. I've applied the simplified fix (just

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-24 Thread Tom Lane
I wrote: > Still wondering about the SSL_read end of it, though. And on that front, some digging around in the OpenSSL source code indicates that they do all their work in internal buffers, and transfer data into SSL_read's result buffer only when ready to return it. So the claim in the documentat

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-24 Thread Tom Lane
I wrote: > So this does look like it'd fix the issue for SSL_write, without needing > to introduce a concept of a "write frozen" buffer state. I am wondering > though how far back support for this flag exists in OpenSSL, A bit of archaeology reveals that the flag was introduced in OpenSSL 0.9.4,

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-24 Thread Tom Lane
Martin Pihlak writes: > On 07/16/2011 12:46 AM, Tom Lane wrote: >> I think the direction to move in ought to be to use the existing buffer >> as-is, and have pqCheckOutBufferSpace refuse to enlarge the buffer while >> we are in "write frozen" state. It should be OK to append data to the >> buffer

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-24 Thread Martin Pihlak
On 07/16/2011 12:46 AM, Tom Lane wrote: > I think the direction to move in ought to be to use the existing buffer > as-is, and have pqCheckOutBufferSpace refuse to enlarge the buffer while > we are in "write frozen" state. It should be OK to append data to the > buffer, though, so long as we remem

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-15 Thread Tom Lane
Robert Haas writes: > On Fri, Jul 8, 2011 at 9:36 AM, Martin Pihlak wrote: >> On 07/03/2011 05:08 AM, Steve Singer wrote: >>> Since the original patch was submitted as a WIP patch and this version >>> wasn't sent until well into the commit fest I am not sure if it >>> qualifies for a committer du

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-08 Thread Robert Haas
On Fri, Jul 8, 2011 at 9:36 AM, Martin Pihlak wrote: > On 07/03/2011 05:08 AM, Steve Singer wrote: >> Since the original patch was submitted as a WIP patch and this version >> wasn't sent until well into the commit fest I am not sure if it >> qualifies for a committer during this commitfest or if

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-08 Thread Martin Pihlak
On 07/03/2011 05:08 AM, Steve Singer wrote: > Since the original patch was submitted as a WIP patch and this version > wasn't sent until well into the commit fest I am not sure if it > qualifies for a committer during this commitfest or if it needs to wait > until the next one. > If possible I wo

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-02 Thread Steve Singer
On 11-06-28 02:14 PM, Martin Pihlak wrote: Thanks for the review! I have since simplified the patch to assume that partial SSL writes are disabled -- according to SSL_write(3) this is the default behaviour. Now the SSL retry buffer only holds the data to be retried, the remainder is moved to the

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-06-30 Thread Steve Singer
On 11-06-28 02:14 PM, Martin Pihlak wrote: Hmm, I thought I thought about that. There was a check in the original patch: "if (conn->sslRetryBytes || (conn->outCount - remaining)> 0)" So if the SSL retry buffer was emptied it would return 1 if there was something left in the regular output buffe

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-06-28 Thread Martin Pihlak
On 06/25/2011 12:14 AM, Steve Singer wrote: > I'm not a libpq guru but I've given your patch a look over. > Thanks for the review! I have since simplified the patch to assume that partial SSL writes are disabled -- according to SSL_write(3) this is the default behaviour. Now the SSL retry buffer

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-06-27 Thread Robert Haas
On Fri, Jun 24, 2011 at 5:14 PM, Steve Singer wrote: > A few things I noticed (that you might be aware of since you mentioned it > needs cleanup) > > -The patch doesn't compile with non-ssl builds,  the debug at the bottom of > PQSendSome isn't in an #ifdef > > -I don't think your handling the ret

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-06-24 Thread Steve Singer
On 11-06-15 03:20 PM, Martin Pihlak wrote: Yes, that sounds like a good idea -- especially considering that COPY is not the only operation that can cause SSL_write retries. This is of course still "work in progress", needs cleaning up and definitely more testing. But at this point before going

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-06-15 Thread Martin Pihlak
On 06/12/2011 04:22 AM, Robert Haas wrote: > One idea is that we could add outBuffer2/outBufSize2 to struct > pg_conn, or something along those lines with less obviously stupid > naming. Normally those would be unused, but in the special case where > SSL indicates that we must retry the call with

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-06-11 Thread Robert Haas
On Thu, Jun 9, 2011 at 10:39 AM, Martin Pihlak wrote: > In PQputCopyData #2 it is visible that the first SSL_write called from > pqFlush failed with SSL_ERROR_WANT_WRITE. The next SSL_write should > have been a retry with the same parameters, but instead was passed a > buffer with a different addr

[HACKERS] libpq SSL with non-blocking sockets

2011-06-09 Thread Martin Pihlak
I believe I have found a bug in libpq COPY handling with non-blocking SSL connections. The bug manifests itself by dropping the connection in PGputCopyData() with "server closed the connection unexpectedly" message. The connection drop only occurs with nonblocking connections - blocking connections