Re: libpq copy error handling busted

2020-06-06 Thread Tom Lane
Andres Freund writes: > On 2020-06-03 18:41:28 -0400, Tom Lane wrote: >> * pqSendSome() is responsible not only for pushing out data, but for >> calling pqReadData in any situation where it can't get rid of the data >> promptly. 1f39a1c06 overlooked that requirement, and the upshot is >> that we

Re: libpq copy error handling busted

2020-06-05 Thread Andres Freund
Hi, On 2020-06-03 18:41:28 -0400, Tom Lane wrote: > Andres Freund writes: > > When libpq is used to COPY data to the server, it doesn't properly > > handle errors. > > This is partially an old problem, and partially got recently > > worse. Before the below commit we detected terminated connection

Re: libpq copy error handling busted

2020-06-04 Thread Tom Lane
I wrote: > * As for control-C not getting out of it: there is > if (CancelRequested) > break; > in pgbench's loop, but this does nothing in this scenario because > fe-utils/cancel.c only sets that flag when it successfully sends a > Cancel ... which it certainly

Re: libpq copy error handling busted

2020-06-04 Thread Thomas Munro
On Thu, Jun 4, 2020 at 6:22 PM Oleksandr Shulgin wrote: > On Thu, Jun 4, 2020 at 5:37 AM Thomas Munro wrote: >> Here's what I tested. First, I put this into pgdata/postgresql.conf: > Would it be feasible to capture this in a sort of a regression (TAP?) test? If I'm remembering correctly, it wo

Re: libpq copy error handling busted

2020-06-03 Thread Oleksandr Shulgin
On Thu, Jun 4, 2020 at 5:37 AM Thomas Munro wrote: > On Thu, Jun 4, 2020 at 1:53 PM Thomas Munro > wrote: > > On Thu, Jun 4, 2020 at 1:35 PM Tom Lane wrote: > > > Ah, it's better if I put the pqReadData call into *both* the paths > > > where 1f39a1c06 made pqSendSome give up. The attached patc

Re: libpq copy error handling busted

2020-06-03 Thread Thomas Munro
On Thu, Jun 4, 2020 at 3:36 PM Thomas Munro wrote: > Here's what I tested. In passing, I noticed that this: $ psql ... psql: error: could not connect to server: private key file "src/test/ssl/ssl/client-revoked.key" has group or world access; permissions should be u=rw (0600) or less ... leads

Re: libpq copy error handling busted

2020-06-03 Thread Thomas Munro
On Thu, Jun 4, 2020 at 1:53 PM Thomas Munro wrote: > On Thu, Jun 4, 2020 at 1:35 PM Tom Lane wrote: > > Ah, it's better if I put the pqReadData call into *both* the paths > > where 1f39a1c06 made pqSendSome give up. The attached patch seems > > to fix the issue for the "pgbench -i" scenario, wit

Re: libpq copy error handling busted

2020-06-03 Thread Thomas Munro
On Thu, Jun 4, 2020 at 1:35 PM Tom Lane wrote: > I wrote: > > * pqSendSome() is responsible not only for pushing out data, but for > > calling pqReadData in any situation where it can't get rid of the data > > promptly. 1f39a1c06 overlooked that requirement, and the upshot is > > that we don't ne

Re: libpq copy error handling busted

2020-06-03 Thread Tom Lane
I wrote: > * pqSendSome() is responsible not only for pushing out data, but for > calling pqReadData in any situation where it can't get rid of the data > promptly. 1f39a1c06 overlooked that requirement, and the upshot is > that we don't necessarily notice that the connection is broken (it's > pqR

Re: libpq copy error handling busted

2020-06-03 Thread Tom Lane
Andres Freund writes: > When libpq is used to COPY data to the server, it doesn't properly > handle errors. > This is partially an old problem, and partially got recently > worse. Before the below commit we detected terminated connections, but > we didn't handle copy failing. Yeah. After poking

libpq copy error handling busted

2020-06-03 Thread Andres Freund
Hi, When libpq is used to COPY data to the server, it doesn't properly handle errors. An easy way to trigger the problem is to start pgbench -i with a sufficiently large scale, and then just shut the server down. pgbench will happily use 100% of the cpu trying to send data to the server, even tho