Re: Testing LISTEN/NOTIFY more effectively

2019-07-28 Thread Tom Lane
Andres Freund writes: > On 2019-07-27 12:46:51 -0400, Tom Lane wrote: >> 2. Change psql so that there's a way to get NOTIFY messages without >> the sending PID. For testing purposes, it'd be sufficient to know >> whether the sending PID is our own backend's PID or not. This idea >> is not horrib

Re: Testing LISTEN/NOTIFY more effectively

2019-07-27 Thread Tom Lane
Andres Freund writes: > I wonder if there aren't similar dangers around the notify handling. In > your patch we don't print them particularly eagerly. Doesn't that also > open us up to timing concerns? I think probably not, because of the backend-side restrictions on when notify messages will be

Re: Testing LISTEN/NOTIFY more effectively

2019-07-27 Thread Andres Freund
Hi, On 2019-07-27 20:02:13 -0400, Tom Lane wrote: > Andres Freund writes: > I'm slightly more worried about the case of more than one bufferful > of NOTICE messages: calling PQconsumeInput isn't entirely guaranteed to > absorb *all* available input. But for the cases we actually need to > deal w

Re: Testing LISTEN/NOTIFY more effectively

2019-07-27 Thread Tom Lane
Andres Freund writes: > Polling for notices on the blocked connection before printing anything > ought to practically be reliable. Theoretically I think it still allows > for some reordering, e.g. because there was packet loss on one, but not > the other connection. As long as it's a local connec

Re: Testing LISTEN/NOTIFY more effectively

2019-07-27 Thread Andres Freund
Hi, On 2019-07-27 19:27:17 -0400, Tom Lane wrote: > Andres Freund writes: > > We could of course just send the pids in binary ;). No, not worth it > > just to avoid a small redundant array ;) > > IIRC, we'd have to do htonl on them, so we'd still end up with > two representations ... Yea. Altho

Re: Testing LISTEN/NOTIFY more effectively

2019-07-27 Thread Tom Lane
I wrote: > Andres Freund writes: >> Hm. I wonder if all that's happening with prairedog is that the notice >> is sent a bit later. I think that could e.g. conceivably happen because >> it TCP_NODELAY isn't supported on prariedog? Or just because the machine >> is very slow? > The notices (not not

Re: Testing LISTEN/NOTIFY more effectively

2019-07-27 Thread Tom Lane
Andres Freund writes: > We could of course just send the pids in binary ;). No, not worth it > just to avoid a small redundant array ;) IIRC, we'd have to do htonl on them, so we'd still end up with two representations ... > Hm. I wonder if all that's happening with prairedog is that the notice

Re: Testing LISTEN/NOTIFY more effectively

2019-07-27 Thread Andres Freund
Hi, On 2019-07-27 18:20:52 -0400, Tom Lane wrote: > diff --git a/src/test/isolation/isolationtester.c > b/src/test/isolation/isolationtester.c > index 6ab19b1..98e5bf2 100644 > --- a/src/test/isolation/isolationtester.c > +++ b/src/test/isolation/isolationtester.c > @@ -23,10 +23,12 @@ > > /*

Re: Testing LISTEN/NOTIFY more effectively

2019-07-27 Thread Tom Lane
Andres Freund writes: > On 2019-07-27 15:39:44 -0400, Tom Lane wrote: >> Unfortunately, I just found out that on a slow enough machine >> (prairiedog's host) there *is* some variation in when that test's >> notices come out. I am unsure whether that's to be expected or >> whether there's somethin

Re: Testing LISTEN/NOTIFY more effectively

2019-07-27 Thread Tom Lane
I wrote: > Andres Freund writes: >> Perhaps we could just have isolationtester check to which >> isolationtester session the backend pid belongs? And then print the >> session name instead of the pid? That should be fairly easy, and would >> probably give us all we need? > Oh, that's a good idea

Re: Testing LISTEN/NOTIFY more effectively

2019-07-27 Thread Peter Geoghegan
On Sat, Jul 27, 2019 at 12:39 PM Tom Lane wrote: > Unfortunately, I just found out that on a slow enough machine > (prairiedog's host) there *is* some variation in when that test's > notices come out. I am unsure whether that's to be expected or > whether there's something wrong there --- Peter,

Re: Testing LISTEN/NOTIFY more effectively

2019-07-27 Thread Andres Freund
Hi, On 2019-07-27 15:39:44 -0400, Tom Lane wrote: > Andres Freund writes: > >> diff --git a/src/test/isolation/expected/insert-conflict-specconflict.out > >> b/src/test/isolation/expected/insert-conflict-specconflict.out > >> index 5726bdb..20cc421 100644 > >> --- a/src/test/isolation/expected/i

Re: Testing LISTEN/NOTIFY more effectively

2019-07-27 Thread Tom Lane
Andres Freund writes: > On 2019-07-27 14:15:39 -0400, Tom Lane wrote: >> So I think we should apply something like the attached, and if the >> buildfarm shows any instability as a result, dealing with that by >> taking out the RAISE NOTICE commands. > +1 >> diff --git a/src/test/isolation/expect

Re: Testing LISTEN/NOTIFY more effectively

2019-07-27 Thread Andres Freund
Hi, On 2019-07-27 14:15:39 -0400, Tom Lane wrote: > So I think we should apply something like the attached, and if the > buildfarm shows any instability as a result, dealing with that by > taking out the RAISE NOTICE commands. +1 > diff --git a/src/test/isolation/expected/insert-conflict-speccon

Re: Testing LISTEN/NOTIFY more effectively

2019-07-27 Thread Tom Lane
While I'm looking at isolationtester ... my eye was immediately drawn to this bit, because it claims to be dealing with NOTIFY messages --- though that's wrong, it's really blocking NOTICE messages: /* * Suppress NOTIFY messages, which otherwise pop into results at odd *

Re: Testing LISTEN/NOTIFY more effectively

2019-07-27 Thread Tom Lane
Andres Freund writes: > On 2019-07-27 12:46:51 -0400, Tom Lane wrote: >> I'm finding alternative #3 the most attractive, because we really >> want isolation-style testing for LISTEN/NOTIFY, and this solution >> doesn't require designing a psql feature that we'd need to get >> consensus on. > Perh

Re: Testing LISTEN/NOTIFY more effectively

2019-07-27 Thread Andres Freund
Hi, On 2019-07-27 12:46:51 -0400, Tom Lane wrote: > So, if we'd like to have more thorough NOTIFY coverage without going > to that much work, what to do? I thought of a few alternatives: > > 1. Write a TAP test instead of using the old test frameworks, and > use regexps to check the expected out

Testing LISTEN/NOTIFY more effectively

2019-07-27 Thread Tom Lane
Since we have three or four different NOTIFY improvement proposals floating around in the current CF, I got a bit distressed at the lack of test coverage for that functionality. While the code coverage report makes it look like commands/async.c isn't so badly covered, that's all coming from src/te