Re: Cleanup isolation specs from unused steps

2019-08-27 Thread Michael Paquier
On Tue, Aug 27, 2019 at 07:05:50PM +0530, Asim R P wrote: > Thank you for the feedback. I've changed patch 0002 accordingly, please > take another look: > https://www.postgresql.org/message-id/CANXE4TdvSi7Yia_5sV82%2BMHf0WcUSN9u6_X8VEUBv-YStphd%3DQ%40mail.gmail.com Thanks! Let's move the discuss

Re: Cleanup isolation specs from unused steps

2019-08-27 Thread Asim R P
On Fri, Aug 23, 2019 at 9:08 PM Alvaro Herrera wrote: > > On 2019-Aug-23, Asim R P wrote: > > > As part of the fault injector patch set [1], I added a new "blocking" > > keyword to isolation grammar so that a step can be declared as blocking. > > See patch 0002-Add-syntax-to-declare-a-step-that-is

Re: Cleanup isolation specs from unused steps

2019-08-23 Thread Michael Paquier
(Re-adding pgsql-hackers) On Fri, Aug 23, 2019 at 05:28:35PM +0530, Asim R P wrote: > Both the patches look good to me, thank you. +1 to removing dry-run and > tracking unused steps. Thanks, both have been committed. There have been issues with the isolation tests of logical decoding I have tak

Re: Cleanup isolation specs from unused steps

2019-08-23 Thread Alvaro Herrera
On 2019-Aug-23, Asim R P wrote: > As part of the fault injector patch set [1], I added a new "blocking" > keyword to isolation grammar so that a step can be declared as blocking. > See patch 0002-Add-syntax-to-declare-a-step-that-is-expected-to-block. One point to that implementation is that in t

Re: Cleanup isolation specs from unused steps

2019-08-23 Thread Asim R P
On Thu, Aug 22, 2019 at 12:47 AM Alvaro Herrera wrote: > > On 2019-Aug-21, Melanie Plageman wrote: > > > In Greenplum, we mainly add new tests to a separate isolation > > framework (called isolation2) which uses a completely different > > syntax. It doesn't use isolationtester at all. So, I haven'

Re: Cleanup isolation specs from unused steps

2019-08-22 Thread Michael Paquier
On Thu, Aug 22, 2019 at 09:19:47PM -0700, Melanie Plageman wrote: > It's a connection parameter that allows you to connect to a single Postgres > node in a Greenplum cluster. I only included it as an example of the kind of > "Greenplum-specific" things that are in the test framework. Ah, I see. I

Re: Cleanup isolation specs from unused steps

2019-08-22 Thread Melanie Plageman
On Thu, Aug 22, 2019 at 6:53 PM Michael Paquier wrote: > On Thu, Aug 22, 2019 at 10:20:48AM -0700, Melanie Plageman wrote: > > So, there is some historical context as to why it is a separate test > suite. > > And some of the differences are specific to Greenplum -- e.g. needing to > > connect to

Re: Cleanup isolation specs from unused steps

2019-08-22 Thread Michael Paquier
On Thu, Aug 22, 2019 at 10:20:48AM -0700, Melanie Plageman wrote: > So, there is some historical context as to why it is a separate test suite. > And some of the differences are specific to Greenplum -- e.g. needing to > connect to a specific database in "utility mode" to do something. What is "ut

Re: Cleanup isolation specs from unused steps

2019-08-22 Thread Robert Eckhardt
On Thu, Aug 22, 2019 at 1:45 PM Melanie Plageman wrote: > > > On Wed, Aug 21, 2019 at 12:16 PM Alvaro Herrera > wrote: >> >> On 2019-Aug-21, Melanie Plageman wrote: >> >> > In Greenplum, we mainly add new tests to a separate isolation >> > framework (called isolation2) which uses a completely di

Re: Cleanup isolation specs from unused steps

2019-08-22 Thread Melanie Plageman
On Wed, Aug 21, 2019 at 12:16 PM Alvaro Herrera wrote: > On 2019-Aug-21, Melanie Plageman wrote: > > > In Greenplum, we mainly add new tests to a separate isolation > > framework (called isolation2) which uses a completely different > > syntax. It doesn't use isolationtester at all. So, I haven't

Re: Cleanup isolation specs from unused steps

2019-08-21 Thread Michael Paquier
On Wed, Aug 21, 2019 at 11:07:19AM -0700, Melanie Plageman wrote: > So, I think I completely misunderstood the purpose of 'dry-run'. If no > one is using it, having a check for unused steps in dry-run may not be > useful. Okay. After sleeping on it and seeing how this thread evolves, it looks tha

Re: Cleanup isolation specs from unused steps

2019-08-21 Thread Alvaro Herrera
On 2019-Aug-21, Melanie Plageman wrote: > In Greenplum, we mainly add new tests to a separate isolation > framework (called isolation2) which uses a completely different > syntax. It doesn't use isolationtester at all. So, I haven't had a use > case to add long options to isolationtester yet :) I

Re: Cleanup isolation specs from unused steps

2019-08-21 Thread Melanie Plageman
On Tue, Aug 20, 2019 at 6:34 PM Michael Paquier wrote: > On Tue, Aug 20, 2019 at 09:54:56AM -0400, Alvaro Herrera wrote: > > On 2019-Aug-20, Tom Lane wrote: > >> If you can warn in both cases, that'd be OK perhaps. But Alvaro's > >> description of the intended use of dry-run makes it sound like

Re: Cleanup isolation specs from unused steps

2019-08-21 Thread Melanie Plageman
On Mon, Aug 19, 2019 at 7:01 PM Michael Paquier wrote: > > It is rather a pain to pass down custom options to isolationtester. > For example, I have tested the updated version attached after > hijacking -n into isolation_start_test(). Ugly hack, but for testing > that's enough. Do you make use

Re: Cleanup isolation specs from unused steps

2019-08-20 Thread Michael Paquier
On Tue, Aug 20, 2019 at 09:54:56AM -0400, Alvaro Herrera wrote: > On 2019-Aug-20, Tom Lane wrote: >> If you can warn in both cases, that'd be OK perhaps. But Alvaro's >> description of the intended use of dry-run makes it sound like >> it would be expected for there to be unreferenced steps, since

Re: Cleanup isolation specs from unused steps

2019-08-20 Thread Alvaro Herrera
On 2019-Aug-20, Tom Lane wrote: > If you can warn in both cases, that'd be OK perhaps. But Alvaro's > description of the intended use of dry-run makes it sound like > it would be expected for there to be unreferenced steps, since there'd > be no permutations yet in the input. Well, Heikki/Kevin'

Re: Cleanup isolation specs from unused steps

2019-08-20 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Aug-20, Michael Paquier wrote: >> On Mon, Aug 19, 2019 at 10:23:19AM -0700, Melanie Plageman wrote: >>> Could you do the check that all steps have been used in dry_run mode >>> instead of when running the tests for real? >> Sure, I was hesitating to do so. I have

Re: Cleanup isolation specs from unused steps

2019-08-19 Thread Michael Paquier
On Tue, Aug 20, 2019 at 12:34:45AM -0400, Alvaro Herrera wrote: > I created the dry-run mode to be able to easily generate the set of > possible permutations for a new test, then edit the result and put it > back in the spec file; but after the deadlock tests were added (with > necessary hacking of

Re: Cleanup isolation specs from unused steps

2019-08-19 Thread Alvaro Herrera
On 2019-Aug-20, Michael Paquier wrote: > On Mon, Aug 19, 2019 at 10:23:19AM -0700, Melanie Plageman wrote: > > Could you do the check that all steps have been used in dry_run mode > > instead of when running the tests for real? > > Sure, I was hesitating to do so. I have no issue in moving the c

Re: Cleanup isolation specs from unused steps

2019-08-19 Thread Michael Paquier
On Mon, Aug 19, 2019 at 10:23:19AM -0700, Melanie Plageman wrote: > Could you do the check that all steps have been used in dry_run mode > instead of when running the tests for real? Sure, I was hesitating to do so. I have no issue in moving the check into run_testspec(). So done as attached. I

Re: Cleanup isolation specs from unused steps

2019-08-19 Thread Michael Paquier
On Mon, Aug 19, 2019 at 11:02:42AM -0400, Tom Lane wrote: > Michael Paquier writes: >> I have been looking at the isolation tests, and we have in some specs >> steps which are defined but not used in any permutations. > > Hmm, might any of those represent actual bugs? Or are they just > leftover

Re: Cleanup isolation specs from unused steps

2019-08-19 Thread Melanie Plageman
On Mon, Aug 19, 2019 at 1:08 AM Michael Paquier wrote: > Hi all, > > I have been looking at the isolation tests, and we have in some specs > steps which are defined but not used in any permutations. In order to > detect them, I have been using the attached trick to track which > permutations are

Re: Cleanup isolation specs from unused steps

2019-08-19 Thread Tom Lane
Michael Paquier writes: > I have been looking at the isolation tests, and we have in some specs > steps which are defined but not used in any permutations. Hmm, might any of those represent actual bugs? Or are they just leftovers from test development? > In order to > detect them, I have been u

Cleanup isolation specs from unused steps

2019-08-19 Thread Michael Paquier
Hi all, I have been looking at the isolation tests, and we have in some specs steps which are defined but not used in any permutations. In order to detect them, I have been using the attached trick to track which permutations are used. This allows to find immediately any over-engineered spec by