On Thu, Jun 13, 2024 at 12:20 PM Robert Haas <robertmh...@gmail.com> wrote:
> I interpreted Jacob's original email as articulating a goal ("For the
> v18 cycle, I would like to try to get pytest [1] in as a supported
> test driver, in addition to the current offerings") rather than a
> plan.That's the first part of it. > There's no patch set yet and, as I understand it, no detailed > plan for a patch set: that email seemed to focus on the question of > desirability, rather than on outlining a plan of work, which I assume > is still to come. There was a four-step plan sketch at the end of that email, titled "A Plan". That was not intended to be "the final detailed plan", because I was soliciting feedback on the exact pieces people wanted to try to implement first, and I am not the God Emperor of Pytest. But it was definitely A Plan. > Some things I'd like to see when a patch set does > show up are: > > - good documentation for people who have no previous experience with > Python and/or pytest e.g. here's how to set up your environment on > Linux, Windows, macOS, *BSD so you can run the tests, here's how to > run the tests, here's how it's different from the Perl framework we > have now +1 > - no external dependencies on PostgreSQL connectors. psql or libpq > foreign function interface. the latter would be a cool increment of > progress over the status quo. If this is a -1 for psycopg, then I will cast my vote for ctypes/CFFI and against psql. > - at least as much in-tree support for writing tests as we have today > with PostgreSQL::Test::whatever, but not necessarily a 1:1 reinvention > of the stuff we have now, and documentation of those facilities that > is as good or, ideally, better than what we have today. I think this is way too much expectation for a v1 patch. If you were committing this by yourself, would you agree to develop the entirety of PostgreSQL::Test in a single commit, without the benefit of the buildfarm checking you as you went, and other people trying to write tests with it? > - high overall code quality and level of maturity, not just something > someone threw together for parity with the Perl system. +1 > - enough tests written for or converted to the new system to give > reviewers confidence that it's truly usable and fit for purpose. This is that "know everything up front" tax that I think is not reasonable for a test framework. If the thing you're trying to avoid is the foot-in-the-door phenomenon, I would agree with you for a Postgres feature. But these are tests; we don't ship them, we have different rules for backporting them, they are developed in a very different way. > The important thing to me here (as it so often is) is to think like a > maintainer. Imagine that immediately after the patches for this > feature are committed, the developers who did the work all disappear > from the community and are never heard from again. How much pain does > that end us causing? The answer doesn't need to be zero; that is > unrealistic. But it also shouldn't be "well, if that happens we're > going to have to rip the feature out" Can you elaborate on why that's not an okay outcome? > or "well, a bunch of committers > who didn't want to write tests in Python in the first place are now > going to have to do a lot of work in Python to stabilize the work > already committed." Is it that? If the problem is that, we should address that. Because if that is truly the fear, I cannot assuage that fear without showing you something, and I cannot show you something you do not want to see, if you don't want to write tests in Python in the first place. --Jacob
