Hello, Noah! > Stepping back, any variation in the total ordering of step-start and > step-complete events necessarily changes the expected output. Hence, we want > to freeze that ordering. We could do that directly, by having isolationtester > read a format that lists start and complete events. We could also have the > isolationtester write the same format, so you could draft a permutation > without blocking conditions and just save the order that isolationtester > discovers by experiment. I'd find that less mentally taxing than following > rules to reach a frozen ordering via blocking conditions.
> How well would that meet your test's needs? It feels a bit strange to me to force isolationtest to output something in a specific order, especially since we don't actually care about the order in the test. But why do we need that output at all? I think a simpler solution would be to mark a step in a way that prevents isolationtester from outputting detailed information about the completion of that step—just an overview of the steps launched and completed at the end of the permutation. In my tests, I evaluate how [CREATE|RE] INDEX CONCURRENTLY interferes with UPSERTs. I’m not testing the exact ordering; I simply set the backends into a specific state (with different index information) and let them proceed one by one. I don’t care which one finishes first or last—I’m only interested in identifying errors in the output. Best regards, Mikhail.
