On Sun, Jan 19, 2025 at 07:26:58PM +0100, Michail Nikolaev wrote: > Mostly idea is next: > > Let's imagine we have two steps - step_before and step_after which may end > in either order. > Then instead of such step/markers structure: > > step_before(step_after) > stepN > stepN+1 > step_after > > use the next: > > step_before > stepN > stepN+1 > step_after(step_before) > > In the first case, there are two possible results: > 1) step_before is finished before step_after - reported as step_before, > step_after > 2) step_after is launched before step_before ends - reported as step_after, > step_before > > But in the case second variant: > 1) step_before is finished before step_after - reported as step_before, > step_after > 2) step_after is launched before step_before ends - reported as > step_before, step_after > > So, the second option provides the same result regardless of order of > finishing of step_before and step_after, which is the thing I want to > achieve here.
Stepping back, any variation in the total ordering of step-start and step-complete events necessarily changes the expected output. Hence, we want to freeze that ordering. We could do that directly, by having isolationtester read a format that lists start and complete events. We could also have the isolationtester write the same format, so you could draft a permutation without blocking conditions and just save the order that isolationtester discovers by experiment. I'd find that less mentally taxing than following rules to reach a frozen ordering via blocking conditions. As an example of one concrete way to achieve that, have isolationtester write a results/SPECNAME.wait file that contains syntax like "permutation-event waiting:S1 S2 complete:S1 S3". You'd copy that file to expected/SPECNAME.wait, which isolationtester would read at startup. This wouldn't replace "notices" blocking conditions for certain complex tests, and tests with multiple expected outputs might not want it. It would cover the rest. Any unprefixed entry (e.g. the plain "S2") would get the simplification of skipping the check for whether the step has entered a wait. How well would that meet your test's needs?
