Ah yes we've run into this exact thing doing tests in Clickhouse. Even for postgres, this would be amazing for testing features like pg_notify or other postgres features that depend on an actual COMMIT.
On Tuesday, June 16, 2026 at 10:06:05 AM UTC-4 [email protected] wrote: > > What we typically do is that we never delete it by default, instead we > clean up when the next test runs. The rationale is: > > > > 1. Leave resources for debugging > > 2. If tests are interrupted (ctrl+c or whatever other reason), you need > to deal with trailing resources anyway > > > > Would that be a problem here? > > What I'm toying with is a fork of *Ecto.Adapters.SQL.Sandbox* that > implements test isolation by doing a full clean copy of the test database > during setup per test, each copy named after the test in question, and > redirecting all test interaction to the copy, rather than using > transactions to get isolation. The goal is to support concurrent tests for > Sqlite and to allow introspecting database state on test failure for Sqlite > and Postgres. So in this case, > > 1. Tearing down the resource unconditionally *on_exit* is not desired (to > have a debuggable state left) > 2. Leaving behind every resource on successful tests is not desirable > (that's a lot of wasted disk space after running 2000 stateful tests) > 3. Keeping resources for failed tests and juggling *N* extra copies on > disk where test parallelism is *N* is just about manageable > > Under this mechanism, leaving test failure resources behind is manageable, > as is leaving *N* copies behind on test interruption, and all leftover > state is overwritable on next test run, but leaving *all copies* behind > is not viable for non-trivial test suites sizes or large seed databases. > The worst case then becomes some errant global configuration causing every > stateful test to fail—the mostly likely way that would happen is by > providing a bad database configuration itself, which would prevent creation > in the first place, preventing disk bloat from happening to begin with. > On Tuesday, June 16, 2026 at 3:19:03 AM UTC-5 José Valim wrote: > >> > However, I would also like to leave the resource in-place on test >> failure for inspection. (My *prepare_resource* function can handle the >> situation where a resource already exists on setup. This is opposite to the >> common tmpdir pattern where the resource cleans itself up automatically >> upon test conclusion but would have unexpected side effects if already >> setup.) >> >> What we typically do is that we never delete it by default, instead we >> clean up when the next test runs. The rationale is: >> >> 1. Leave resources for debugging >> 2. If tests are interrupted (ctrl+c or whatever other reason), you need >> to deal with trailing resources anyway >> >> Would that be a problem here? >> >> *José Valimhttps://dashbit.co/ <https://dashbit.co/>* >> >> >> On Tue, Jun 16, 2026 at 1:31 AM Christopher Keele <[email protected]> >> wrote: >> >>> Assuming, of course I wrote this proposal without the typo in the final >>> sentence and properly used an anonymous function for *on_exit(fn state >>> -> #... end)* in my examples. >>> >>> On Monday, June 15, 2026 at 6:27:48 PM UTC-5 Christopher Keele wrote: >>> >>>> I am creating a resource during test setup based on a test's context >>>> (namely, its module and name): >>>> >>>> >>>> >>>> *setup context do prepare_resource_for_test_context(context)end* >>>> >>>> I would like to be able to teardown this resource upon test conclusion. >>>> This is possible today: >>>> >>>> >>>> *setup context do prepare_resource_for_test_context(context)* >>>> * on_exit do* >>>> * cleanup_resource_for_test_context(context)* >>>> * end* >>>> *end* >>>> >>>> *Proposal* >>>> >>>> However, I would also like to leave the resource in-place on test >>>> failure for inspection. (My *prepare_resource* function can handle the >>>> situation where a resource already exists on setup. This is opposite to >>>> the >>>> common tmpdir pattern where the resource cleans itself up automatically >>>> upon test conclusion but would have unexpected side effects if already >>>> setup.) >>>> >>>> As far as I know, this is not possible today. I would like to do >>>> something like receiving the *ExUnit.state()* >>>> <https://ex-unit.hexdocs.pm/ExUnit.html#t:state/0> in the *on_exit* >>>> callback: >>>> >>>> >>>> *setup context do prepare_resource_for_test_context(context)* >>>> * on_exit state do* >>>> *case state do* >>>> * {:failed, _} -> :ok* >>>> * _ -> cleanup_resource_for_test_context(context)* >>>> * end* >>>> * end* >>>> *end* >>>> >>>> Are there reasons to not entertain this functionality? Is there another >>>> way to accomplish it that doesn't rely on a test formatter to notice >>>> *{:test_finished, >>>> test}* and do the cleanup at a global level? >>>> >>>> *Implementation* >>>> >>>> AFAICT we could enable this usecase trivially by threading >>>> *test_or_case.state* into >>>> <https://github.com/elixir-lang/elixir/blob/1d599978f7d36ea6896b294b3d26d110212be7ab/lib/ex_unit/lib/ex_unit/runner.ex#L541>*ExUnit.OnExitHandler.run >>>> >>>> <https://github.com/elixir-lang/elixir/blob/1d599978f7d36ea6896b294b3d26d110212be7ab/lib/ex_unit/lib/ex_unit/runner.ex#L541> >>>> *and >>>> on to its helper functions. >>>> >>>> We could retain backwards-compatibility by having *exec_callback(callback, >>>> state)* >>>> <https://github.com/elixir-lang/elixir/blob/1d599978f7d36ea6896b294b3d26d110212be7ab/lib/ex_unit/lib/ex_unit/on_exit_handler.ex#L139> >>>> >>>> check the arity of the callback before invoking it. >>>> >>>> Documentation could describe the optional callback parameter and >>>> elaborate that an *on_exit* callback defined in a *setup_all* would >>>> have some other behaviour (raise an error, receive *nil* or the case >>>> name instead of a state, etc—open to ideas). >>>> >>>> Are would such an implementation be welcome? >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elixir-lang-core" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion visit >>> https://groups.google.com/d/msgid/elixir-lang-core/71b223e9-3815-4b8d-9ac0-cb32b98b05e8n%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/elixir-lang-core/71b223e9-3815-4b8d-9ac0-cb32b98b05e8n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/elixir-lang-core/4f3fd88f-6af7-4d35-a618-2314901f1c01n%40googlegroups.com.
