I think the point is that an exclusion group needs to be managed manually, which means if you add a new module you have to check every test to see if the module needs adding to an exclusion group there... which seems worse than just making the test async: false.
Worth pointing out (you may or may not know) you can pull your synchronous tests into their own module so the absolute minimum number of tests are running async which seems to be what you are after? Also that module can be in the same test file as a test module with async tests inside them. Best Adam > On 26 Nov 2021, at 16:42, Paul Dann <[email protected]> wrote: > > On Fri, 26 Nov 2021 at 14:24, José Valim <[email protected] > <mailto:[email protected]>> wrote: > Right, my concern is not about the existing tests, which indeed need to > revert, but future changes. Taking the application environment example, now > anyone in the future that adds a feature that reads from that environment or > writes a test that reads from that environment, they need to remember to > annotate their new test case to exclude a completely unrelated file, that may > have been written 1 week, 1 month, or 1 year ago. > > I think you're saying that in the Application.put_env() case, the safe option > is to always mark a test file async: false as soon as global state such as > this is used, and then future tests are isolated from this test as well. And > that's true; that probably remains the safest approach. But maybe in that > situation, when you're about to switch async: true to false in your new test > file, a second alternative is to check for existing uses of the same global > state in the test suite, and adding an exclusion group instead. > > If code is refactored to use affected global state, it shouldn't take too > long for the race conditions to start appearing in the CI, at which point in > my experience it's not too difficult to track down where global state is used > in that test file. I think that can pretty easily happen to regular async > tests too. The time saved in test suite runs can outweigh the pain of > diagnosing a false-failure, I think, depending on the project. I'm certainly > not saying this is the best way for all projects - it requires care, much > like running Ecto in async tests requires care and a good understanding of > the problem. > > Paul > > > > -- > You received this message because you are subscribed to the Google Groups > "elixir-lang-core" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elixir-lang-core/CALZj-Vp2YZRooJtB2oCd4HkHwHFBL9kXb6VapZFRPGV8j89XBw%40mail.gmail.com > > <https://groups.google.com/d/msgid/elixir-lang-core/CALZj-Vp2YZRooJtB2oCd4HkHwHFBL9kXb6VapZFRPGV8j89XBw%40mail.gmail.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/0E0554D5-3C7D-4650-B77B-53B2A158DC26%40a-corp.co.uk.
