#32769: Fix for sporadically crashing parallel test runner
-----------------------------------+--------------------------------------
Reporter: quinox | Owner: nobody
Type: Bug | Status: closed
Component: Testing framework | Version: 2.2
Severity: Normal | Resolution: duplicate
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------------+--------------------------------------
Changes (by Mariusz Felisiak):
* cc: Chris Jerdonek (added)
* status: new => closed
* resolution: => duplicate
Comment:
Thanks for the report. I think it's a duplicate of #27734, it's not an
issue with the number of workers, but an issue with a shared counter (see
[https://code.djangoproject.com/ticket/27734#comment:5 comment]). Reducing
the number of workers just make it less likely. Can you reproduce it with
Python 3.8+ and on the main branch? Django 2.2 is in extended support so
it doesn't receive bugfixes anymore (except security patches).
I agree that we could reduce the number of workers but it's only a small
cleanup that will not fix the main issue described in #27734. I would
reduce the number of processes directly in the `ParallelTestSuite`, e.g.
{{{
diff --git a/django/test/runner.py b/django/test/runner.py
index ab5512e628..50f66d3d87 100644
--- a/django/test/runner.py
+++ b/django/test/runner.py
@@ -405,7 +405,9 @@ class ParallelTestSuite(unittest.TestSuite):
def __init__(self, suite, processes, failfast=False, buffer=False):
self.subsuites = partition_suite_by_case(suite)
- self.processes = processes
+ # Since tests are distributed across processes on a per-TestCase
basis,
+ # there's no need for more processes than TestCases.
+ self.processes = min(processes, len(self.subsuites))
self.failfast = failfast
self.buffer = buffer
super().__init__()
@@ -662,14 +664,8 @@ class DiscoverRunner:
self.failfast,
self.buffer,
)
-
- # Since tests are distributed across processes on a per-
TestCase
- # basis, there's no need for more processes than TestCases.
- parallel_units = len(parallel_suite.subsuites)
- self.parallel = min(self.parallel, parallel_units)
-
# If there's only one TestCase, parallelization isn't needed.
- if self.parallel > 1:
+ if parallel_suite.processes > 1:
suite = parallel_suite
return suite
}}}
Feel-free to prepare a patch with `Refs #27734 -- ...`.
Duplicate of #27734.
--
Ticket URL: <https://code.djangoproject.com/ticket/32769#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/064.5e243e562a749f8cde93959ded93ca19%40djangoproject.com.
