Hi all, There are some important design decisions to be made if we include unittest2 discovery into Django.
This message documents a proposed API, the pros and cons, and decisions that need to be made. https://code.djangoproject.com/ticket/17365 -- Limitation of current test setup: -- * Tests can only be in tests.py. * Tests must be included inside the app. * Full dotted paths to test cases don't work. -- How unittest2 discovery works: -- Unittest discovery has two advantages: 1) Better discovery for the existing case of testing an app Besides the old model and doc tests, current test discovery is limited to tests in tests.py. Unittest2 discovery allows files to be matched by pattern. For instance, these are all valid test modules: tests.py test_views.py test_models.py Further, a custom pattern can be set if the test files don't match the naming convention "test*.py". 2) Custom root folders can be specified for test discovery If the tests aren't in the app directory--a common case for reusable apps-- a custom root folder can be specified for discovery. ./manage.py test --root=../tests 3) Can use Python dotted paths to modules. These would all work: ./manage.py test myapp ./manage.py test myapp.tests ./manage.py test myapp.test_views ./manage.py test myapp.tests.MyTest ./manage.py test myapp.tests.MyTest.test_method ** The current "myapp.<testcase>" notation is not part of unittest2, and would go away. -- The Proposed API -- The unittest2 discovery is roughly-compatible with the existing custom Django discovery. Here's what the API could look like: # Discover tests under current directory ./manage.py test # Test with discovery under different root ./manage.py test root="../tests # Test with a custom top level directory ./manage.py test --root="../tests --top_level=../../top # Test with different pattern matching ./manage.py test --pattern="tests_*" # Test single installed app with discovery ./manage.py test contact # Test multiple apps with discovery ./manage.py test myapp1 myapp2 myapp3 # Test specific test module ./manage.py test myapp1.tests ./manage.py test myapp1.tests_views ./manage.py test myapp1.test_models # Test with mixed arguments ./manage.py test myapp.test_views myapp.test_models # Test single testcase ./manage.py test contact.tests.ContactTest # Test single method ./manage.py test contact.tests.ContactTest.test_contact # Test all installed apps with discovery ./manage.py test --installed -- Notes on above API: -- * Running `./manage.py test myapp` does discovery within the app and runs the tests. That means test in existing `tests.py` will be found, as well as in any file name "test*.py" * Root, pattern, and top-level options are passed from the `test` command to the unittest discovery. * Running a single test case uses a full dotted path, rather than the short path that Django currently uses (myapp.tests.MyTest vs myapp.MyTest) * The old behavior of running all installed app tests can be triggered by using the `--installed` flag. -- Opting into the new discovery -- To allow a deprecation cycle, the new discovery is an opt-in. To use the discover runner, a user must update their settings.py: TEST_RUNNER = 'django.test.runner.DiscoverRunner' Until Django 1.8, the 'django.test.simple.DjangoTestSuiteRunner' is still available, including it's ability to run doctest, model tests, etc. -- Backwards-incompatible changes -- * The new runner doesn't run doctests or tests in models.py * Some valid test structures in Django don't work with unittest2. For instance, tests in `tests/__init__.py` don't match a patter than unittest would recognize if running discovery on a module. Design Decisions ---------------- 1) Any changes to the API? I think the above API covers all the common use-cases. Is anything missing? 2) Should this be opt-in or not? The current patches are set up as opt-in. Until Django 1.8, a user has to manually update their test runner setting to use the DiscoverRunner. This lets current setups continue working, doctests and model tests included. Is there a reason not to follow the normal deprecation cycle? 3) What do to about doctests? There is discussion to roll doctest removal into this patch. https://code.djangoproject.com/ticket/18727 On one hand that's okay, but removing doctests has no bearing on using unitest2 discovery. It's simply a backwards-compatibly break to the old `django.test.simple.DjangoTestSuiteRunner`. Do we really want to do a hard cut-off for that? Should it really be part of the new discovery branch? If we remove doctests, why not remove support for discovery in models.py at the same time? 4) Option names for the `test` command. What should the discovery option names be? Above, I proposed this: ./manage.py test --root=../tests --top-level=../../top -- pattern="tests_*.py" If you use python -m unittest discover, the same options are: -s, --start-directory -p, --pattern -t, --top-level-directory Should we use the same options? -- Notes -- Because discovery is off-loaded to unittest2, the changes to Django itself are pretty small. Previous discussion and links to branches can be found in the ticket: https://code.djangoproject.com/ticket/17365 Preston -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
