Hi, This is exciting.
Rather than impose a complete directory/layout schema before-hand I'd lean towards starting with a little more chaos and then letting the structure of the test directory develop /naturally/. From the discussion below it sounds like an initial structure of testing/lisp/ testing/contrib/lisp/ may make sense, reserving the top level for "meta" testing stuff, like functions for running tests, common fixtures, example files, etc... I have two questions. 1) while waiting for ert to be included into Emacs, should we include an ert distribution as part of the Org-mode repository (maybe using git sub-modules) or should we just agree that users should have a certain version of ert installed locally? I'm honestly not sure which of these options sounds preferable. 2) should the initial population of the testing/ directory take place in a separate branch of the repository or in the master branch? Again I don't know which I would prefer, branches add complication but could result in cleaner commit histories. Best -- Eric Carsten Dominik <carsten.domi...@gmail.com> writes: > Hi Sebastian, > > the lack of a testing suite for Org-mode is really frustrating, > and even more frustrating is that we have had like seven attempts > to start one, and each of these lead to nothing. So I would > be perfectly happy to give a free hand, write access to the repo > and a full directory in the distribution to implement one. > Once there is a framework, I am sure many people would be > willing to contribute tests. > > More comments below. > > On Oct 2, 2010, at 5:51 AM, Sebastian Rose wrote: > >> Hi, >> >> >> I thought about testing again recently. This is something, that never >> really got started. For a reason: there's no framework for testing. >> >> I therefore wrote a very rough proposal, found on >> http://github.com/SebastianRose/org-test >> >> The idea is, to provide two simple commands: >> >> >> * org-test-test-current-defun >> will search for tests for the defun point is in or behind >> (`beginning-of-defun') and execute them surrounded by >> >> (let ((select (or selector "^org")) >> (deactivate-mark nil)) >> (save-excursion >> (save-match-data >> >> >> * org-test-test-buffer-file >> will search for tests for the entire file and execute them the >> same >> way. > > FIrst: I have *no* clue about testing. > > Second, I am surprised that you want to structure it by function. I > would have > thought that it could be structure by file at the most. And then > there will > be tests that involve code from many files. > > But I guess > >> >> If you use one of these commands, all currently registered ERT tests >> are >> deleted, and files are reloaded (since you're likely to work on the >> tests, too). To repeat the tests without reloading, you will use the >> ERT commands like `ert-results-rerun-all-tests', bound to `r' in the >> ERT >> results buffer. >> >> >> >> I choose ERT (git clone http://github.com/ohler/ert.git) because >> that's >> likely to go into Emacs core (or elpa.gnu.org). >> >> >> >> >> The idea is to search the directory structure from the current source >> file upwards for a directory named "tests/" if it exists. Else ask >> the >> user. Similar to what `add-change-log-entry' does. >> >> Below that directory, a tree like the source tree exists: >> >> project >> +-- lisp/ >> | +-- a.el >> | `-- b/ >> | +-- b.el >> | >> `-- tests/ >> +-- a.el/ >> | +-- tests.el >> | `-- a-defun.el >> `-- b/ >> +-- b.el/ >> +-- tests.el >> `-- b-defun.el >> >> If this setup exists, when editing defun-x in lisp/a.el, >> `M-x org-test-test-current-defun' will load tests/a.el/defun-x.el >> (fallback: tests.el there) and execute all tests with selector >> "^a-defun". > > Well, OK, this is fine. But under a.el and b.el there should also be > general tests that are not function dependent, and there should be a > place > to put tests that you do not want to assign to a specific file. > > We do have a "testing" directory already, you can use that. > I would prefer the tests to be in testing, not in lisp/testing > if possible. I would like to have the lisp directory contain > only code. If possible. > > It would be OK to have a lisp subdirectory in testing, > just as it would be OK to have contrib/lisp in testing > for the contributed packages. > > PLEASE, go ahead. I do not think you have write access > yet on repo - give me your user name and I'll activate you. > > - Carsten > >> `M-x org-test-test-buffer-file' in that same source file will load all >> *.el files in tests/a.el/ and execute all ERT tests for selector "^a". >> >> >> Thus tests for >> org-mode/lisp/org-protocol.el >> will be searched in the directory >> org-mode/tests/lisp/org-protocol.el/*.el >> >> >> Once the basic route of testing is clear, I'd like to "translate" the >> existing tests for org-html.el to work with ERT, which will involve >> writing more tools (create output buffers, compare output with control >> files using ediff etc.). I know Lennart Borgman has wrote that stuff >> for nXhtml already. I hope we can use his stuff and help here. >> >> The directory org-mode/lisp/tests/ would not need to be part of the >> "official" Org mode package. It could as well be checked out >> separately, if "tests" is part of org-mode/lisp/.gitignore (e.g.). >> >> >> Any thoughts? >> >> >> >> >> Sebastian > > > _______________________________________________ > Emacs-orgmode mailing list > Please use `Reply All' to send replies to the list. > Emacs-orgmode@gnu.org > http://lists.gnu.org/mailman/listinfo/emacs-orgmode _______________________________________________ Emacs-orgmode mailing list Please use `Reply All' to send replies to the list. Emacs-orgmode@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-orgmode
