Hi Sebastian,
the lack of a testing suite for Org-mode is really frustrating,
and even more frustrating is that we have had like seven attempts
to start one, and each of these lead to nothing. So I would
be perfectly happy to give a free hand, write access to the repo
and a full directory in the distribution to implement one.
Once there is a framework, I am sure many people would be
willing to contribute tests.
More comments below.
On Oct 2, 2010, at 5:51 AM, Sebastian Rose wrote:
Hi,
I thought about testing again recently. This is something, that never
really got started. For a reason: there's no framework for testing.
I therefore wrote a very rough proposal, found on
http://github.com/SebastianRose/org-test
The idea is, to provide two simple commands:
* org-test-test-current-defun
will search for tests for the defun point is in or behind
(`beginning-of-defun') and execute them surrounded by
(let ((select (or selector "^org"))
(deactivate-mark nil))
(save-excursion
(save-match-data
* org-test-test-buffer-file
will search for tests for the entire file and execute them the
same
way.
FIrst: I have *no* clue about testing.
Second, I am surprised that you want to structure it by function. I
would have
thought that it could be structure by file at the most. And then
there will
be tests that involve code from many files.
But I guess
If you use one of these commands, all currently registered ERT tests
are
deleted, and files are reloaded (since you're likely to work on the
tests, too). To repeat the tests without reloading, you will use the
ERT commands like `ert-results-rerun-all-tests', bound to `r' in the
ERT
results buffer.
I choose ERT (git clone http://github.com/ohler/ert.git) because
that's
likely to go into Emacs core (or elpa.gnu.org).
The idea is to search the directory structure from the current source
file upwards for a directory named "tests/" if it exists. Else ask
the
user. Similar to what `add-change-log-entry' does.
Below that directory, a tree like the source tree exists:
project
+-- lisp/
| +-- a.el
| `-- b/
| +-- b.el
|
`-- tests/
+-- a.el/
| +-- tests.el
| `-- a-defun.el
`-- b/
+-- b.el/
+-- tests.el
`-- b-defun.el
If this setup exists, when editing defun-x in lisp/a.el,
`M-x org-test-test-current-defun' will load tests/a.el/defun-x.el
(fallback: tests.el there) and execute all tests with selector
"^a-defun".
Well, OK, this is fine. But under a.el and b.el there should also be
general tests that are not function dependent, and there should be a
place
to put tests that you do not want to assign to a specific file.
We do have a "testing" directory already, you can use that.
I would prefer the tests to be in testing, not in lisp/testing
if possible. I would like to have the lisp directory contain
only code. If possible.
It would be OK to have a lisp subdirectory in testing,
just as it would be OK to have contrib/lisp in testing
for the contributed packages.
PLEASE, go ahead. I do not think you have write access
yet on repo - give me your user name and I'll activate you.
- Carsten
`M-x org-test-test-buffer-file' in that same source file will load all
*.el files in tests/a.el/ and execute all ERT tests for selector "^a".
Thus tests for
org-mode/lisp/org-protocol.el
will be searched in the directory
org-mode/tests/lisp/org-protocol.el/*.el
Once the basic route of testing is clear, I'd like to "translate" the
existing tests for org-html.el to work with ERT, which will involve
writing more tools (create output buffers, compare output with control
files using ediff etc.). I know Lennart Borgman has wrote that stuff
for nXhtml already. I hope we can use his stuff and help here.
The directory org-mode/lisp/tests/ would not need to be part of the
"official" Org mode package. It could as well be checked out
separately, if "tests" is part of org-mode/lisp/.gitignore (e.g.).
Any thoughts?
Sebastian
_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode
