On Sat, Jul 11, 2009 at 12:12 AM, Pavel Sanda <sa...@lyx.org> wrote:
> > >> Would keycodes be useful for submitting manual bugs as well?
> > >
> > > some links to keys is a must. most of the bugs wouldn't be reachible
> just from
> > > the bakctrace and screenshot.
> >
> > I mean bugs that are manually found. If we have the precise keycodes
> > to reproduce it eliminates some possible ambiguities in the bug
> > report,
>
> i think this is overengineering for normal bug reports
>
> >and opens the possibility that these bugs could be
> > automatically reproduced (e.g. for regression tests)
>
> Abdel tried to propose such kind of regression testing. if you feel like
> to be the main lyx regression tester, why not, but be warned that this will
> be mainly your project without much help from the others i guess... :)
>
> that said it would be actually good to have some automatical regression
> testing.
I was thinking that I could submit bug reports as regression tests. To
submit a *good* bug report requires various tasks:
- Checking which revision of svn we are using.
- Checking if I can reproduce with the latest version of SVN.
- Pasting in Arch/System/Locale type information
- Checking that the steps to reproduce actually work, that there are no
steps missed out.
It would be easier for me to write something like:
$ cat TestSelectInset.kt
KEYCODE="\Ca\[BackSpace]\Cmasdf\[Right]\D9\S\[Left]\D9X\Cs\D9"
TEST="! grep asdf $F.lyx"
DESC="r30414 unselects insets after a fraction of a second. 1.5.7 doesn't
exhibit this bug"
Then I could get the computer to check that the steps are correct:
$ ly6 lyxtest_16x &
$ ly5 lyxtest_157 &
$ ( F=lyxtest_16x ./do_regression_test.sh TestSelectInset.kt )
\begin_inset Formula $asdf$
FAIL: TestSelectInset.kt
$ ( F=lyxtest_157 ./do_regression_test.sh TestSelectInset.kt )
PASS: TestSelectInset.kt
Finally I could add TestSelectInset.kt to keytest, and let keytest handle
all the boring details.
Once "FAIL: TestSelectInset.kt" shows up in the keytest report, we could
move it to Trac. Alternatively if the bug is easy to fix, we could just fix
it. Then keytest could determine that a new svn revision was availiable, and
re-run the regression tests. Then somewhere in the report, keytest could add
a note like
"TestSelectInset.kt fixed by alice_w in r30999"
This would have a few of downsides. E.g. people who rearrange the GUI would
"fix" a bunch of regressions and also "break" quite a few as well. These
things could be worked around, e.g. by having a regression2regression
script. It would be interesting to discover to what extent this work flow
would be useful.
FYI, here is do_regression_test.sh, although it just a proof of concept:
$ cat ./do_regression_test.sh
sendtext() {
xvkbd -delay 50 -text "$1" -xsendevent 2> /dev/null
}
wmctrl -R "$F" && (
(. "./$1" # Load KEYCODE and TEST from file
sendtext "$KEYCODE"
if bash -c "$TEST"
then
echo PASS: $1
else
echo FAIL: $1
fi
)
)
--
John C. McCabe-Dansted
PhD Student
University of Western Australia
