On Sun, Jul 08, 2018 at 01:12:59AM +0200, Mattias Andrée wrote: > On Sat, 7 Jul 2018 23:29:53 +0100 > Dimitris Papastamos <[email protected]> wrote: > > > On Sun, Jul 08, 2018 at 12:12:08AM +0200, Mattias Andrée wrote: > > > On Sat, 7 Jul 2018 22:55:28 +0100 > > > Dimitris Papastamos <[email protected]> wrote: > > > > > > > This is too intrusive, what's wrong with using a shell script to test > > > > the commands? > > > > > > > > The test framework is more complicated than most sbase commands. > > > > > > > > It would have been nice to discuss this in advance before writing a > > > > 1000 line patch that might not get merged. > > > > > > > > > > Writing all tests in shell isn't the best idea I think. > > > This frameworks makes it easy to write test and it will > > > tell you everything you need to know to figure out what > > > failed. I believe that in the need this will reduce the > > > amount of test code. There are things that are difficult > > > to do in shell: for example create a terminal which is > > > need to test tty(1). Look for example at the tests in > > > https://github.com/maandree/base-util-tests, they aren't > > > that nice, and they even require bash(1), it would be > > > even worse with portable sh(1). > > > > so what you are saying is that your shell code sucks so > > it is better done in C? > > > > sorry im not buying it, this is overkill > > > > No, I'm saying that if you want to do it in sh(1) you will > need more test, or make really crappy helper functions, and > you will also need to write special utilities in C just to > test some utilities that cannot be tested entirely in sh(1). > > The C code contains: > > * a way to print where in a loop a test fails. > this will be useful for example when when adding test > cases for the *sum utilities, > > * a way to make tests asynchronously, this could be removed > but will be useful for testing sleep(1), so it does not > take too long, > > * a simple way to measure and test how long a test took, > > * a set of tiny functions to declare how to program shall > be started, > > * a way to create sockets and TTYs. The socket part can > probably be removed but it would be useful for testing > different file types. Support for regular files should > probably be added. TTYs are required to testing tty(1), > > * ways to test the exit status with support for both normal > exit and kill by signal, > > * ways to check the output to stdout and stderr. In sh(1), > test(1) and grep(1) could be used, > > * some code to make to test cases smaller, > > * a function to spawn a process with the requested files > and input, and > > * a function to read a process's output and wait for it. > > I wouldn't say that it's complex, its just a few functions, and > 2 or 3 of them is are bit long, but not particularly long. I > think this is worth it for the consistence, short tests and > easily readable tests, a convenient way to locate the failing > test and what's actually wrong, and to not have extra utilities > (also written in C) to do the parts of the tests that cannot be > done in C. I'm not sure how mean that the code is complicated, > it's just long.
https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/regress/usr.bin/ whatever you do, this test code should be in a different repo like sbase-regress or similar.
