On Sun, 8 Jul 2018 10:06:14 +0100 Dimitris Papastamos <[email protected]> wrote:
> On Sun, Jul 08, 2018 at 01:12:59AM +0200, Mattias Andrée wrote: > > On Sat, 7 Jul 2018 23:29:53 +0100 > > Dimitris Papastamos <[email protected]> wrote: > > > > > On Sun, Jul 08, 2018 at 12:12:08AM +0200, Mattias Andrée wrote: > > > > On Sat, 7 Jul 2018 22:55:28 +0100 > > > > Dimitris Papastamos <[email protected]> wrote: > > > > > > > > > This is too intrusive, what's wrong with using a shell script to test > > > > > the commands? > > > > > > > > > > The test framework is more complicated than most sbase commands. > > > > > > > > > > It would have been nice to discuss this in advance before writing a > > > > > 1000 line patch that might not get merged. > > > > > > > > > > > > > Writing all tests in shell isn't the best idea I think. > > > > This frameworks makes it easy to write test and it will > > > > tell you everything you need to know to figure out what > > > > failed. I believe that in the need this will reduce the > > > > amount of test code. There are things that are difficult > > > > to do in shell: for example create a terminal which is > > > > need to test tty(1). Look for example at the tests in > > > > https://github.com/maandree/base-util-tests, they aren't > > > > that nice, and they even require bash(1), it would be > > > > even worse with portable sh(1). > > > > > > so what you are saying is that your shell code sucks so > > > it is better done in C? > > > > > > sorry im not buying it, this is overkill > > > > > > > No, I'm saying that if you want to do it in sh(1) you will > > need more test, or make really crappy helper functions, and > > you will also need to write special utilities in C just to > > test some utilities that cannot be tested entirely in sh(1). > > > > The C code contains: > > > > * a way to print where in a loop a test fails. > > this will be useful for example when when adding test > > cases for the *sum utilities, > > > > * a way to make tests asynchronously, this could be removed > > but will be useful for testing sleep(1), so it does not > > take too long, > > > > * a simple way to measure and test how long a test took, > > > > * a set of tiny functions to declare how to program shall > > be started, > > > > * a way to create sockets and TTYs. The socket part can > > probably be removed but it would be useful for testing > > different file types. Support for regular files should > > probably be added. TTYs are required to testing tty(1), > > > > * ways to test the exit status with support for both normal > > exit and kill by signal, > > > > * ways to check the output to stdout and stderr. In sh(1), > > test(1) and grep(1) could be used, > > > > * some code to make to test cases smaller, > > > > * a function to spawn a process with the requested files > > and input, and > > > > * a function to read a process's output and wait for it. > > > > I wouldn't say that it's complex, its just a few functions, and > > 2 or 3 of them is are bit long, but not particularly long. I > > think this is worth it for the consistence, short tests and > > easily readable tests, a convenient way to locate the failing > > test and what's actually wrong, and to not have extra utilities > > (also written in C) to do the parts of the tests that cannot be > > done in C. I'm not sure how mean that the code is complicated, > > it's just long. > > https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/regress/usr.bin/ > > whatever you do, this test code should be in a different repo > like sbase-regress or similar. > I will reduce code in the test-common.[ch]. If the tests are in a separate repository, package maintainers need to download both, do you really think this is a good idea? What's the advantage with make it a separate repository?
pgp83S4qBiPq_.pgp
Description: OpenPGP digital signature
