2011/9/15 Hannes Magnusson <hannes.magnus...@gmail.com>: > On Wed, Sep 14, 2011 at 12:36, Alexey Shein <con...@gmail.com> wrote: >> 2011/9/14 Ferenc Kovacs <tyr...@gmail.com>: >>>> >>>> Why do we even have this tmp-php.ini? Why not just make test without >>>> any .ini files, i.e. just with -n option? >>> >>> [20:44:09] <bjori> Tyrael: tmp-php.ini is built from the system ini >>> [20:44:45] <bjori> Tyrael: if he builds php with >>> --with-config-file-scan-dir=/where/ever/he/has/another/php.ini, then >>> it will use that ini >>> [20:45:09] <bjori> Tyrael: and thats the whole point of tmp-php.ini, >>> to not only test the default ini, but also random user configurations >>> >>> I agree that it is a bad practice to test random user configuration in >>> our test suite. >>> that's just another moving part in the system. >> >> I understand how it works, maybe should we do then this behavior >> optional (if user wants to test his current configuration) and default >> make test will run without any php.inis? > > No? > > For tests that require certain ini options we have the --INI-- section. > For all other tests, we need to ensure they work in all environments, > be it under openbasedir, different precision, > call_time_pass_reference, variable order, and what have you. > > And we do want to load the shared extensions the user is using to test > them too and ensure there isn't any new unexpected conflicts. > > > Running tests exclusively in an environment known to work doesn't help us.
Agreed. But you should cross the line somewhere. The test must be repeatable or it won't be useful, you will have all kinds of erratic tests (described for example here: http://xunitpatterns.com/Erratic%20Test.html). How do you know if the test failed for true or is that user borked his php.ini? It could create a lot of false negative bug reports on qa.php.net (but it's no problem since nobody is looking there :) ) Maybe run failed tests again without user's php.ini? > When you write a test case you need to know what you are testing. Just > writing a test to write one doesn't give us any benefits. > We can easily write test cases that give us 90%+ code coverage, but > that has absolutely no meaning if it doesn't actually test anything. > > -Hannes > -- Regards, Shein Alexey -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
