Hi!
These tests should be skipped or marked as XFAIL on platforms they are known to fail on. Better to have no test than one that cannot be relied upon. All supported platforms should pass with 0 fails. These intentional
Of course they should, and you (or anybody else) are welcome to make the patches that make them so :) I promise they'll be in 5.4.1.
Its precisely this unreliability that forced me to take a conservative approach for Ubuntu 12.04 and recommend to the community that we ship 5.3.9 instead of 5.4.0. I would much rather have the new stuff in, but
5.4.0 is better, not worse, than 5.3.9 in this regard - especially because of the work that was done in 5.4 branch to fix or improve tests that were failing and improve test coverage. I would advise to give your users a choice once 5.4.0 is released and support both packages. On my Linux environment, I have now 0 fails (though I don't run all modules so some of non-default ones may be missing).
even if all the tests pass on the machine we run the test suite on, how can we be sure they won't fail in another time zone, or in some other strange configuration?
Well, you never can be *sure* - tests can test only things they know about, unless somebody invents a way to make unit test test every possible combination of environments at once. So far I never heard about such tests.
The fact that its all being running regularly is a fantastic improvement. I'd like to see a commitment to getting 100% pass/xfail/skip for every release/tested environment in future releases though.
I'd like that too. I hope more people will step up and help to make this a reality.
-- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/ (408)454-6900 ext. 227 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
