Excerpts from André Rømcke's message of Mon Mar 26 04:44:53 -0700 2012: > On Fri, Jan 20, 2012 at 1:43 AM, Clint Byrum <cl...@ubuntu.com> wrote: > > > Excerpts from Stas Malyshev's message of Thu Jan 19 16:08:28 -0800 2012: > > > Hi! > > > > > > > - According to this website there are still 94 test failures in 5.4 . > > > > Can you confirm all of them are minor problems? > > > > http://gcov.php.net/viewer.php?version=PHP_5_4 > > > > > > Most of them appear so, I'll go through them again to be sure and > > > encourage others to do so too and raise red flags if somebody sees > > > something bad there. > > > Unfortunately, some tests are environment-dependent or otherwise have > > > subtle dependencies or structure that make them work on one system and > > > fail on another not because of the bug in PHP but because of the test > > > itself. So, I have 0 fails on my Linux build but 6 fails on my Mac > > > build. Other times some systems may not support some capability, use old > > > version of the library, etc. and the test may not account for that. > > > > These tests should be skipped or marked as XFAIL on platforms they are > > known to fail on. Better to have no test than one that cannot be relied > > upon. All supported platforms should pass with 0 fails. These intentional > > skips should have open bugs that are documented in the test code so that > > a developer can find out why this test was disabled when trying to make > > a change covered by the test. > > > > > > > > I do not think it is practical to postpone release until we solve all of > > > such problems, since this being volunteer-driven open-source project > > > this means not having any release schedule at all. I prefer having the > > > schedule even if that means we'd have to release with some known > > > deficiencies. > > > > > > > Its pretty bad actually. For all of PHP's success, this is something that > > continues to baffle me, and many others I have talked to who are charged > > with measuring quality and with patching systems in a timely manner. How > > better to document unreliable tests than to skip them with something like > > "SKIPPED - known to fail on Mac". > > > > Its precisely this unreliability that forced me to take a conservative > > approach for Ubuntu 12.04 and recommend to the community that we ship > > 5.3.9 instead of 5.4.0. I would much rather have the new stuff in, but > > even if all the tests pass on the machine we run the test suite on, > > how can we be sure they won't fail in another time zone, or in some > > other strange configuration? > > > > > Given that 12.04 beta2 will be out on Thursday, and the unit tests where > fixed before shipping 5.4 (I naively assume), is this in or out? > ref: https://blueprints.launchpad.net/ubuntu/+spec/servercloud-p-php54 > > With beta freeze on march 22 I guess the "mistake" of bundeling 5.3 instead > of 5.4 is already made, and we will have to live with that for the next 2 > years for prod, and the next 7 months for dev. >
Our hands are tied, as the security team still does not feel comfortable shipping a PHP without Suhosin. Perhaps more can be done to convince the world that this is a safe thing to do now, but for now, we're taking the extremely conservative stance and shipping 5.3.10 with the Suhosin patch. Thanks everyone for chiming in, and especially thanks to Ondrej for pushing hard to get things tested and rebuilt. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
