Hi Zoe and everyone :) This thread is a proof that many people want to help the QA in PHP. Although, many things done so far are redundant.
Specific valgrind reports per platform/architecture seems useless to me, mainly because results are unreadable (most of the time the leak is in a library that is not part of PHP). Valgrind reports should be prepared with caution (the one on http://gcov.php.net seems sufficient). Concerning code coverage, differences between platforms are tiny and maybe should we aggregate all results (does it has any sense to show all results separately ?) Olivier sent a note earlier this week as an attempt to map out requirements > what should happen with the ad-hoc test reports, however I'm wondering if > we should just scrap the ad-hoc test reports and put the effort into getting > results from a defined set of platforms? Aside from the fact that they go to > a mailing list which is hard to view, would anyone make use of the ad-hoc > reports if we could get test results from a representative set of platforms > in a more controlled way? I'm sure that with good tools you can make good things :) Of course the ad-hoc reports will be useful ! a "controlled way" is not changing anything for php tests (this is different for valgrind/gcov). Question is "How to get results from a wide variety of platforms/configurations and aggregate them all to highlight where bugs (leaks ?) are ?". We already have the solution through the 'make test' result, even if they are sent to a mailing list and not easily readable. To get back to my note posted earlier: it would be easy to have an interface collecting all phpruntests results (ie from all platforms used by the testers) and display them by all filters/aggregate functions you ever dreamt of. I'd like to be able to see test results across several platforms (various > *ix, windows, mac....), 64 bit, 32 bit. And when a test fails to be able to > get to the relevant files. This is similar to the gcov pages (but more) with > the ability to have different views of the results, I like Mark's summary > page but would want to be able have a few different views, the test fest > results page has a few of the attributes that I'd like to see (not the color > scheme of course). Is the sort of thing we would be able to do? The interface I was talking about a few days ago is all about that :) And what's awesome in it is that providing feedback for PHP become very easy : just a small script (less than 20 bash lines) to get latest php snapshot, configure + make + make test with test results automatically sent to this interface (instead of a mailing list) and you're done. Such script for linux/pc/others could be made available on php-qa website or php website. What to do for this interface (whether use existing tool, or write one from scratch) is exactly the draft I wrote on Thursday. You are all welcome to give feedback on it (how to do it and what should it do) on php-qa. Olivier
