When building PFTT, the PHP Full Test Tool, one goal was maximizing speed so that we would be more likely to frequently Fully test PHP (Full-coverage of many scenarios), and to do that I built it to run tests concurrently...
Concurrent/parallel execution can cause breaks that otherwise work with normal serial execution, so definitely don't enable concurrency in run-tests by default. run-tests.php's two main benefits are that it supports running the huge library of 15k+pecl PHPTs (compatiblity) and its a lightweight tool so that anyone who just built PHP can run `make test` to test their build (whereas PFTT Is a heavyweight tool). In making/merging major changes (like concurrency) the most important thing is to not break that compatibility and dependability: Your patch will fail without the /proc filesystem(mac osx?). On Windows, you can check the %NUMBER_OF_PROCESSORS% environment variable. For unices that don't have /proc there may not be a single simple way to check. For better performance, PFTT runs a pool of 3 to 4 processes per CPU, when the processes finish quickly, it runs fewer and when they slow down (due to pending IO ops), it runs more processes. See: http://git.php.net/?p=pftt2.git;a=blob;f=src/com/mostc/pftt/scenario/CLIScenario.java;h=30c81aabcb56698eebe4884d915298c92332b50d;hb=refs/heads/master#l61 and for other SAPI scenarios like Apache it runs 2 or 3 processes, see: http://git.php.net/?p=pftt2.git;a=blob;f=src/com/mostc/pftt/scenario/ProductionWebServerScenario.java;h=d509bddda2b3756f05321ddc273600aeb38372be;hb=refs/heads/master For more speed, rather than grouping all, fe open_basedir tests into a single `Concurrency Group` its better to have them all operate in separate temporary directories and database tables (then you could run 32 or 64 openbase dir tests at once). The ext/standard/tests/file and ext/mysql* PHPTs take the longest to run and mostly do IO operations so they benefit the most from maximizing concurrency. For databases, PFTT will create a new database (and for phpunit tests populate the database) for each test runner thread, See: http://git.php.net/?p=pftt2.git;a=blob;f=src/com/mostc/pftt/scenario/MySQLScenario.java;h=f768f58c4494cb2705a571ece22d533bfdc15192;hb=refs/heads/master#l164 On Wed, Sep 10, 2014 at 3:36 AM, Michael Wallner <m...@php.net> wrote: > On 09/07/14 22:00, Stas Malyshev wrote: > > Hi! > > > >> Having thought a bit about the subject, the heuristic I came up with > was to > >> 1) do not run tests from the same extension in parallel, and 2) do not > run > >> tests from certain extensions in parallel (e.g., mysql and pdo_mysql). > > > > How about making it a whitelist? For tests/extensions we know it's easy > > to paralellize (like ones not having external dependencies/servers/etc) > > simple script could whitelist a lot of them. For those more tricky, we > > could whitelist them gradually. > > > > So how about adding the concurrency switch, but leaving it disabled by > default? > > -- > Regards, > Mike > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: http://www.php.net/unsub.php > >
