Hi Sammy, > -----Original Message----- > From: sam...@sammykmedia.com [mailto:sam...@sammykmedia.com] On > Behalf Of Sammy Kaye Powers > Sent: Wednesday, May 17, 2017 4:29 PM > To: PHP Internals <internals@lists.php.net> > Subject: [PHP-DEV] Implement formal process for run-tests.php refactor > > Hello internals folks! > > As you may already know, run-tests.php is an old legacy app that is in serious > need of a refactor. Some things that are badly needed amongst other things > are: > > - Better maintainability > - Unit tests > - Concurrency > - Prettier output > > I have seen numerous attempts of rewriting run-tests from scratch after some > rallying at a UG or PHP conference only to see the initial excitement die > off. The > general advice for refactoring legacy apps is to not start with a clean slate, > rather refactor a little bit at a time. We should treat run-tests as we would > any > other legacy app. > > We need a structured process to allow the run-tests app to get refactored a > little bit at a time. I propose the following (open to discussion of course). > > 1) Create a new run-tests karma with access to php-src.git/run-tests.php (and > hopefully the soon-to-be created php-src.git/run-tests folder). > > 2) Elect 2-3 people and grant the new run-tests karma to them. They will > champion the PR's that incrementally improve run-tests. > > 3) The new run-tests champions (we'll call them "champions" because, why > not?) will run the entire test suite ensuring that the refactor didn't > inadvertently > break the functionality and if all good, merge. (We might be able to get a > solid > Docker container or an automated CI process for an end-to-end test - > thoughts?) > > 4) The run-tests champions could be elected for a year-long term with the > possibility of being reelected for another year commitment. This would help > prevent run-test champion drop-off (because sometimes, life happens). > > Once we get a process in place, I've drafted up a possible proposal of next > steps > to get run-tests refactoring kicked off: > > https://github.com/SammyK/run-tests#proposed-refactoring-path > > It's time to circle the wagons on run-tests! :) > That's a fair amount on pre evaluation already 😉.
AFM, the parallelism is the most wanted feature that would improve QA a lot, so that is what should stand in the foreground. Having a better structured app might help on it, yes, but having the parallelism to be targeted "some when" is probably not a good sign. Otherwise, an nice app with colors and styles, etc. would be indeed nice but less helpful, IMHO, as the current run-tests.php is great for simplicity and still maintainable well. I would suggest therefore, to put the harder goals into the foreground, as that would require and warrant a certain rewrite. It's not for nothing, that previous rewrite attempts targeting similar things was abandoned previously, and it's probably not wise to allow for the same mistakes. I'd see as first step an app that - is a very minimal rewrite of run-tests.php, made OO and whatsoever - mimics all the existing features and options, but only them - has as less dependencies as possible, be it PHP extensions or PHP code itself, best requires pure core only - is compatible with the current test format - is able to run tests in parallel - some simple tests are ported for parallel runs already I'd see such a bare app first developed as a pull request, agreed upon so then run-tests.php is completely replaced by it. Then where the main work can begin - the most of it will be anyway not the app itself, but the porting of the existing tests. The required karma would include the app itself, ext/*/tests, Zend/tests, tests, and other related areas. Perhaps, the tests able to run in parallel can be marked as such with a new section or alike. There are probably thousands of tests that have to be touched to run in parallel otherwise, be it DB tests utilizing same table name or built-in server based tests that want to occupy same port. If there were such an app merged into master, that supports current test format and can run a mix of tests in old/new ways, it'd allow an organic integration and step-by-step improvement of the test suite. Then also, the other features from your evaluation could be developed while the test porting work is being done. Thanks Anatol
