Title: [90650] trunk/Tools
- Revision
- 90650
- Author
- [email protected]
- Date
- 2011-07-08 12:05:17 -0700 (Fri, 08 Jul 2011)
Log Message
Teach buildbot to figure out how many webkitpy/webkitperl tests failed
Fixes <http://webkit.org/b/64192> It's hard to tell how many test-webkitpy/test-webkitperl
tests failed when looking at build.webkit.org
Reviewed by Eric Seidel.
* BuildSlaveSupport/build.webkit.org-config/master.cfg:
(TestWithFailureCount): New class that represents a test build step which has an associated
failure count. Eventually we should move more of our test classes to deriving from this.
(TestWithFailureCount.countFailures): Method for subclasses to override to say how many
failures occurred.
(TestWithFailureCount.commandComplete):
(TestWithFailureCount.evaluateCommand):
(TestWithFailureCount.getText):
(TestWithFailureCount.getText2):
These were all based on RunGtkAPITests.
(RunPythonTests): Changed to inherit from TestWithFailureCount.
(RunPythonTests.countFailures): Parses the test-webkitpy output looking for the count of
failures.
(RunPerlTests): Changed to inherit from TestWithFailureCount.
(RunPerlTests.countFailures): Parses the test-webkitperl output looking for the count of
failures.
Modified Paths
Diff
Modified: trunk/Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg (90649 => 90650)
--- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg 2011-07-08 18:59:27 UTC (rev 90649)
+++ trunk/Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg 2011-07-08 19:05:17 UTC (rev 90650)
@@ -331,20 +331,74 @@
self.setCommand(self.command + ['--no-build'])
return shell.Test.start(self)
-class RunPythonTests(shell.Test):
+
+class TestWithFailureCount(shell.Test):
+ failedTestsFormatString = "%d tests failed"
+
+ def countFailures(self, cmd):
+ return 0
+
+ def commandComplete(self, cmd):
+ shell.Test.commandComplete(self, cmd)
+ self.failedTestCount = self.countFailures(cmd)
+
+ def evaluateCommand(self, cmd):
+ if self.failedTestCount:
+ return FAILURE
+
+ if cmd.rc != 0:
+ return FAILURE
+
+ return SUCCESS
+
+ def getText(self, cmd, results):
+ return self.getText2(cmd, results)
+
+ def getText2(self, cmd, results):
+ if results != SUCCESS and self.failedTestCount:
+ return [self.failedTestsFormatString % self.failedTestCount]
+
+ return [self.name]
+
+
+class RunPythonTests(TestWithFailureCount):
name = "webkitpy-test"
description = ["python-tests running"]
descriptionDone = ["python-tests"]
command = ["python", "./Tools/Scripts/test-webkitpy"]
+ failedTestsFormatString = "%d python tests failed"
+ def countFailures(self, cmd):
+ logText = cmd.logs['stdio'].getText()
+ # We're looking for the line that looks like this: FAILED (failures=2, errors=1)
+ regex = re.compile(r'^FAILED \((?P<counts>[^)]+)\)')
+ for line in logText.splitlines():
+ match = regex.match(line)
+ if not match:
+ continue
+ return sum(int(component.split('=')[1]) for component in match.group('counts').split(', '))
+ return 0
-class RunPerlTests(shell.Test):
+
+class RunPerlTests(TestWithFailureCount):
name = "webkitperl-test"
description = ["perl-tests running"]
descriptionDone = ["perl-tests"]
command = ["perl", "./Tools/Scripts/test-webkitperl"]
+ failedTestsFormatString = "%d perl tests failed"
+ def countFailures(self, cmd):
+ logText = cmd.logs['stdio'].getText()
+ # We're looking for the line that looks like this: Failed 2/19 test programs. 5/363 subtests failed.
+ regex = re.compile(r'^Failed \d+/\d+ test programs\. (?P<count>\d+)/\d+ subtests failed\.')
+ for line in logText.splitlines():
+ match = regex.match(line)
+ if not match:
+ continue
+ return int(match.group('count'))
+ return 0
+
class RunBindingsTests(shell.Test):
name = "bindings-generation-tests"
description = ["bindings-tests running"]
Modified: trunk/Tools/ChangeLog (90649 => 90650)
--- trunk/Tools/ChangeLog 2011-07-08 18:59:27 UTC (rev 90649)
+++ trunk/Tools/ChangeLog 2011-07-08 19:05:17 UTC (rev 90650)
@@ -1,5 +1,33 @@
2011-07-08 Adam Roben <[email protected]>
+ Teach buildbot to figure out how many webkitpy/webkitperl tests failed
+
+ Fixes <http://webkit.org/b/64192> It's hard to tell how many test-webkitpy/test-webkitperl
+ tests failed when looking at build.webkit.org
+
+ Reviewed by Eric Seidel.
+
+ * BuildSlaveSupport/build.webkit.org-config/master.cfg:
+ (TestWithFailureCount): New class that represents a test build step which has an associated
+ failure count. Eventually we should move more of our test classes to deriving from this.
+ (TestWithFailureCount.countFailures): Method for subclasses to override to say how many
+ failures occurred.
+
+ (TestWithFailureCount.commandComplete):
+ (TestWithFailureCount.evaluateCommand):
+ (TestWithFailureCount.getText):
+ (TestWithFailureCount.getText2):
+ These were all based on RunGtkAPITests.
+
+ (RunPythonTests): Changed to inherit from TestWithFailureCount.
+ (RunPythonTests.countFailures): Parses the test-webkitpy output looking for the count of
+ failures.
+ (RunPerlTests): Changed to inherit from TestWithFailureCount.
+ (RunPerlTests.countFailures): Parses the test-webkitperl output looking for the count of
+ failures.
+
+2011-07-08 Adam Roben <[email protected]>
+
Ensure $CHANGE_LOG_EMAIL_ADDRESS is set when testing webkitpy's commit-log-editor integration
Fixes <http://webkit.org/b/64180> REGRESSION (r90564): test-webkitpy failing on multiple
_______________________________________________
webkit-changes mailing list
[email protected]
http://lists.webkit.org/mailman/listinfo.cgi/webkit-changes
