Hello Steve Carlin, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21866
to look at the new patch set (#8).
Change subject: IMPALA-13415: Add a special testing mode to track Calcite
progress
......................................................................
IMPALA-13415: Add a special testing mode to track Calcite progress
This introduces a Calcite report mode for pytests. The mode
changes the behavior for run_test_case() so that it continues
past failures and exports information about the test results
to JSON. The JSON files can then be processed into an HTML
summary by the bin/calcite_report_generator.py. The HTML has
multiple layers of reporting:
1. Reporting on individual tests shows the test section along
with the reported result
2. There are multiple aggregations of these individual results
to have clear summaries and organization for browing:
a. Level 1 is a report at the test function level
(e.g. query_test/test_foo.py::TestFoo::test_foo)
with links to the individaul results.
b. Level 2 is at the test file level (e.g. query_test/test_foo.py)
with links down to the test function level.
c. Level 3 is a top level view with a summary across all the tests
with links down to the test file level.
The errors are classified into different categories (e.g.
parse failures, analysis failures, result differences, etc).
In general, parse failures and unsupported features are lower
priority issues while result differences and runtime failures
are higher priority.
The report is designed to compare two different points in time
to see differences. For example, someone can run tests for a
baseline and then do a comparison run with a new commit.
Testing:
- Ran on the code change for IMPALA-13468 and browsed the results
- Ran tests normally and verified that they continue to work
Change-Id: I453c219c22b6cbc253574e0467d2c0d7b1fac092
---
A bin/calcite_report_generator.py
M tests/common/impala_test_suite.py
M tests/conftest.py
3 files changed, 548 insertions(+), 117 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/66/21866/8
--
To view, visit http://gerrit.cloudera.org:8080/21866
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I453c219c22b6cbc253574e0467d2c0d7b1fac092
Gerrit-Change-Number: 21866
Gerrit-PatchSet: 8
Gerrit-Owner: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Steve Carlin <[email protected]>
