[
https://issues.apache.org/jira/browse/IMPALA-13415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17896490#comment-17896490
]
ASF subversion and git services commented on IMPALA-13415:
----------------------------------------------------------
Commit 7369ebb8ba02edfedcef071029b7bcd62157f452 in impala's branch
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=7369ebb8b ]
IMPALA-13415: Add a special testing mode to track Calcite progress
This introduces a Calcite report mode for pytests. The mode
changes the behavior for run_test_case() so that it continues
past failures and exports information about the test results
to JSON. The JSON files can then be processed into an HTML
summary by the bin/calcite_report_generator.py. The HTML has
multiple layers of reporting:
1. Reporting on individual tests shows the test section along
with the reported result
2. There are multiple aggregations of these individual results
to have clear summaries and organization for browing:
a. Level 1 is a report at the test function level
(e.g. query_test/test_foo.py::TestFoo::test_foo)
with links to the individaul results.
b. Level 2 is at the test file level (e.g. query_test/test_foo.py)
with links down to the test function level.
c. Level 3 is a top level view with a summary across all the tests
with links down to the test file level.
The errors are classified into different categories (e.g.
parse failures, analysis failures, result differences, etc).
In general, parse failures and unsupported features are lower
priority issues while result differences and runtime failures
are higher priority.
The report is designed to compare two different points in time
to see differences. For example, someone can run tests for a
baseline and then do a comparison run with a new commit.
Testing:
- Ran on the code change for IMPALA-13468 and browsed the results
- Ran tests normally and verified that they continue to work
Change-Id: I453c219c22b6cbc253574e0467d2c0d7b1fac092
Reviewed-on: http://gerrit.cloudera.org:8080/21866
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Michael Smith <[email protected]>
Reviewed-by: Joe McDonnell <[email protected]>
> Add a reporting mode that can proceed past query failures in run_test_case()
> ----------------------------------------------------------------------------
>
> Key: IMPALA-13415
> URL: https://issues.apache.org/jira/browse/IMPALA-13415
> Project: IMPALA
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: Impala 4.5.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Major
>
> For the new Calcite planner, we want to run many queries to track progress in
> adding functionality and fixing issues. One way to do that is to have a mode
> that changes run_test_case() to keep going past individual failure and
> instead export information at the end. These exported files can be combined
> to produce a report that shows improvements / regressions even when there are
> still numerous failures.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
