[
https://issues.apache.org/jira/browse/IMPALA-14401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18020494#comment-18020494
]
ASF subversion and git services commented on IMPALA-14401:
----------------------------------------------------------
Commit 0fe8de0f3f5b1c8d9590de35f27f7cc244fb66cc in impala's branch
refs/heads/master from jasonmfehr
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=0fe8de0f3 ]
IMPALA-14401: Deflake/Improve OpenTelemetry Tracing Tests
Contains the following improvements to the Impala queries as
OpenTelemetry traces custom cluster tests:
1. Supporting code for asserting traces was moved to
'tests/util/otel_trace.py'. The moved code was modified to remove
all references to 'self'. Since this code used
'self.assert_impalad_log_contains', it had to be modified so the
caller provides the correct log file path to search. The
'__find_span_log' function was updated to call a new generic file
grep function to run the necessary log file search regex. All
other code was moved unmodified.
2. Classes 'TestOtelTraceSelectsDMLs' and 'TestOtelTraceDDLs'
contained a total of 11 individual tests that used the
'unique_database' fixture. When this fixture is used in a test, it
results in two DDLs being run before the test to drop/create the
database and one DDL being run after the test to drop the database.
These classes now create a test database once during 'setup_class'
and drop it once during 'teardown_class' because creating a new
database for each test was unnecessary. This change dropped test
execution time from about 97 seconds to about 77 seconds.
3. Each test now has comments describing what the test is asserting.
4. The unnecessary sleep in 'test_query_exec_fail' was removed saving
five seconds of test execution time.
5. New test 'test_dml_insert_fail' added. Previously, the situation
where an insert DML failed was not tested. The test passed without
any changes to backend code.
6. Test 'test_ddl_createtable_fail' is greatly simplified by using a
debug action to fail the query instead of multiple parallel
queries where one dropped the database the other was inserting
into. The simplified setup eliminated test flakiness caused by
timing differences and sped up test execution by about 5 seconds.
7. Fixed test flakiness was caused by timing issues. Depending on
when the close process was initiated, span events are sometimes in
the QueryExecution span and sometimes in the Close span. Test
assertions cannot handle these situations. All span event
assertions for the Close span were removed. IMPALA-14334 will fix
these assertions.
8. The function 'query_id_from_ui' which retrieves the query profile
using the Impala debug ui now makes multiple attempts to retrieve
the query. In slower test situations, such as ASAN, the query may
not yet be available when the function is called initially which
used to cause tests to fail. This test flakiness is now eliminated
through the addition of the retries.
Testing accomplished by running tests in test_otel_trace.py both
locally and in a full Jenkins build.
Generated-by: Github Copilot (Claude Sonnet 3.7)
Change-Id: I0c3e0075df688c7ae601c6f2e5743f56d6db100e
Reviewed-on: http://gerrit.cloudera.org:8080/23385
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Deflake/Improve OTel Custom Cluster Tests
> -----------------------------------------
>
> Key: IMPALA-14401
> URL: https://issues.apache.org/jira/browse/IMPALA-14401
> Project: IMPALA
> Issue Type: Improvement
> Affects Versions: Impala 5.0.0
> Reporter: Jason Fehr
> Assignee: Jason Fehr
> Priority: Critical
>
> Several improvements can be made to the "test_otel_trace.py" custom cluster
> test to speed up the tests and deflake them:
> 1. In test_otel_trace.py, the test_ddl_createtable_fail has a potential to be
> flaky. In the test, the second coordinator that drops the database may run
> that DDL too quickly before the create database operation executed on the
> first coordinator propagates to the second coordinator.
> In the test, first retrieve the number of databases from the
> "catalog.num-databases" metric:
> {code:python}
> num_dbs =
> self.cluster.get_first_impalad().service.get_metric_value("catalog.num-databases")
> {code}
> Then, before the drop database DDL, wait for the second coordinator's
> "catalog.num-databases" metric to increment by 1:
> {code:python}
> self.cluster.impalads[1].service.wait_for_metric_value("catalog.num-databases",
> num_dbs + 1, 60)
> {code}
> The other option for this test is to use a debug action to fail the DDL.
> 2. The "unique_database" fixture is used across multiple tests. This causes
> 3 extra queries to be run each time this fixture is used (during setup --
> drop db and create db, during teardown -- drop db). Create the database
> during test class setup and drop it during test class teardown.
> 3. There is no test for insert dml failure.
> 4. Several tests are missing comments explaining their purpose.
> 5. The code to assert traces would be better in tests/util/otel_trace.py.
> Having it in test_otel_trace.py distracts from the test functionality.
> 6. test_query_exec_fail has an unnecessary sleep in the debug action.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
