[
https://issues.apache.org/jira/browse/IMPALA-14334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18020495#comment-18020495
]
ASF subversion and git services commented on IMPALA-14334:
----------------------------------------------------------
Commit 0fe8de0f3f5b1c8d9590de35f27f7cc244fb66cc in impala's branch
refs/heads/master from jasonmfehr
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=0fe8de0f3 ]
IMPALA-14401: Deflake/Improve OpenTelemetry Tracing Tests
Contains the following improvements to the Impala queries as
OpenTelemetry traces custom cluster tests:
1. Supporting code for asserting traces was moved to
'tests/util/otel_trace.py'. The moved code was modified to remove
all references to 'self'. Since this code used
'self.assert_impalad_log_contains', it had to be modified so the
caller provides the correct log file path to search. The
'__find_span_log' function was updated to call a new generic file
grep function to run the necessary log file search regex. All
other code was moved unmodified.
2. Classes 'TestOtelTraceSelectsDMLs' and 'TestOtelTraceDDLs'
contained a total of 11 individual tests that used the
'unique_database' fixture. When this fixture is used in a test, it
results in two DDLs being run before the test to drop/create the
database and one DDL being run after the test to drop the database.
These classes now create a test database once during 'setup_class'
and drop it once during 'teardown_class' because creating a new
database for each test was unnecessary. This change dropped test
execution time from about 97 seconds to about 77 seconds.
3. Each test now has comments describing what the test is asserting.
4. The unnecessary sleep in 'test_query_exec_fail' was removed saving
five seconds of test execution time.
5. New test 'test_dml_insert_fail' added. Previously, the situation
where an insert DML failed was not tested. The test passed without
any changes to backend code.
6. Test 'test_ddl_createtable_fail' is greatly simplified by using a
debug action to fail the query instead of multiple parallel
queries where one dropped the database the other was inserting
into. The simplified setup eliminated test flakiness caused by
timing differences and sped up test execution by about 5 seconds.
7. Fixed test flakiness was caused by timing issues. Depending on
when the close process was initiated, span events are sometimes in
the QueryExecution span and sometimes in the Close span. Test
assertions cannot handle these situations. All span event
assertions for the Close span were removed. IMPALA-14334 will fix
these assertions.
8. The function 'query_id_from_ui' which retrieves the query profile
using the Impala debug ui now makes multiple attempts to retrieve
the query. In slower test situations, such as ASAN, the query may
not yet be available when the function is called initially which
used to cause tests to fail. This test flakiness is now eliminated
through the addition of the retries.
Testing accomplished by running tests in test_otel_trace.py both
locally and in a full Jenkins build.
Generated-by: Github Copilot (Claude Sonnet 3.7)
Change-Id: I0c3e0075df688c7ae601c6f2e5743f56d6db100e
Reviewed-on: http://gerrit.cloudera.org:8080/23385
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Assert QueryExecution/Close Span Events in Tests
> ------------------------------------------------
>
> Key: IMPALA-14334
> URL: https://issues.apache.org/jira/browse/IMPALA-14334
> Project: IMPALA
> Issue Type: Improvement
> Affects Versions: Impala 5.0.0
> Reporter: Jason Fehr
> Priority: Critical
>
> The [custom cluster tests covering Impala
> queries|https://github.com/apache/impala/blob/master/tests/custom_cluster/test_otel_trace.py]
> as OpenTelemetry traces does not assert the event spans for the
> QueryExecution and Close child spans. The reason for this lack of assertions
> is due to the complexity of determining which span events should and should
> not be present. The async nature of query execution makes it particularly
> difficult to know which span events should be present in the QueryExecution
> span and which should be in Close.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
