[
https://issues.apache.org/jira/browse/IMPALA-13624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17951538#comment-17951538
]
ASF subversion and git services commented on IMPALA-13624:
----------------------------------------------------------
Commit 7ad7a86c0eedbb2d056b8f8dfb3d4d7cfc145993 in impala's branch
refs/heads/master from Surya Hebbar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=7ad7a86c0 ]
IMPALA-13624: Implement textual representation for aggregate event sequences
This adds support for a summarized textual representation of timestamps
for the event sequences present in the aggregated profile.
With the verbose format present in profile V1 and V2, it becomes
difficult to analyze an event's timestamps across instances.
The event sequences are now displayed in a histogram format, based on
the number of timestamps present, in order to support an easier view
for skew analysis and other possible use cases.
(i.e. based on json_profile_event_timestamp_limit)
The summary generated from aggregated instance-level timestamps
(i.e. IMPALA-13304) is used to achieve this within the profile V2,
which covers the possbility of missing events.
Example,
Verbosity::DEFAULT
json_profile_event_timestamp_limit = 5 (default)
Case #1, Number of instances exceeded limit
Node Lifecycle Event Timeline Summary :
- Open Started (4s880ms):
Min: 2s312ms, Avg: 3s427ms, Max: 4s880ms, Count: 12
HistogramCount: 4, 4, 0, 0, 4
Case #2, Number of instances within the limit
Node Lifecycle Event Timeline:
- Open Started: 5s885ms, 1s708ms, 3s434ms
- Open Finished: 5s885ms, 1s708ms, 3s435ms
- First Batch Requested: 5s885ms, 1s708ms, 3s435ms
- First Batch Returned: 6s319ms, 2s123ms, 3s570ms
- Last Batch Returned: 7s878ms, 2s123ms, 3s570ms
With Verbosity::EXTENDED or more, all events and timestamps are printed
with full verbosity as before.
Tests:
For test_profile_tool.py, updated the generated outputs for text
and JSON profiles.
Change-Id: I4bcc0e2e7fccfa8a184cfa8a3a96d68bfe6035c0
Reviewed-on: http://gerrit.cloudera.org:8080/22245
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Riza Suminto <[email protected]>
> Implement textual representation of aggregate event sequences
> -------------------------------------------------------------
>
> Key: IMPALA-13624
> URL: https://issues.apache.org/jira/browse/IMPALA-13624
> Project: IMPALA
> Issue Type: New Feature
> Reporter: Surya Hebbar
> Assignee: Surya Hebbar
> Priority: Major
> Attachments: timeline_representation.png
>
>
> Currently, the node lifecycle event timeline among fragment instances is
> expressed in list format, without any form of summarization.
> Now, with the inclusion of aggregate metrics in the experimental profile in
> JSON format, it is possible to display them in a comprehensive format, which
> would help in skew analysis or other such cases.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
