Surya Hebbar has uploaded a new patch set (#2). (
http://gerrit.cloudera.org:8080/22245 )
Change subject: IMPALA-13624: Implement textual representation for aggregate
event sequences
......................................................................
IMPALA-13624: Implement textual representation for aggregate event sequences
This adds support for textual representation of event sequences present
in the aggregated profile(v2).
The patch also tries[WIP] to refactor parts of RuntimeProfileBase
into RuntimeProfile(i.e. info_strings_) as proposed during implementation
of profile V2 in IMPALA-9846.
With the format present in V1 and V2, it becomes difficult to analyze
an event's timestamps across instances.
The event sequences are now displayed in the following comprehensive
format, which would help with an easier view for skew analysis among
other possible use cases.
Example, Case 1 - Multiple spans
As instances exceeded the limit, aggregated into spans.
Verbosity::DEFAULT
json_profile_event_timestamp_limit = 3
#instances = 12
Node Lifecycle Event Timeline Summary :
- Open Started:
Span #Inst Min Avg Max
33% 8 1s198ms 1s363ms 1s529ms
100% 4 4s197ms 4s197ms 4s197ms
Example, Case 2 - Single Span
As instances exceeded the limit, aggregated into spans.
Verbosity::DEFAULT
json_profile_event_timestamp_limit = 1
#instances = 12
Node Lifecycle Event Timeline Summary :
- Open Started:
Span #Inst Min Avg Max
100% 3 2s213ms 4s410ms 6s810ms
Example, Case 3 - Verbose timestamps
As instances did not exceed the limit, no aggregation.
Verbosity::DEFAULT
json_profile_event_timestamp_limit = 3
#instances = 3
Node Lifecycle Event Timeline:
- Open Started: 4s383ms, 1s895ms, 2s114ms
- Open Finished: 4s383ms, 1s895ms, 2s114ms
- First Batch Requested: 4s383ms, 1s895ms, 2s114ms
- First Batch Returned: 4s600ms, 4s012ms, 3s576ms
- Last Batch Returned: 8s792ms, 4s012ms, 3s576ms
- Closed: 8s793ms, 8s775ms, 8s769ms
Example, Case 4 - Verbose timestamps, no limit
As there is no limit, no aggregation is performed.
Verbosity::DEFAULT
json_profile_event_timestamp_limit = 0
#instances = 3
Node Lifecycle Event Timeline:
- Open Started: 4s383ms, 1s895ms, 2s114ms
- Open Finished: 4s383ms, 1s895ms, 2s114ms
- First Batch Requested: 4s383ms, 1s895ms, 2s114ms
- First Batch Returned: 4s600ms, 4s012ms, 3s576ms
- Last Batch Returned: 8s792ms, 4s012ms, 3s576ms
- Closed: 8s793ms, 8s775ms, 8s769ms
We achieve this by reusing the generation of aggregated event sequences
within the profile V2, as the extensive considerations of missing events
is already implemented there.
With Verbosity::EXTENDED or more, all events and timestamps are printed
with full verbosity as before.
Change-Id: I4bcc0e2e7fccfa8a184cfa8a3a96d68bfe6035c0
---
M be/src/util/runtime-profile.cc
1 file changed, 132 insertions(+), 42 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/22245/2
--
To view, visit http://gerrit.cloudera.org:8080/22245
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4bcc0e2e7fccfa8a184cfa8a3a96d68bfe6035c0
Gerrit-Change-Number: 22245
Gerrit-PatchSet: 2
Gerrit-Owner: Surya Hebbar <sheb...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kdesc...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com>
Gerrit-Reviewer: Surya Hebbar <sheb...@cloudera.com>
