Hello Kurt Deschler, Yida Wu, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21698
to look at the new patch set (#17).
Change subject: IMPALA-13186: Tag query option scope for tuple cache
......................................................................
IMPALA-13186: Tag query option scope for tuple cache
Constructs a hash of non-default query options that are relevant to
query results; by default query options are included in the hash.
Passes this hash to the frontend for inclusion in the tuple cache key
on plan leaf nodes (which will be included in parent key calculation).
Modifies MurmurHash3 to be re-entrant so the backend can construct a
hash incrementally. This is slightly slower but more memory efficient
than accumulating all hash inputs in a contiguous array first.
Adds TUPLE_CACHE_EXEMPT_QUERY_OPT_FN to mark query options that can be
ignored when calculating a tuple cache hash.
Adds startup flag 'tuple_cache_exempt_query_options' as a safety valve
for query options that might be important to exempt that we missed.
Removes duplicate printing logic for query options from child-query.cc
in favor of re-using TQueryOptionsToMap, which does the same thing.
Cleans up query-options.cc helpers so they're static and reduces
duplicate printing logic.
Adds test that different values for a relevant query option use
different cache entries. Adds startup flag
'tuple_cache_ignore_query_options' to omit query options for testing
certain tuple cache failure modes, where we need to use debug actions.
Change-Id: I1f4802ad9548749cd43df8848b6f46dca3739ae7
---
M be/src/service/child-query.cc
M be/src/service/impala-server.cc
M be/src/service/query-options.cc
M be/src/service/query-options.h
M be/src/thirdparty/datasketches/MurmurHash3.h
M common/thrift/ImpalaService.thrift
M common/thrift/Query.thrift
M fe/src/main/java/org/apache/impala/planner/PlanNode.java
M fe/src/main/java/org/apache/impala/planner/TupleCachePlanner.java
M tests/custom_cluster/test_tuple_cache.py
10 files changed, 277 insertions(+), 124 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/98/21698/17
--
To view, visit http://gerrit.cloudera.org:8080/21698
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4802ad9548749cd43df8848b6f46dca3739ae7
Gerrit-Change-Number: 21698
Gerrit-PatchSet: 17
Gerrit-Owner: Michael Smith <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Kurt Deschler <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Reviewer: Yida Wu <[email protected]>
