Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/21541 )
Change subject: IMPALA-12906: Incorporate scan range information into the tuple cache key ...................................................................... Patch Set 2: (6 comments) http://gerrit.cloudera.org:8080/#/c/21541/1/be/src/exec/hdfs-scan-node-base.cc File be/src/exec/hdfs-scan-node-base.cc: http://gerrit.cloudera.org:8080/#/c/21541/1/be/src/exec/hdfs-scan-node-base.cc@171 PS1, Line 171: const string& avro_schema_str = hdfs_table_->avro_schema(); > Good point, dropped this extra field Ack http://gerrit.cloudera.org:8080/#/c/21541/1/be/src/exec/tuple-cache-node.h File be/src/exec/tuple-cache-node.h: http://gerrit.cloudera.org:8080/#/c/21541/1/be/src/exec/tuple-cache-node.h@62 PS1, Line 62: // This combination is unique for a given fragment instance. > Changed this to drop input_scan_node_ids_ and compile_time_key_ as fields o I think we have a pattern of making unnecessary copy of Thrift fields. May be worth a pass at some point. http://gerrit.cloudera.org:8080/#/c/21541/1/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java: http://gerrit.cloudera.org:8080/#/c/21541/1/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1959 PS1, Line 1959: // There reason we skip setting the deterministic scan range assignment field for > I added a comment here. Basically, for computing the tuple cache key, this Done http://gerrit.cloudera.org:8080/#/c/21541/2/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java: http://gerrit.cloudera.org:8080/#/c/21541/2/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1959 PS2, Line 1959: // There reason we skip setting the deterministic scan range assignment field for nit: "The" not "There" http://gerrit.cloudera.org:8080/#/c/21541/2/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@2065 PS2, Line 2065: // To avoid polluting the explain output, only add this if this is mt_dop>0 and nit: "this" is used in two different ways here. I'd shorten it to "only add this if mt_dop>0 and..." http://gerrit.cloudera.org:8080/#/c/21541/1/tests/custom_cluster/test_tuple_cache.py File tests/custom_cluster/test_tuple_cache.py: http://gerrit.cloudera.org:8080/#/c/21541/1/tests/custom_cluster/test_tuple_cache.py@196 PS1, Line 196: result = self.cached_query_w_debugaction(query, > Added a base class for these tests and pulled these tests out into their ow Ack -- To view, visit http://gerrit.cloudera.org:8080/21541 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibe298fff0f644ce931a2aa934ebb98f69aab9d34 Gerrit-Change-Number: 21541 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Yida Wu <[email protected]> Gerrit-Comment-Date: Fri, 28 Jun 2024 23:46:42 +0000 Gerrit-HasComments: Yes
