[
https://issues.apache.org/jira/browse/HIVE-25938?focusedWorklogId=725173&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-725173
]
ASF GitHub Bot logged work on HIVE-25938:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 11/Feb/22 14:39
Start Date: 11/Feb/22 14:39
Worklog Time Spent: 10m
Work Description: asolimando commented on a change in pull request #3011:
URL: https://github.com/apache/hive/pull/3011#discussion_r804710821
##########
File path: ql/src/test/results/clientpositive/llap/excluded_rule_explain.q.out
##########
@@ -0,0 +1,112 @@
+PREHOOK: query: EXPLAIN CBO SELECT 1
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN CBO SELECT 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+CBO PLAN:
+HiveProject(_o__c0=[1])
+ HiveTableScan(table=[[_dummy_database, _dummy_table]],
table:alias=[_dummy_table])
+
+PREHOOK: query: EXPLAIN FORMATTED CBO SELECT 1
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN FORMATTED CBO SELECT 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+{"CBOPlan":"{\n \"rels\": [\n {\n \"id\": \"0\",\n \"relOp\":
\"org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableScan\",\n
\"table\": [\n \"_dummy_database\",\n \"_dummy_table\"\n
],\n \"table:alias\": \"_dummy_table\",\n \"inputs\": [],\n
\"rowCount\": 1.0,\n \"avgRowSize\": 129.0,\n \"rowType\": [\n
{\n \"type\": \"BIGINT\",\n \"nullable\": true,\n
\"name\": \"BLOCK__OFFSET__INSIDE__FILE\"\n },\n {\n
\"type\": \"VARCHAR\",\n \"nullable\": true,\n \"precision\":
2147483647,\n \"name\": \"INPUT__FILE__NAME\"\n },\n {\n
\"fields\": [\n {\n \"type\": \"BIGINT\",\n
\"nullable\": true,\n \"name\": \"writeid\"\n
},\n {\n \"type\": \"INTEGER\",\n
\"nullable\": true,\n \"name\": \"bucketid\"\n },\n
{\n \"type\": \"BIGINT\",\n \"nullable\":
true,\n \"name\": \"rowid\"\n }\n ],\n
\"name\": \"ROW__ID\"\n },\n {\n \"type\":
\"BOOLEAN\",\n \"nullable\": true,\n \"name\":
\"ROW__IS__DELETED\"\n }\n ]\n },\n {\n \"id\": \"1\",\n
\"relOp\":
\"org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveProject\",\n
\"fields\": [\n \"_o__c0\"\n ],\n \"exprs\": [\n {\n
\"literal\": 1,\n \"type\": {\n \"type\":
\"INTEGER\",\n \"nullable\": false\n }\n }\n
],\n \"rowCount\": 1.0\n }\n ]\n}"}
+PREHOOK: query: EXPLAIN SELECT 1
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN SELECT 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+STAGE DEPENDENCIES:
+ Stage-0 is a root stage
+
+STAGE PLANS:
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ TableScan
+ alias: _dummy_table
+ Row Limit Per Split: 1
+ Select Operator
+ expressions: 1 (type: int)
+ outputColumnNames: _col0
+ ListSink
+
+PREHOOK: query: EXPLAIN FORMATTED SELECT 1
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN FORMATTED SELECT 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+{"CBOPlan":"{\n \"rels\": [\n {\n \"id\": \"0\",\n \"relOp\":
\"org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableScan\",\n
\"table\": [\n \"_dummy_database\",\n \"_dummy_table\"\n
],\n \"table:alias\": \"_dummy_table\",\n \"inputs\": [],\n
\"rowCount\": 1.0,\n \"avgRowSize\": 0.0,\n \"rowType\": [\n
{\n \"type\": \"BIGINT\",\n \"nullable\": true,\n
\"name\": \"BLOCK__OFFSET__INSIDE__FILE\"\n },\n {\n
\"type\": \"VARCHAR\",\n \"nullable\": true,\n \"precision\":
2147483647,\n \"name\": \"INPUT__FILE__NAME\"\n },\n {\n
\"fields\": [\n {\n \"type\": \"BIGINT\",\n
\"nullable\": true,\n \"name\": \"writeid\"\n
},\n {\n \"type\": \"INTEGER\",\n
\"nullable\": true,\n \"name\": \"bucketid\"\n },\n
{\n \"type\": \"BIGINT\",\n \"nullable\":
true,\n \"name\": \"rowid\"\n }\n ],\n
\"name\": \"ROW__ID\"\n },\n {\n \"type\":
\"BOOLEAN\",\n \"nullable\": true,\n \"name\":
\"ROW__IS__DELETED\"\n }\n ]\n },\n {\n \"id\": \"1\",\n
\"relOp\":
\"org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveProject\",\n
\"fields\": [\n \"_o__c0\"\n ],\n \"exprs\": [\n {\n
\"literal\": 1,\n \"type\": {\n \"type\":
\"INTEGER\",\n \"nullable\": false\n }\n }\n
],\n \"rowCount\": 1.0\n }\n ]\n}","optimizedSQL":"SELECT 1 AS
`_o__c0`\nFROM `_dummy_database`.`_dummy_table`","cboInfo":"Plan optimized by
CBO.","STAGE DEPENDENCIES":{"Stage-0":{"ROOT STAGE":"TRUE"}},"STAGE
PLANS":{"Stage-0":{"Fetch Operator":{"limit:":"-1","Processor
Tree:":{"TableScan":{"alias:":"_dummy_table","database:":"_dummy_database","Row
Limit Per
Split:":"1","table:":"_dummy_table","isTempTable:":"false","OperatorId:":"TS_0","children":{"Select
Operator":{"expressions:":"1 (type:
int)","columnExprMap:":{"_col0":"1"},"outputColumnNames:":["_col0"],"OperatorId:":"SEL_1","children":{"ListSink":{"OperatorId:":"LIST_SINK_3"}}}}}}}}}}
+PREHOOK: query: EXPLAIN CBO SELECT 1
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN CBO SELECT 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+Excluded rules: HiveJoinPushTransitivePredicatesRule|HivePreFilteringRule
+
+CBO PLAN:
+HiveProject(_o__c0=[1])
+ HiveTableScan(table=[[_dummy_database, _dummy_table]],
table:alias=[_dummy_table])
+
+PREHOOK: query: EXPLAIN FORMATTED CBO SELECT 1
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN FORMATTED CBO SELECT 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+{"CBOPlan":"{\n \"rels\": [\n {\n \"id\": \"0\",\n \"relOp\":
\"org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableScan\",\n
\"table\": [\n \"_dummy_database\",\n \"_dummy_table\"\n
],\n \"table:alias\": \"_dummy_table\",\n \"inputs\": [],\n
\"rowCount\": 1.0,\n \"avgRowSize\": 129.0,\n \"rowType\": [\n
{\n \"type\": \"BIGINT\",\n \"nullable\": true,\n
\"name\": \"BLOCK__OFFSET__INSIDE__FILE\"\n },\n {\n
\"type\": \"VARCHAR\",\n \"nullable\": true,\n \"precision\":
2147483647,\n \"name\": \"INPUT__FILE__NAME\"\n },\n {\n
\"fields\": [\n {\n \"type\": \"BIGINT\",\n
\"nullable\": true,\n \"name\": \"writeid\"\n
},\n {\n \"type\": \"INTEGER\",\n
\"nullable\": true,\n \"name\": \"bucketid\"\n },\n
{\n \"type\": \"BIGINT\",\n \"nullable\":
true,\n \"name\": \"rowid\"\n }\n ],\n
\"name\": \"ROW__ID\"\n },\n {\n \"type\":
\"BOOLEAN\",\n \"nullable\": true,\n \"name\":
\"ROW__IS__DELETED\"\n }\n ]\n },\n {\n \"id\": \"1\",\n
\"relOp\":
\"org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveProject\",\n
\"fields\": [\n \"_o__c0\"\n ],\n \"exprs\": [\n {\n
\"literal\": 1,\n \"type\": {\n \"type\":
\"INTEGER\",\n \"nullable\": false\n }\n }\n
],\n \"rowCount\": 1.0\n }\n ]\n}","cboInfo":"Excluded rules:
HiveJoinPushTransitivePredicatesRule|HivePreFilteringRule"}
+PREHOOK: query: EXPLAIN SELECT 1
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN SELECT 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+Excluded rules: HiveJoinPushTransitivePredicatesRule|HivePreFilteringRule
Review comment:
That's not displayed for `EXPLAIN CBO` (see
https://github.com/apache/hive/blob/2b7c3a844156077a8a57f75b548aeb7bc7685cfe/ql/src/java/org/apache/hadoop/hive/ql/parse/ExplainSemanticAnalyzer.java#L204).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 725173)
Time Spent: 0.5h (was: 20m)
> Print excluded rules from CBO
> -----------------------------
>
> Key: HIVE-25938
> URL: https://issues.apache.org/jira/browse/HIVE-25938
> Project: Hive
> Issue Type: Improvement
> Components: CBO
> Affects Versions: 4.0.0
> Reporter: Alessandro Solimando
> Assignee: Alessandro Solimando
> Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> HIVE-25880 introduced a configuration parameter for excluding CBO rules based
> on a regex on their description.
> Calcite logs when a rule is excluded (see
> [AbstractRelOptPlanner.java#L316|https://github.com/apache/calcite/blob/e42b85a45bd16dd58db1546736e653deda5463fe/core/src/main/java/org/apache/calcite/plan/AbstractRelOptPlanner.java#L316]
> and
> [VolcanoRuleCall.java#L169|https://github.com/apache/calcite/blob/e42b85a45bd16dd58db1546736e653deda5463fe/core/src/main/java/org/apache/calcite/plan/volcano/VolcanoRuleCall.java#L169]).
> To ease investigations, this should be complemented by printing the regex
> used (if not blank) in Hive DEBUG logs, and the same in CBO information
> (i.e., EXPLAIN CBO's output).
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
