songwdfu opened a new pull request, #15999:
URL: https://github.com/apache/pinot/pull/15999
Implemented knobs as queryOption that controls turning on / off rules per
query.
Currently supported all rules in optProgram. One can specify disabling rule
like:
```
SET plannerRule.skipPinotAggregateReduceFunctionsRule=true;
SET plannerRule.skipAggregateCaseToFilterRule=true;
```
The purpose of these knobs are for users to turn off rules that are taking
too long / produces suboptimal plan. All rules are enabled by default.
The naming convention here is consistent with the rule name shown on query
console from `EXPLAIN PLAN FOR`, it's just `plannerRule.skip<name as shown on
console>`.
The only naming exceptions are `PruneEmptyRules` with pattern `XXX(left)`
and `XXX(right)` are changed to `XXXLeft` and `XXXRight`, and
`plannerRule.skipAggregateJoinTransposeRuleExtended` (this rule will be
introduced in future PR). All constants introduced are declared in
pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java.
(A side note here is the query console prints rule description if it's not
null, else the simple class name. So there are rules whose names are misleading
here, but I'm keeping knob name consistent with them for now. For example,
`PinotEvaluateLiteralRule.Project` shows as `Project` only, and the knob would
be `plannerRule.skipProject`)
A major change to QueryEnvironment is, the optProgram in PlannerContext is
now constructed per-query (to contain only enabled rules), this is acceptable
as creating HepProgram is not expensive. TraitProgram is already created
per-query before this PR.
A logger is added to QueryEnvironment, which will be used to debug parsed /
decorrelated / trimmed plans in the future.
Another minor improvement made in this commit is the introduction of
`PruneEmptyRules.CORRELATE_LEFT_INSTANCE` and its right equivalent into
`PRUNE_RULES`.
These help remove the dummy correlated left / right joins produced by
decorrelating EXISTS / NOT EXISTS subqueries, which Pinot doesn't support.
Several tests are added to QueryCompilationTest and
QueryPlannerRuleOptionsTest to test the new rule and knobs.
P.S.
A complete list of introduced queryOptions, valid values are `true` or
`false`:
```
plannerRule.skipPinotFilterIntoJoinRule
plannerRule.skipFilterAggregateTransposeRule
plannerRule.skipFilterSetOpTransposeRule
plannerRule.skipPinotProjectJoinTransposeRule
plannerRule.skipProjectSetOpTransposeRule
plannerRule.skipFilterProjectTransposeRule
plannerRule.skipPinotJoinConditionPushRule
plannerRule.skipProjectRemoveRule
plannerRule.skipProjectToLogicalProjectAndWindowRule
plannerRule.skipProjectWindowTransposeRule
plannerRule.skipProject // for
PinotEvaluateLiteralRule.Project
plannerRule.skipFilter // for
PinotEvaluateLiteralRule.Filter
plannerRule.skipJoinPushExpressionsRule
plannerRule.skipProjectToSemiJoinRule
plannerRule.skipPinotSeminJoinDistinctProjectRule
plannerRule.skipUnionToDistinctRule
plannerRule.skipAggregateRemoveRule
plannerRule.skipAggregateJoinTransposeRule
plannerRule.skipAggregateUnionAggregateRule
plannerRule.skipPinotAggregateReduceFunctionsRule
plannerRule.skipAggregateCaseToFilterRule
plannerRule.skipFilterIntoJoinRule
plannerRule.skipProjectFilterTransposeRule
plannerRule.skipProjectMergeRule
plannerRule.skipAggregateProjectMergeRule
plannerRule.skipFilterMergeRule
plannerRule.skipSortRemoveRule
plannerRule.skipAggregateJoinTransposeRuleExtended
plannerRule.skipPruneEmptyAggregate
plannerRule.skipPruneEmptyFilter
plannerRule.skipPruneEmptyProject
plannerRule.skipPruneEmptySort
plannerRule.skipUnion // for PruneEmptyRules.Union
plannerRule.skipPruneEmptyCorrelateLeft
plannerRule.skipPruneEmptyCorrelateRight
plannerRule.skipPruneEmptyJoinLeft
plannerRule.skipPruneEmptyJoinRight
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
