924060929 opened a new pull request, #17126: URL: https://github.com/apache/doris/pull/17126
# Proposed changes This pr refactor the rewrite framework from memo to plan tree, and speed up the analyze/rewrite stage. Changes: - abandoned memo in the analysis/rewrite stage, so that we can skip some action , like new GroupExpression, distinct GroupExpression in the memo(high cost), update children to GroupPlan - change the most of rule to static rule, so that we can skip initialize Analyzer/Rewriter and lots of rule in every query. but some rules need context, like visitor rule, create rule at the runtime make it is easy to use, so make `custom` rule can help us to create it. - remove the `logger` field in the Job, Job are generated in large quantities at runtime, we don't need to use logger so save huge time to init logger. - skip some rule as far as possible, e.g. `SelectMaterializedIndexWithoutAggregate`, skip select mv if the table not exist rullup. - add some cache for frequent operation, like get Job.getDisableRule, Plan.getUnboundExpression - new bottom up rewrite rule, it can keep traverse multiple new plan which return by rules, this feature depends on `Plan.mutableState`, it is necessary to add this variable field for plan. if the plan is fully immutable, we must use withXxx to renew the plan and set the state for it, this not only take more runtime overhead and developing workload. another reason is we need multiple mutable state, e.g. whether is applied the rule, whether this plan is manage by the rewrite framework. the good side of mutable state is the efficient, but I suggest we don't direct use mutable state in the rule as far as possible, if we need use it, please wrap the mutable state in the framework to correct modification and release it, a good example is `AppliedAwareRuleCondition`, it can update and get the state: whether this plan is applied to a rule before. - refactor the `EliminateUnnecessaryProject` by CustomRewritor, fix the problem which eliminate some Project which decided the query output order, the case is limit(project), sort(project). ## Checklist(Required) * [ ] Does it affect the original behavior * [x] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
