Wenzhuang Zhu created CALCITE-7422:
--------------------------------------
Summary: Support large plan optimizaion for HepPlanner
Key: CALCITE-7422
URL: https://issues.apache.org/jira/browse/CALCITE-7422
Project: Calcite
Issue Type: Sub-task
Reporter: Wenzhuang Zhu
This subtask includes the following items (controlled by isLargePlanMode):
1.HepPlanner supports graph reuse across different HepPrograms.
Reason: Avoid calling setRoot multiple times when the user has multiple
optimization phases. (HepSubProgram cannot be used because some global state
must be changed between different HepPrograms.)
2.Fast graph GC for the subtree of a discarded vertex (discardedVertex).
Reason: Avoid performing a full-graph traversal during garbage collection. Full
graph GC is O(n). In the worst case, every transform in TOP_DOWN/BOTTOM_UP mode
can trigger a full graph GC.
3.Use a large-plan-friendly graph iterator.
Reason: DepthFirstIterator builds a list containing all nodes in the graph,
which costs O(n) memory and additional time to build the list. This is not
suitable for large plans with 10K+ nodes. BreadthFirstIterator does not build
such a list, and after firedRulesCache is enabled, BreadthFirstIterator and
DepthFirstIterator fire the same number of rules per node. Therefore, for
large-plan mode, using BreadthFirstIterator is generally better than
DepthFirstIterator.
4.Recursivelly contract vertices.
Reason: Large plans provide more opportunities for recursive contraction.
Notes: A very deep plan may cause a stackoverflow when build the graph
(recursive addVertex), but this is not the bottomneck. Compiler or other
modules may need more stacks than optimizer, so we assume the -Xss is large
enough.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
