[
https://issues.apache.org/jira/browse/CALCITE-7422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenzhuang Zhu updated CALCITE-7422:
-----------------------------------
Summary: Support large plan optimizaion mode for HepPlanner (was: Support
large plan optimizaion for HepPlanner)
> Support large plan optimizaion mode for HepPlanner
> --------------------------------------------------
>
> Key: CALCITE-7422
> URL: https://issues.apache.org/jira/browse/CALCITE-7422
> Project: Calcite
> Issue Type: Sub-task
> Reporter: Wenzhuang Zhu
> Priority: Major
>
> This subtask includes the following items (controlled by isLargePlanMode):
> 1.HepPlanner supports graph reuse across different HepPrograms.
> Reason: Avoid calling setRoot multiple times when the user has multiple
> optimization phases. (HepSubProgram cannot be used because some global state
> must be changed between different HepPrograms.)
> 2.Fast graph GC for the subtree of a discarded vertex (discardedVertex).
> Reason: Avoid performing a full-graph traversal during garbage collection.
> Full graph GC is O(n). In the worst case, every transform in
> TOP_DOWN/BOTTOM_UP mode can trigger a full graph GC.
> 3.Use a large-plan-friendly graph iterator.
> Reason: DepthFirstIterator builds a list containing all nodes in the graph,
> which costs O(n) memory and additional time to build the list. This is not
> suitable for large plans with 10K+ nodes. BreadthFirstIterator does not build
> such a list, and after firedRulesCache is enabled, BreadthFirstIterator and
> DepthFirstIterator fire the same number of rules per node. Therefore, for
> large-plan mode, using BreadthFirstIterator is generally better than
> DepthFirstIterator.
> 4.Recursivelly contract vertices.
> Reason: Large plans provide more opportunities for recursive contraction.
> Notes: A very deep plan may cause a stackoverflow when build the graph
> (recursive addVertex), but this is not the bottomneck. Compiler or other
> modules may need more stacks than optimizer, so we assume the -Xss is large
> enough.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
