Re: [DISCUSS] Adding a option for planner to decide which join reorder rule to choose

+1 for the option and the default value. Best, Jark > 2023年1月10日 16:24，yh z 写道： > > Hi all, > > Thanks for yours reply. > > After receiving your comments and making targeted modifications. The > conclusion is that the option > "table.optimizer.bushy-join-reorder-threshold" can be added. Rele

Re: [DISCUSS] Adding a option for planner to decide which join reorder rule to choose

Hi all, Thanks for yours reply. After receiving your comments and making targeted modifications. The conclusion is that the option "table.optimizer.bushy-join-reorder-threshold" can be added. Relevant PR [1] has been submitted. Sincerely welcome to review it. Thank you. This discussion will be c

Re: [DISCUSS] Adding a option for planner to decide which join reorder rule to choose

Hi Yunhong, Thanks for driving this discuss! This option looks good to me, and looking forward to contributing this rule back to Apache Calcite. Best, Godfrey yh z 于2023年1月5日周四 15:32写道： > > Hi Benchao, > > Thanks for your reply. > > Since our existing test results are based on multiple perfo

Re: [DISCUSS] Adding a option for planner to decide which join reorder rule to choose

Hi Benchao, Thanks for your reply. Since our existing test results are based on multiple performance optimization points on the TPC-DS benchmark[1][2], we haven't separately tested the performance improvement brought by new bushy join reorder rule. I will complete this test recently and update th

Re: [DISCUSS] Adding a option for planner to decide which join reorder rule to choose

Hi Yunhong, Thanks for the updating. And introducing the new bushy join reorder algorithm would be great. And I also agree with the newly added config option "table.optimizer.bushy-join-reorder-threshold" and 12 as the default value. > As for optimization > latency, this is the problem to be sol

Re: [DISCUSS] Adding a option for planner to decide which join reorder rule to choose

Hi Jark, Thanks for your reply. We are going to use bushy join reorder rule and Lopt join reorder rule at the same time. By setting the threshold "table.optimizer.bushy-join-reorder-threshold", when the number of tables need to be reordered is less than/equals this threshold, bushy join reorder r

Re: [DISCUSS] Adding a option for planner to decide which join reorder rule to choose

Hi Yuhong, Thanks for driving the feature. I just have one question. Is the bushy join reorder optimization enabled by default? Does the bushy join reorder will replace the existing Lopt join reorder rule? Besides, I guess the option "table.oprimizer.busy-join-reorder-threshold” should be "t

Re: [DISCUSS] Adding a option for planner to decide which join reorder rule to choose

Hi Benchao, Thanks for your reply. Actually, I mistakenly wrote the name "bushy join reorder" to "busy join reorder". I'm sorry for the trouble bring to you. "Bushy join reorder" means we can build a bushy join tree based on cost model, but now Flink can only build a left-deep tree using Calcite

Re: [DISCUSS] Adding a option for planner to decide which join reorder rule to choose

Hi Yunhong, Thanks for driving this~ I haven't gone deep into the implementation details yet. Regarding the general description, I would ask a few questions firstly: #1, Is there any benchmark results about the optimization latency change compared to current approach? In OLAP scenario, query opt

[DISCUSS] Adding a option for planner to decide which join reorder rule to choose

Hi, devs, I'd like to start a discuss about adding an option called "table.oprimizer.busy-join-reorder-threshold" for planner rule while we try to introduce a new busy join reorder rule[1] into Flink. This join reorder rule is based on dynamic programing[2], which can store all possible intermedi