Re: Question about parallel query planning

2021-03-09 Thread Vladimir Ozerov
Hi Jihoon, I would say that 5 sec could be actually a pretty good result at such. Did you have a chance to check which exact rules contributed to the planning time? You may inject a listener to VolcanoPlanner to check that. Regards, Vladimir Вт, 9 марта 2021 г. в 05:37, Jihoon Son : > Hi all, >

Re: Question about parallel query planning

2021-03-09 Thread Vladimir Ozerov
*at such = at such scale Вт, 9 марта 2021 г. в 11:01, Vladimir Ozerov : > Hi Jihoon, > > I would say that 5 sec could be actually a pretty good result at such. Did > you have a chance to check which exact rules contributed to the planning > time? You may inject a listener to VolcanoPlanner to che

Re: [DISCUSS] Default disable the RexNode normalization(or operands reorder)

2021-03-09 Thread Stamatis Zampetakis
I think we agree on this Vladimir, although I don't want to focus too much on the approach of Hive which I used mostly to have a concrete example but rather on the fact that normalization should be customizable. If downstream projects can pass their own normalizer in the planner or possibly in Rel

Re: [DISCUSS] Default disable the RexNode normalization(or operands reorder)

2021-03-09 Thread Vladimir Sitnikov
Stamatis>If downstream projects can pass their own normalizer in the planner or Stamatis>possibly in RelBuilder then I guess we can cover pretty much every use Stamatis>case. This is my understanding as well. Vladimir

[jira] [Created] (CALCITE-4528) Upgrade Avatica version to 1.18.0

2021-03-09 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created CALCITE-4528: Summary: Upgrade Avatica version to 1.18.0 Key: CALCITE-4528 URL: https://issues.apache.org/jira/browse/CALCITE-4528 Project: Calcite Issue T

Re: [DISCUSS] Towards Calcite 1.27.0

2021-03-09 Thread Stamatis Zampetakis
Many thanks for moving this forward Julian, much appreciated. At the moment the main blocker is the Avatica release ([1,2]) that in turn waits for CALCITE-4503 [3]. It would be great if somebody has some cycles to review and merge the respective PR [4]. Best, Stamatis [1] https://issues.apache.o

[jira] [Created] (CALCITE-4529) JoinUnionTransposeRule规则里string字段在join里会被转为varchar导致Cannot add expression of different type to set

2021-03-09 Thread panhua (Jira)
panhua created CALCITE-4529: --- Summary: JoinUnionTransposeRule规则里string字段在join里会被转为varchar导致Cannot add expression of different type to set Key: CALCITE-4529 URL: https://issues.apache.org/jira/browse/CALCITE-4529

Re: Question about parallel query planning

2021-03-09 Thread Jihoon Son
Hi Vladimir, thank you for your reply. 5 sec might not be bad from a technical point of view, but our user wants their queries to finish in 2 - 3 seconds including planning time. The actual query execution time for this particular query was 2 seconds which can be improved to 20 ms in my testing. H

Re: [DISCUSS] Default disable the RexNode normalization(or operands reorder)

2021-03-09 Thread Julian Hyde
> normalization should be customizable. I'll make the counter-argument. If people can't agree what should be the "right" normalization, then it isn't normalization, it's optimization. A good design doesn't need 100 ways to customize. It needs just a few powerful ones. We have rewrite rules, and w

Re: Question about parallel query planning

2021-03-09 Thread Julian Hyde
At a high level, the Volcano/Cascades planning algorithm is amenable to parallelization. It uses a "work queue" (of matched rules that have not been applied yet) and each task is additive (adds relational expressions to the graph of relational expressions and their equivalence sets, and things are

Re: Make RelNode attribute order part of MEMO

2021-03-09 Thread Julian Hyde
I investigated something similar a long time ago. We noticed that a lot of trivial Project operators were being generated to compensate for field re-ordering due to join transposition. And so the idea was to allow each RelNode (and especially Join) to permute its output fields. Here is the case: h