Re: Stored Proc to Relational Expression

2020-04-21 Thread Ravi Kapoor
Yeah Stamatis, I know that as I can see the github page for linq4j that lots of task is in Backlog and Link-To-SQL Provider is one of them But the intention of the project is to make java objects as Queryable talk to DB. Expressions and various Statement used in linq4j is to generate the different

Re: [DISCUSS] Towards Cascades Optimizer

2020-04-21 Thread Danny Chan
> Is there any recommended approach to make that happen smoothly besides coding and testing work? We need to be aware that the new planner might be co-exist with VolcanoPlanner for 5 or more years, or even never replace VolcanoPlanner. If that is true, i might say the new planner is probably with

Re: Understanding annotations of SqlGroupingFunction

2020-04-21 Thread XING JIN
Hi Vineet ~ +1 on your analysis. Checking below case in agg.iq. We can see that the behavior of GROUPING function in Calcite is the same as Hive. # GROUPING in SELECT clause of CUBE query select deptno, job, count(*) as c, grouping(deptno) as d, grouping(job) j, grouping(deptno, job) as x from "

[jira] [Created] (CALCITE-3949) RelDistributions.of() and RelCollations.of() should canonize trait instance

2020-04-21 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3949: -- Summary: RelDistributions.of() and RelCollations.of() should canonize trait instance Key: CALCITE-3949 URL: https://issues.apache.org/jira/browse/CALCITE-3949 Pro

Re: [DISCUSS] Towards Cascades Optimizer

2020-04-21 Thread Haisheng Yuan
Hi Andrii, > Obviously, from what is written here, I could guess that this would require > me to change my physical planning rules, even if only by implementing a > marker interface. You don't need to change your physical rules, it will be treated as equal as logical rules and be applied togeth

Calcite-Master - Build # 1712 - Still Failing

2020-04-21 Thread Apache Jenkins Server
The Apache Jenkins build system has built Calcite-Master (build #1712) Status: Still Failing Check console output at https://builds.apache.org/job/Calcite-Master/1712/ to view the results.

Re: RelMetadataQuery.getRowCount stackoverflow

2020-04-21 Thread Haisheng Yuan
I think ProjectMergeRule related cyclic triggering has been walked around by CALCITE-3774 [1]. Haisheng [1] https://issues.apache.org/jira/browse/CALCITE-3774 On 2020/04/21 21:20:35, Stamatis Zampetakis wrote: > I've seen it also quite a few times and most often the reason was cycles in > the

Re: Building a Calcite Adapter

2020-04-21 Thread Stamatis Zampetakis
Hi Jon, Thanks for your kind words. I'm sure people working on the project are very happy to receive some positive feedback for their work from time to time :) I had a quick look on your project and definitely looks interesting. If your engine (Crux) uses better join algorithms than the ones pro

Calcite-Master - Build # 1711 - Failure

2020-04-21 Thread Apache Jenkins Server
The Apache Jenkins build system has built Calcite-Master (build #1711) Status: Failure Check console output at https://builds.apache.org/job/Calcite-Master/1711/ to view the results.

Re: RelMetadataQuery.getRowCount stackoverflow

2020-04-21 Thread Stamatis Zampetakis
I've seen it also quite a few times and most often the reason was cycles in the subsets that were triggering some rules (e.g., ProjectMergeRule [1]) to fire an infinite number of times [2, 3, 4]. Best, Stamatis [1] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcit

Re: Understanding annotations of SqlGroupingFunction

2020-04-21 Thread Vineet G
I expect that the user behavior for the GROUPING in both hive and calcite is same. It’s just the documentation which is a bit confusing. e.g. comment line on grouping : if both deptno and gender are being grouped should really mean that the row which represents the grand total i.e without grou

[jira] [Created] (CALCITE-3948) Improve operand's RelSubset matching handling in VolcanoRuleCall

2020-04-21 Thread Botong Huang (Jira)
Botong Huang created CALCITE-3948: - Summary: Improve operand's RelSubset matching handling in VolcanoRuleCall Key: CALCITE-3948 URL: https://issues.apache.org/jira/browse/CALCITE-3948 Project: Calcite

Re: Stored Proc to Relational Expression

2020-04-21 Thread Stamatis Zampetakis
Hi Ravi, As Julian already mentioned, we are quite far from what you would like to achieve. Nevertheless, I would like to mention that you can model certain kind of loops and recursion using the RepeatUnion [1] and Spool [2] relational expressions. Note that these APIs are experimental. Moreover

[jira] [Created] (CALCITE-3947) AbstractRelOptPlanner.classes should be LinkedHashSet so that rule match order is deterministic across runs

2020-04-21 Thread Botong Huang (Jira)
Botong Huang created CALCITE-3947: - Summary: AbstractRelOptPlanner.classes should be LinkedHashSet so that rule match order is deterministic across runs Key: CALCITE-3947 URL: https://issues.apache.org/jira/browse

Re: [DISCUSS] Towards Cascades Optimizer

2020-04-21 Thread Андрей Цвелодуб
Hello Haisheng, > To keep backward compatibility, all the un-marked rules will be treated as logical rules, except rules that uses AbstractConverter as rule operand, these rules still need to applied top-down, or random order. Obviously, from what is written here, I could guess that this would req

Re: [DISCUSS] Towards Cascades Optimizer

2020-04-21 Thread Haisheng Yuan
Hi Andrii, > I guess changing the planner would lead to changes in tons of rules and even > more tests. Obviously you didn't read through my email. You are not required to do any changes to your rule if you don't want to, but if you do, just need to mark the rule to tell planner whether it is a

[jira] [Created] (CALCITE-3946) Add parser support for MULTISET/SET and VOLATILE modifiers in CREATE TABLE statements

2020-04-21 Thread dasch (Jira)
dasch created CALCITE-3946: -- Summary: Add parser support for MULTISET/SET and VOLATILE modifiers in CREATE TABLE statements Key: CALCITE-3946 URL: https://issues.apache.org/jira/browse/CALCITE-3946 Project:

Re: Understanding annotations of SqlGroupingFunction

2020-04-21 Thread Julian Hyde
Suppose we have one row that represents the total for department 10, and another that represents the grand total of all departments. Which row would we say that department is “grouped” (in Calcite’s parlance) or “aggregated” in (Hive’s parlance)? I find the terms confusing. It’s possible that C

[jira] [Created] (CALCITE-3945) Update Streaming documentation

2020-04-21 Thread Rui Wang (Jira)
Rui Wang created CALCITE-3945: - Summary: Update Streaming documentation Key: CALCITE-3945 URL: https://issues.apache.org/jira/browse/CALCITE-3945 Project: Calcite Issue Type: Sub-task

[jira] [Created] (CALCITE-3944) Move dumpSets and dumpGraphviz out of VolcanoPlanner

2020-04-21 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3944: -- Summary: Move dumpSets and dumpGraphviz out of VolcanoPlanner Key: CALCITE-3944 URL: https://issues.apache.org/jira/browse/CALCITE-3944 Project: Calcite

[jira] [Created] (CALCITE-3943) Remove the JSON functions keyword from parser

2020-04-21 Thread Danny Chen (Jira)
Danny Chen created CALCITE-3943: --- Summary: Remove the JSON functions keyword from parser Key: CALCITE-3943 URL: https://issues.apache.org/jira/browse/CALCITE-3943 Project: Calcite Issue Type: I

Re: [DISCUSS] Towards Cascades Optimizer

2020-04-21 Thread Chunwei Lei
Haisheng and Xiening, thanks for sharing these wonderful ideas. I believe this will be a huge improvement and definitely benefits all users. >From my experience of upgrading calcite version, there are always some changes in the new version which may lead to unexpected behavior due to a lack of eno

[jira] [Created] (CALCITE-3942) Move type-coercion configurations into SqlValidator.Config

2020-04-21 Thread Danny Chen (Jira)
Danny Chen created CALCITE-3942: --- Summary: Move type-coercion configurations into SqlValidator.Config Key: CALCITE-3942 URL: https://issues.apache.org/jira/browse/CALCITE-3942 Project: Calcite