Re: [QUESTION] Window functions and Relational Operator Tree

2022-04-23 Thread Jacques Nadeau
As you think about this, I might suggest trying to use Substrait as an intermediate serialized representation between ZetaSQL and Calcite. We have already done some work on Substrait <=> Calcite [1] and are hoping some work can be done to add support for ZetaSQL => Substrait [2]. The benefit here i

Re: [QUESTION] Window functions and Relational Operator Tree

2022-04-22 Thread Miguel Hernández Sandoval
Thanks Gavin/Julian, this is very helpful, I really appreciate it. About contributing this feature as a module in Calcite, I'll let know the owners of this translation feature, I think they know it better and could contribute with more knowledge and experience. Mike On Thu, Apr 21, 2022 at 4:36 P

Re: [QUESTION] Window functions and Relational Operator Tree

2022-04-21 Thread Julian Hyde
It’s complicated. Windowed aggregates start off as instances of RexOver expressions inside a Project and are later converted (by ProjectToWindowRule [1]) into Window relational operators. Both of these representations exist in RelNode-land and I’m not sure which should be considered ‘canonical’

Re: [QUESTION] Window functions and Relational Operator Tree

2022-04-20 Thread Julian Hyde
It’s very interesting that you want to convert ZetaSQL to and from Calcite. I think other people will be interested in this. (It would help some of the things I am doing in my $dayjob at Google.) Would you consider contributing this as a new module in Calcite? Or creating a standalone project on

Re: [QUESTION] Window functions and Relational Operator Tree

2022-04-20 Thread Gavin Ray
This might help: https://github.com/apache/calcite/blob/e08ca8c5df5dee154ae1a1e6adfa2f9ce693ccf7/core/src/test/java/org/apache/calcite/plan/RelWriterTest.java#L570-L595 And it's JSON + text representations: https://github.com/apache/calcite/blob/e08ca8c5df5dee154ae1a1e6adfa2f9ce693ccf7/core/src/

[QUESTION] Window functions and Relational Operator Tree

2022-04-20 Thread Miguel Hernández Sandoval
Hi everyone, I'm currently working on a converter from a ZetaSQL AST to a CalciteSQL Relational Operator Tree as part of a BeamSQL feature for Apache Beam. Right now, I'm trying to build a Relational Operator Tree that represents a window function (e.g. "*select SUM(a) over(), COUNT(b) over() From