Re: License header vs package-info.java vs whitespace

2019-12-09 Thread Rui Wang
In Apache Beam we actually using the style you mentioned: package-info.java has one blank line after copyright. I did a test. It turns out Spotless think it's a violation if there are more than one blank line. Spotless can also fix it by deleting extra blank lines and keep only one line. Another i

Re: License header vs package-info.java vs whitespace

2019-12-09 Thread Rui Wang
>The inconsistency with style across different files is sad Yep. And to be clear I am not against an effort to unify the file style. Two tests attached as the following: (this is the test that replace the blank line with random string, and spotless just throw exception) $git diff diff --git a/s

Re: [Discuss] Make flattening on Struct/Row optional

2019-12-09 Thread Rui Wang
should not be too many problems. The > flattening was mainly for the benefit of the runtime. > > > > On Sep 5, 2018, at 11:32 AM, Rui Wang wrote: > > > > Thanks for your helpful response! It seems like disabling the flattening > > will at least affect some rules in o

Re: Quicksql

2019-12-10 Thread Rui Wang
The co-routine model sounds fitting into Streaming cases well. I was thinking how should Enumerable interface work with streaming cases but now I should also check Interpreter. -Rui On Tue, Dec 10, 2019 at 1:33 PM Julian Hyde wrote: > The goal (or rather my goal) for the interpreter is to rep

Re: [Discuss] Make flattening on Struct/Row optional

2019-12-11 Thread Rui Wang
Thanks Stamatis's suggestion. Indeed a recent effort in [1] enhanced the support that reconstructs ROW in the top SELECT, which is supposed to solve the problem. [1]: https://jira.apache.org/jira/browse/CALCITE-3138 On Mon, Dec 9, 2019 at 3:21 PM Rui Wang wrote: > Hello, > >

Re: [Discuss] Make flattening on Struct/Row optional

2019-12-12 Thread Rui Wang
Absolutely. Thanks lgor for the contribution! :) -Rui On Wed, Dec 11, 2019 at 10:54 PM Stamatis Zampetakis wrote: > So basically thanks to Igor :) > > On Wed, Dec 11, 2019 at 9:56 PM Rui Wang wrote: > > > Thanks Stamatis's suggestion. Indeed a recent effort in [1] e

Re: [DISCUSS] Tests vs multiline strings

2019-12-14 Thread Rui Wang
Replace $ with something else will have a large impact. E.g. projects using Calcite might have tests that verify plans by plan.toString() thus $ is exposed in many places. Can we only use Kotlin for tests that does not evaluate plans? I think there are still many tests that does not have "$" can

Re: Calcite-Master - Build # 1503 - Failure

2019-12-15 Thread Rui Wang
This could be fixed by https://github.com/apache/calcite/pull/1663. -Rui On Sun, Dec 15, 2019 at 9:20 PM Apache Jenkins Server < jenk...@builds.apache.org> wrote: > The Apache Jenkins build system has built Calcite-Master (build #1503) > > Status: Failure > > Check console output at https://bui

Re: [VOTE] Release apache-calcite-avatica-1.16.0 (release candidate 1)

2019-12-16 Thread Rui Wang
Mac OS 10.15.2, jdk 1.8.0_181, Gradle 6.0.1 - checked that the git tag has the right commit. - run "./gradlew build -PskipSigning" in git repo successfully. - run "./gradlew clean build" in staged sources successfully. - verified the sha512 hash of apache-calcite-avatica-1.16.0-src.tar.gz by runni

Re: Re: [ANNOUNCE] New Calcite PMC chair: Stamatis Zampetakis

2019-12-18 Thread Rui Wang
Congratulations and Thanks Stamatis! -Rui On Wed, Dec 18, 2019 at 6:52 PM XING JIN wrote: > Congratulations Stamatis! > > -Jin > > Chunwei Lei 于2019年12月19日周四 上午10:33写道: > > > Congratulations Stamatis! > > > > > > Best, > > Chunwei > > > > > > On Thu, Dec 19, 2019 at 9:36 AM Danny Chan wrote

Re: [DISCUSS] Restart the work of introducing TIME/TIMESTAMP WITH TIME ZONE types to Calcite

2019-12-19 Thread Rui Wang
Thanks Zhenghua sharing [1], which really explaining three different semantics of TIMESTAMP and clarified some of my long term confusion about TIMESTAMP. Julian> We need all 3, regardless of what they are called Can I confirm that Calcite already have the following two semantics support: 1. time

Re: [DISCUSS] Restart the work of introducing TIME/TIMESTAMP WITH TIME ZONE types to Calcite

2019-12-20 Thread Rui Wang
(a short value) for every supported time > zone id (a string), > and use a single long value to store the millisecond > and time zone key > > What do you think which one should be our storage solution? > > *Best Regards,* > *Zhenghua Gao* > > &g

Re: [DISCUSS] Avatica 1.16.0 dockerfiles broken. Release 1.17.0?

2019-12-20 Thread Rui Wang
I am less familiar with the broken docker stuff. Depending on how people think about it: If it is not a big problem, one thing could be done is to include it into the release notes with a section that mentions it (I have seen such practice before). -Rui On Fri, Dec 20, 2019 at 8:53 AM Stamatis Z

Re: License header vs package-info.java vs whitespace

2019-12-27 Thread Rui Wang
Thanks for updating! It is useful to know that the empty line has a usage. -Rui On Fri, Dec 27, 2019 at 12:30 PM Vladimir Sitnikov < sitnikov.vladi...@gmail.com> wrote: > Ok, it looks like I found yet another reasoning for having a blank line > between consequent comments. > It simplifies shel

Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-30 Thread Rui Wang
I think the concern of breaking plan comparison has appeared more than once. Not only to this proposal but also to others (e.g. replace "$" with "_" in names). >From a think of another perspective, the widely used practice of string comparison based plan evaluation also reduces the flexibility of

A linq4j question

2019-12-30 Thread Rui Wang
Hi community, Let's say I define a function like this: List myFunc() { // function body // do something // return a list of long } Then by linq4j library, I can Expression ret = Expressions.call( FindMethod("myFunc") ) my question is, how to make the returned expression of the call to

Re: A linq4j question

2019-12-30 Thread Rui Wang
> > For example, assume "myFunc" in SqlFunctions. > > > > * public static List myFunc() {return Arrays.asList(1L, 2L); }* > > > *MethodCallExpression mce = Expressions.call(SqlFunctions.class, > "myFunc");List ret = (List) Expressions.eval

Re: A linq4j question

2019-12-30 Thread Rui Wang
nt(mce2); > > > > *//for (Object obj : res) { // > org.apache.calcite.runtime.SqlFunctions.myFunc2((Long) obj);//}* > ForEachStatement forEachStatement = Expressions.forEach(pe1, pe, s); > > > Best, > Feng > > Rui Wang 于2019年12月31日周二 下午1:42写道: > > &g

Re: Draft board report for January 2020

2020-01-02 Thread Rui Wang
Looks nice! Thank you Stamatis! -Rui On Wed, Jan 1, 2020 at 6:52 PM Matt Wang wrote: > +1, looks good. Thanks~ > > > --- > Best, > Matt Wang > > > On 01/2/2020 09:57,Chunwei Lei wrote: > +1, looks good. > Thanks, Stamatis~~ > > > Best, > Chunwei > > > On Thu, Jan 2, 2020 at 8:41 AM Haisheng

Re: [DISCUSS] Stream tables vs hash joins

2020-01-03 Thread Rui Wang
+1 on having a type of property on Relnode to make know which node is steam or non-stream. In Apache Beam SQL's practice, stream joins are already metadata driven in which if there is one side stream and another side non-stream, we use hash-join like implementation but build that table on the non-

Re: [discuss] Enhanced MATH Function

2020-01-06 Thread Rui Wang
Most of those functions are not mentioned in SQL standard 2011. So It's a good idea to decide if implementing a function by checking if other systems provide it. Thanks for putting those together into the spreadsheet! -Rui On Sun, Jan 5, 2020 at 11:03 PM Chunwei Lei wrote: > Thanks for propos

Re: Support for current_timestamp(p) function till precision 6

2020-01-07 Thread Rui Wang
This email thread is relevant to your question: [1] I think (3) precision just mean supporting up to millisecond precision, which is the precision implemented by Calcite functions. If you don't use Calcite function implementation (e.g. just use Calcite as a planner), you can control the precision

Re: Contributors of 2019

2020-01-13 Thread Rui Wang
I found myself using different emails to get involved in Calcite. Emails discussion: amaliu...@apache.org code contribution: amaliu...@163.com, amaliu...@gmail.com(this was my mistake and it should be amaliu...@163.com). jira: wangc...@gmail.com Question: does the Calcite community in favor of t

[DISCUSS] [CALCITE-3271] EMIT syntax proposal for event-timestamp semantic windowing

2020-01-21 Thread Rui Wang
les above that AND and OR can be used to concat different bool expressions. Please let me know your thoughts and any other way you prefer to continue discussing. [1]: https://arxiv.org/pdf/1905.12133.pdf [2]: https://www.youtube.com/watch?v=TWxSLmkWPm4 Thanks, Rui Wang

Re: [DISCUSS] [CALCITE-3271] EMIT syntax proposal for event-timestamp semantic windowing

2020-01-21 Thread Rui Wang
Forgot to mention the SQL syntax change: SELECT clause [FROM TVF windowing] // windowing happen here [Where clause] [GROUP BY clause] [HAVING clause] [EMIT HAVING clause] // materialization latency control -Rui On Tue, Jan 21, 2020 at 11:21 AM Rui Wang wrote: > Hi community, > >

Re: [DISCUSS] [CALCITE-3271] EMIT syntax proposal for event-timestamp semantic windowing

2020-01-21 Thread Rui Wang
om sub-query. yep that's an option. Then open question from it is if we want to mix "EMIT'' semantic (which is not really a classic SQL filter) or "Filter" semantic (WHERE and HAVING) into existing WHERE and HAVING. I prefer not to, to leave classic DB queries u

Re: [DISCUSS] [CALCITE-3271] EMIT syntax proposal for event-timestamp semantic windowing

2020-01-21 Thread Rui Wang
ere is the EMIT clause? Maybe the most clear position is to put it after FROM. -Rui On Tue, Jan 21, 2020 at 1:09 PM Rui Wang wrote: > > > On Tue, Jan 21, 2020 at 12:34 PM Julian Hyde wrote: > >> Does EMIT HAVING have anything to do with aggregate queries (GROUP BY >>

Re: [DISCUSS] [CALCITE-3271] EMIT syntax proposal for event-timestamp semantic windowing

2020-01-22 Thread Rui Wang
racted > and (100, 8) added. But I think we have to live with that. Otherwise EMIT > semantics get a lot more complicated. > > Julian > > > > On Jan 21, 2020, at 1:24 PM, Rui Wang wrote: > > > > I think there was a big missing in my summary about the position of E

Re: Gradle documentation update

2020-01-23 Thread Rui Wang
Not sure if it is related: the change of reference.md that I submitted in [1] hasn't appeared on [2] Question: does the Calcite reference webpage only get updated when releasing? (if not it might have been overwritten by an older version). [1]: https://github.com/apache/calcite/pull/1587 [2]: ht

Re: [DISCUSS] [CALCITE-3271] EMIT syntax proposal for event-timestamp semantic windowing

2020-01-30 Thread Rui Wang
on between watermark (a operator that leaves for engine to implement) and a column. etc. -Rui On Wed, Jan 22, 2020 at 12:18 PM Rui Wang wrote: > That makes sense. I will remove the aggregation constraint (i.e. EMIT > requires a GROUP BY). Let's say now EMIT should work on any query

Re: Is CBO really useful(you can not estimate cost very well )?

2020-02-04 Thread Rui Wang
>From some real world practice I have heard (but we haven't tried it in BeamSQL yet), CBO becomes bad as the complexity of plan increases as estimation will be hard to be close to reality in the later stages for big data processing. Instead, the runtime optimization that you called out, does work.

Re: Failed to run example for JDBC

2020-02-06 Thread Rui Wang
I tried to run this example but also got some troubles: 1. Intellij cannot run it directly. 2. "java -cp main_class" was not easy to use. Needed to add some jars to classpath and it eventually ended up with a "Class Not Found" error. So I create a gradle task for this example and then it can be

Re: Failed to run example for JDBC

2020-02-06 Thread Rui Wang
en. -Rui On Thu, Feb 6, 2020 at 1:01 PM Rui Wang wrote: > I tried to run this example but also got some troubles: > > 1. Intellij cannot run it directly. > 2. "java -cp main_class" was not easy to use. Needed to add some jars to > classpath and it eventually ended up with

Re: Test results and stacktraces in Gradle output

2020-02-10 Thread Rui Wang
I immediately rebased my PR to use it: it is awesome! -Rui On Mon, Feb 10, 2020 at 11:00 AM Julian Hyde wrote: > The coloring is helpful; thanks! I had been finding it difficult to > identity failures in a the shell when there is a lot of output. > > > On Feb 10, 2020, at 10:52 AM, Vladimir S

Re: Current status of Streaming SQL standardrization

2020-02-10 Thread Rui Wang
Hi Shuyi, Julian should know better on the effort of standardizing streaming SQL into the standard. Regarding the engineering effort: I am trying to drive the implementation in both Calcite and Beam. Current TUMBLE as a table-valued function is working in Calcite master [1] while HOP and SESSION

Re: Calcite Spark Adapter

2020-02-18 Thread Rui Wang
Hello Grant! You may find this previous thread is useful for your question: https://lists.apache.org/thread.html/rac51f3110aeb5df563182d8445137ae374edd6e8d643a42f478f2996%40%3Cdev.calcite.apache.org%3E -Rui On Tue, Feb 18, 2020 at 8:20 PM Grant Guo wrote: > Hi, > > > > I am new to Calcite. I

Re: How to trace a column back to its original column

2020-02-19 Thread Rui Wang
If you have to traverse the whole AST/Rel tree to find original columns, this thread discussed that before: https://lists.apache.org/thread.html/11b66dd7f389f0b6e6bce54ad2d7b49a8a0bdd3be4784c441bfafb81%40%3Cdev.calcite.apache.org%3E -Rui On Wed, Feb 19, 2020 at 7:16 AM Seliverstov Igor wrote:

Re: [Needs help] How to config PGP key of Apache

2020-02-21 Thread Rui Wang
To make your key appear on the committer link. As I recalled you have to login https://id.apache.org/ and fill out "OpenPGP Public Key Primary Fingerprint" -Rui On Fri, Feb 21, 2020 at 10:04 PM Francis Chuang wrote: > Another thing I would try is to upload the key to a few other servers. > Se

Re: [VOTE] Release apache-calcite-1.22.0 (release candidate 2)

2020-02-27 Thread Rui Wang
Hi Danny, Thanks a lot for all Danny's contributions so far on this release! verified on platform: 5.2.17-1rodete3-amd64 - calcite-1.22.0 tag has the right commit which is upon voting. OK - check artifacts 512 hash (run sha512sum locally). OK - run "./gradlew :assemble" from sources extracted

Re: [VOTE] Release apache-calcite-1.22.0 (release candidate 2)

2020-02-27 Thread Rui Wang
> And it outputs: > > gpg: 签名建立于 三 2/26 12:47:33 2020 CST > gpg: 使用 RSA 密钥 9A48922F682AB05D1AE4A3E7C2931E4BDB03D5AE > gpg: 完好的签名,来自于 “Danny Chan (yuzhao.cyz gpg) ” [绝对] > > Best, > Danny Chan > 在 2020年2月28日 +0800 AM5:13,Rui Wang ,写道: > > Hi Danny, > > > >

Re: [ANNOUNCE] New committer: Feng Zhu

2020-02-29 Thread Rui Wang
Congratulations Feng! Well deserved! -Rui On Sat, Feb 29, 2020 at 2:10 AM Feng Zhu wrote: > Thank you everyone for your warm welcome! > Currently I am working at SuperSQL team of Tencent in Shenzhen, Guangdong, > China. > The project relies heavily on Calcite to analyze data residing in thous

Re: jdbc server

2020-03-02 Thread Rui Wang
Could you follow this Calcite tutorial and it might solve your problem: https://calcite.apache.org/docs/tutorial.html -Rui On Mon, Mar 2, 2020, 2:23 AM Mohsen KHaxar wrote: > Hi, > I want to run an SQL engine that receives SQL commands. Then, SQL commands > are processed and some results of in

Re: [VOTE] Release apache-calcite-1.22.0 (release candidate 3)

2020-03-02 Thread Rui Wang
+1 (non-binding) - check sha512 OK - check gpg signature OK (last timely I forgot to import calcite KEYS locally, sorry about that) - ./gradlew build OK - check the rc3 tag with right commit OK - compare git tag files with files extracted from artifacts, There is a difference about LICENSE file. A

Re: STREAM keyword

2020-03-20 Thread Rui Wang
>I'm interested in the STREAM case. My question is, if you use the STREAM >keyword (either after SELECT or at the end), which of the tables is treated >as a stream? Order_item, product or both? Does it matter which table is a steam? if the "STREAM" query runs continuously, the output (table) from

Re: STREAM keyword

2020-03-23 Thread Rui Wang
Yep I think you got the idea. >Therefore to execute this query you need an unbounded memory for >`order_item` and `product` relations, so it's not a good candidate for a >streaming query, but let's put that aside, I'm interested in the semantics. The watermark idea is used to give a hint when da

Re: Promotion factoids about Calcite

2020-03-24 Thread Rui Wang
Maybe add that Apache Calcite empowers cloud computing: AWS Kinesis, Google Cloud Dataflow, etc provides SQL that is using the help of Calcite. (Let me know if I mis-understand those products if they are not using Calcite). -Rui On Tue, Mar 24, 2020 at 11:29 AM Stamatis Zampetakis wrote: > I

[DISCUSS] Remove Guava from public API surface in Calcite

2020-04-03 Thread Rui Wang
Hi community, Calcite exposes Guava on some public surface. Do you think it is worth removing it from the public surface (method, field, etc.)? If we cannot do it, is there a particular reason? -Rui

Re: [DISCUSS] Remove Guava from public API surface in Calcite

2020-04-03 Thread Rui Wang
Thanks for sharing the link! -Rui On Fri, Apr 3, 2020 at 4:57 PM Julian Hyde wrote: > Previous discussion: > > https://lists.apache.org/thread.html/5988348d9451255da3b45cb67e92a375d337859711fd0b66c9f42407%40%3Cdev.calcite.apache.org%3E > > On Fri, Apr 3, 2020 at 4:04 PM

[DISCUSS] Deprecate grouped window functions

2020-04-22 Thread Rui Wang
Hi community, I want to kick off a discussion about deprecating grouped window functions (GROUP BY TUMBLE/HOP/SESSION) as the table function windowing support becomes a thing [1] (FROM TABLE(TUMBLE/HOP/SESSION)). The current stage of table function windowing is TUMBLE support is checked in. HOP an

Re: [DISCUSS] Deprecate grouped window functions

2020-04-22 Thread Rui Wang
Made a mistake on the example above, and update it as follows: // Table function windowing syntax. SELECT product_id, count(*), window_start FROM TABLE(TUMBLE(order, DESCRIPTOR(rowtime), INTERVAL '1' hour)) GROUP BY product_id, window_start On Wed, Apr 22, 2020 at 2:31 PM Rui W

Re: [DISCUSS] Deprecate grouped window functions

2020-04-24 Thread Rui Wang
et’s remove TUMBLE etc from the GROUP BY clause. Since this is a SQL > change, not an API change, I don’t we need to give notice. Let’s just do it. > > > > Julian > > > >> On Apr 22, 2020, at 4:05 PM, Rui Wang wrote: > >> > >> Made a mistake on t

Re: [ANNOUNCE] New committer: Vineet Garg

2020-04-27 Thread Rui Wang
Congrats Vineet! -Rui On Mon, Apr 27, 2020 at 11:16 AM Julian Hyde wrote: > Welcome Vineet! Thanks for your contributions so far. > > > On Apr 26, 2020, at 2:38 PM, Vineet G wrote: > > > > Thanks a lot guys! > > > > Just to briefly introduce myself - I work with Cloudera (Hortonworks > befor

Re: How to give type in case of type mismatch while doing union

2020-04-27 Thread Rui Wang
Did a quick test by running a SQL query that has UNION on two different types. Validator gave a correct error message (not NPE) to remind type mismatch. Agreed with Jin, could you provide more context/example how you reach the NPE? (It could be better if you can file a Jira with your context). -

Re: [ANNOUNCE] New committer: Forward Xu

2020-04-28 Thread Rui Wang
Congrats! -Rui On Tue, Apr 28, 2020 at 3:04 PM Francis Chuang wrote: > Congrats, Forward! > > Francis > > On 29/04/2020 7:53 am, Stamatis Zampetakis wrote: > > Apache Calcite's Project Management Committee (PMC) has invited Forward > Xu > > to > > become a committer, and we are pleased to anno

Re: [ANNOUNCE] New committer: Jin Xing

2020-04-28 Thread Rui Wang
Congrats, Jin! -Rui On Tue, Apr 28, 2020 at 3:03 PM Francis Chuang wrote: > Congrats, Jin! > > On 29/04/2020 7:47 am, Stamatis Zampetakis wrote: > > Apache Calcite's Project Management Committee (PMC) has invited Jin Xing > to > > become a committer, and we are pleased to announce that he has

Re: [ANNOUNCE] New committer: Wang Yanlin

2020-04-28 Thread Rui Wang
Congrats, Yanlin! -Rui On Tue, Apr 28, 2020 at 3:04 PM Francis Chuang wrote: > Congrats, Wang! > > Francis > > On 29/04/2020 7:50 am, Stamatis Zampetakis wrote: > > Apache Calcite's Project Management Committee (PMC) has invited Wang > Yanlin > > to > > become a committer, and we are pleased

Re: [DISCUSS] Deprecate grouped window functions

2020-04-28 Thread Rui Wang
20, at 11:10 AM, Rui Wang wrote: > > > > Hi Timo, > > > > My intention is to fully drop concepts such as SqlGroupedWindowFunction > and > > auxiliary group functions, which include relevant code in parser/syntax, > > operator, planner, etc. > > > &g

Re: [DISCUSS] Deprecate grouped window functions

2020-04-30 Thread Rui Wang
ul otherwise we need to maintain our own fork which we could mostly > prevent so far. > > > > Regards, > > Timo > > > > On 29.04.20 00:49, Rui Wang wrote: > >> Agreed. I would like to get more feedback to have a > >> reasonable accommodation for users. >

Re: [DISCUSS] Towards Calcite 1.23.0

2020-05-05 Thread Rui Wang
Thanks for leading 1.23.0 release, Haisheng! I will actively contact Danny and Feng to see if we can reach a conclusion on https://github.com/apache/calcite/pull/1761 by next monday. -Rui On Tue, May 5, 2020 at 11:31 AM Haisheng Yuan wrote: > Yes, no problem. Will have a RC before next Monday

Re: [CANCEL] [VOTE] Release apache-calcite-1.23.0 (release candidate 0)

2020-05-12 Thread Rui Wang
Found something that is worth mentioning in this candidate: 1. hyuan.asc [1] is not found on the server. Maybe the step that login Apache account [2] and filled out the public key fingerprint didn't finish? 2. Verify the artifact by "gpg --verify apache-calcite-1.23.0-src.tar.gz.asc apache-calcite

Re: Question of Calcite Dynamic Code Generation Feature

2020-05-13 Thread Rui Wang
Based on my past experience in distributed computing, there is a difference between a compilation time that generates java code and then compiles java code into bytecode, and a runtime that runs bytecode by JVM. The compilation will take time, but the runtime shouldn't be slower due to a good gene

Re: [DISCUSS] Keep 'master' branch closed after 1.23 release until we address the PR backlog

2020-05-13 Thread Rui Wang
Not saying my opinion of whether supporting this proposal or not, but just add something to the proposal itself: There is a need to have a person (likely the release manager :-)) to check "fix in 1.23" Jira list to identify which Jira has a promising PR, and then create a much smaller Jira list (b

Re: Adapter usage of RexImpTable

2020-05-18 Thread Rui Wang
RexToLixTranslator offers some public static methods to help you translate things to expressions (Ling4j). E.g. RexToLixTranslator.translateProjects might be what you need the most. After you get expressions, a way I know to execute it is by utilizing Janino [1] to compile expressions to bytecode

Re: [VOTE] Release apache-calcite-1.23.0 (release candidate 1)

2020-05-20 Thread Rui Wang
Indeed there is an extra site/fonts in the git tag. I didn't see it in the last two releases (the diff on licenses are normal). So the question is, does it matter for the diff on site/? At least the artifact has the same source code as the tag (thus can build the same jar, behave the same when run

Re: [VOTE] Release apache-calcite-1.23.0 (release candidate 1)

2020-05-20 Thread Rui Wang
knows the exact impact of this diff, we could add it to the release note, in the category of "known issues". -Rui On Wed, May 20, 2020 at 9:33 PM Rui Wang wrote: > Indeed there is an extra site/fonts in the git tag. I didn't see it in the > last two releases (the diff on li

Re: [DISCUSS] Why ProjectRemoveRule dose not consider field name when apply this rule

2020-05-27 Thread Rui Wang
Hello, This topic was discussed in https://issues.apache.org/jira/browse/CALCITE-1584. I am sure if Calcite does it, but in another SQL parser/analyzer implementation I have saw, each logical plan has a top attribute about final output column names, so even nodes might be merged/removed, at leas

Re: [DISCUSS] Why ProjectRemoveRule dose not consider field name when apply this rule

2020-05-27 Thread Rui Wang
Sorry, I meant "I am not sure if Calcite does it" (likely not). -Rui On Wed, May 27, 2020 at 10:31 AM Rui Wang wrote: > Hello, > > This topic was discussed in > https://issues.apache.org/jira/browse/CALCITE-1584. > > I am sure if Calcite does it, but in

Re: [ANNOUNCE] Apache Calcite 1.23.0 released

2020-05-28 Thread Rui Wang
Thanks Haisheng! -Rui On Wed, May 27, 2020 at 11:42 PM Forward Xu wrote: > Nice job, Haisheng! Thanks. > > > Best, > > ForwardXu > > Enrico Olivelli 于2020年5月28日周四 下午1:48写道: > > > Great > > > > > > Enrico > > > > Il Gio 28 Mag 2020, 05:12 Chunwei Lei ha > scritto: > > > > > Nice job, Haishen

Re: SqlNode.toSqlString() not parseable by calcite?

2020-05-29 Thread Rui Wang
You might try to add a test case at [1]. Based on your Drill JIRA, you might test the syntax containing "Literal (select * from UNNEST)". [1]: https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/sql/parser/SqlParserTest.java#L7174 -Rui On Thu, May 28, 2020 at 1

Re: SqlNode.toSqlString() not parseable by calcite?

2020-05-30 Thread Rui Wang
t; unparse, or > more likely, in drill as it configures the unparser with a dialect that is > not > comptible with the dialect it's parsing with. > > > Cheers, > Bozzo > > [1] > https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/

Re: NULL in IN clause

2020-06-05 Thread Rui Wang
Per the commit history I think you can find something in https://issues.apache.org/jira/browse/CALCITE-373. Meanwhile, "dim='a' OR dim=null" can be simplified to dim='a'? -Rui On Fri, Jun 5, 2020 at 2:13 PM Maytas Monsereenusorn wrote: > Hi Calcite, > > I am wondering why we convert a value

Re: NULL in IN clause

2020-06-05 Thread Rui Wang
On Fri, Jun 5, 2020 at 2:27 PM Rui Wang wrote: > Per the commit history I think you can find something in > https://issues.apache.org/jira/browse/CALCITE-373. > > Meanwhile, "dim='a' OR dim=null" can be simplified to dim='a'? > > > > -Rui

Re: [DISCUSS] Some quesitons about the new introduced window table functions (TUMBLE, HOP and SESSION)

2020-06-06 Thread Rui Wang
ent-tabpanel#comment-17107665 -Rui On Thu, Jun 4, 2020 at 6:02 AM Danny Chan wrote: > Hi, all, in the last 2 releases, Rui Wang has contributed the window table > functions to replace the legacy group window syntax, the idea comes from > the SQL-2016 polymorphic table functions. But

Re: [DISCUSS] Some quesitons about the new introduced window table functions (TUMBLE, HOP and SESSION)

2020-06-08 Thread Rui Wang
RVAL DAY TO SECOND, offset INTERVAL DAY TO SECOND DEFAULT INTERVAL '0' SECONDS) > It also determines > the actual output table type. > > I think it's worthwhile to read the PTF spec before implementing this, it's > freely available here: > > https://stan

Re: Why migrate from Maven to Gradle

2020-06-12 Thread Rui Wang
This email thread contains some discussions: https://lists.apache.org/thread.html/d43cf9958613c2b7489860d076f2d714551c1002f6fd41fd1f5c6e18%40%3Cdev.calcite.apache.org%3E -Rui On Fri, Jun 12, 2020 at 10:31 AM 丁小冬(比古) wrote: > > Hi, Calcite developers > > I would like to build calcite with Maven

Re: Why migrate from Maven to Gradle

2020-06-12 Thread Rui Wang
repo 2) you might work it around in your own fork. Additional question: why do you want to migrate back to Maven? -Rui On Fri, Jun 12, 2020 at 11:07 AM Rui Wang wrote: > This email thread contains some discussions: > https://lists.apach

Re: Nested data handling in Caclite

2020-06-15 Thread Rui Wang
>Also I am thinking about turning off the flatten stage but not sure this is >going to happen (seems like a pandora box kind of flag where you do not >know what to expect) It is not happening to make flattener optional by setting a flag (see [1]). If there is anything that is not properly flatten

Re: Nested data handling in Caclite

2020-06-19 Thread Rui Wang
If it goes back to make struct flattening configurable (thus can be turned off), you could check this thread for some context: https://lists.apache.org/thread.html/6bc9fd2e4c8d09e71740b0544df0982acf0d64321b9d5dc68db6acf1%40%3Cdev.calcite.apache.org%3E -Rui On Fri, Jun 19, 2020 at 1:21 AM Danny C

Re: Question about timestampdiff

2020-06-21 Thread Rui Wang
I think it is because there is no clear mapping for how to convert a month (and then same to year) in interval to seconds/milliseconds. Does a month have 30 days, 31 days, 28 days or 29 days? (Probably can find an answer in SQL standard). -Rui On Sun, Jun 21, 2020 at 4:00 PM Rommel Quintanilla

Re: Help to parse spark dialect

2020-06-26 Thread Rui Wang
There was a relevant thread about PostgresQL dialect, you might check it as a reference: https://lists.apache.org/thread.html/r906f15c3f7a276ae5e2c68f79a11dc111c5862579848ab8f379b9925%40%3Cdev.calcite.apache.org%3E -Rui On Fri, Jun 26, 2020 at 1:26 PM Tang, Mingjun wrote: > Hi dear calcite com

Re: Using the Calcite Babel Parser as a Blackbox

2020-06-27 Thread Rui Wang
Hi, I created a maven project and used 1.22.0 Calcite artifacts to make your code work: https://github.com/amaliujia/CalciteParser. Does this work for you? -Rui On Fri, Jun 26, 2020 at 12:44 PM Subin Lee wrote: > Hello! > > I hope this email finds you well. My name is Subin and I am an intern

[DISCUSS] Make RexNode serializable

2020-07-07 Thread Rui Wang
Hi Community, In Apache Beam we are facing a use case where we need to keep RexNode in our distributed primitives. Because of the nature of distributed computing, Beam requires the usage of those primitives be serializable (thus those primitives can be sent over the network to backend/workers for

Re: Correlated subquery losing aggregator operator in inner join?

2020-07-07 Thread Rui Wang
Since this discussion is moved to email thread, I just copy something I found and replied in that JIRA for the reference: SqlToRelConverter ends with creating a RexSubQuery for that subquery, and leaves it to JOIN condition. The string representation of RexSubQuery seems just $7. If you check RexS

Re: [DISCUSS] Make RexNode serializable

2020-07-08 Thread Rui Wang
40e736496a9c3/core/src/test/java/org/apache/calcite/plan/RelWriterTest.java#L88 > > > > > > -- > > > Roman Kondakov > > > > > > On 07.07.2020 22:13, Enrico Olivelli wrote: > > > > Rui > > > > > > > > Il Mar 7 Lug 2020, 20

Re: [DISCUSS] Make RexNode serializable

2020-07-08 Thread Rui Wang
O got it :-) -Rui On Wed, Jul 8, 2020 at 3:55 PM Julian Hyde wrote: > Please call it 'externalize'. 'Serialize' gives some folks PTSD. :) > > On Wed, Jul 8, 2020 at 2:26 PM Rui Wang wrote: > > > > Thanks everyone for your inputs. Now it sounds like

Re: [DISCUSS] Make RexNode serializable

2020-07-08 Thread Rui Wang
BTW, if use 'externalize', what is the opposite side of it? Is it `internalize` (which doesn't sound right)? Or call it "de-externalize"? -Rui On Wed, Jul 8, 2020 at 4:02 PM Rui Wang wrote: > O got it :-) > > > -Rui > > On Wed, Jul 8, 2020 at 3:55

Re: [DISCUSS] Towards Calcite 1.24.0

2020-07-09 Thread Rui Wang
Thanks Chunwei, I am actively working on getting CALCITE-4000 and CALCITE-4015 reviewed and merged by next week. Question: are you planning to follow the practice of the last release (i.e. lock master branch and clean up JIRA with fix version 1.24.0 marked before re-open master branch?) -Rui

Re: Calcite PR CI offten failed due to elasticsearch:test

2020-07-13 Thread Rui Wang
Here Andrei, Here is an example of a failed PR check that was failed due to elasticsearch:test: https://ci.appveyor.com/project/ApacheSoftwareFoundation/calcite/builds/33662741 See that failed Jdk11 check log. -Rui On Sun, Jul 12, 2020 at 4:31 PM Andrei Sereda wrote: > Hello, > > Can somebod

Re: Calcite and Enum Type

2020-07-15 Thread Rui Wang
Hi Talat, I am guessing when you say logical type, you mean something like this in Beam [1]. My question is why do you need Calcite to support ENUM? If you use logical type, you can define a ENUM by yourself and the underlying type can be a Map. Map is supported by Calcite. So ENUM will be transp

Re: [DISCUSS] Towards Calcite 1.24.0

2020-07-15 Thread Rui Wang
Regarding CALCITE-4000, the PR should be ready to merge now. Thanks for all the help from Feng, Danny and Chuwei! -Rui On Tue, Jul 14, 2020 at 11:11 PM Chunwei Lei wrote: > Thank you for all your effort. > > Now there're only two issues(CALCITE-4000[1] and CALCITE-4118[2]) to > be fixed in 1.2

Re: [DISCUSS] Remove "final" from AbstractRelNode#getRelTypeName

2020-07-16 Thread Rui Wang
+1 to make getRelTypeName overridable. Not a java language expert, to me when there are cases that a public API with final cannot solve, it's a good sign to remove that final to allow customization to solve those specific use cases. -Rui On Thu, Jul 16, 2020 at 8:58 AM Haisheng Yuan wrote: >

Re: [DISCUSS] Remove "final" from AbstractRelNode#getRelTypeName

2020-07-16 Thread Rui Wang
; 'final' is sometimes used intentionally to limit the API surface area. > > > > Julian > > > > [1] > > > https://github.com/apache/calcite/commit/6bb7e2d0b65ec3c7a0d82c92ca0564f8caec4af5 > > > > On Thu, Jul 16, 2020 at 10:33 AM Rui Wang wrote: > >

Re: [DISCUSS] New RexNode: RexListCmp

2020-07-19 Thread Rui Wang
Hi Haisheng, Do you know what is the impact on Enumerable implementation? Will RexListCmp still be converted AND/OR something else in that layer or there will be new logic introduced to generate code for RexListCmp? -Rui On Sun, Jul 19, 2020 at 10:13 PM Enrico Olivelli wrote: > Il Lun 20 Lug 2

Re: [DISCUSS] SqlDialect.getCastSpec() doesn't work when unparsing SqlNode tree

2020-07-20 Thread Rui Wang
Hi Pavel, Not sure if I understand your question, does override "SqlDialect.unparseCall" [1] provide an entry point to customize CAST unparsing per dialect? [1]: https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/SqlDialect.java#L442 -Rui On Mon, Jul 20, 20

Re: [VOTE] Release apache-calcite-1.24.0 (release candidate 0)

2020-07-21 Thread Rui Wang
+1 (non-binding) - downloaded the artifacts, checked hash, verified signature, built and ran tests. All are OK - use `diff -r` to compare the artifact and the git tag. Like what Julian has pointed out, I can find a difference about license but it can be addressed after this vote. -Rui On Tue, J

Re: [ANNOUNCE] Apache Calcite 1.24.0 released

2020-07-26 Thread Rui Wang
Thanks Chunwei for leading this release! -Rui On Sun, Jul 26, 2020 at 1:48 AM Forward Xu wrote: > Thanks! Chunwei! Good job. > > The link address of the Presto dialect implementation in > https://calcite.apache.org/news/2020/07/24/release-1.24.0/ seems to be > wrong. > > Best, > Forward > > C

Re: Avatica and HepPlanner

2020-08-02 Thread Rui Wang
I tried to dig into the Calcite codebase and found there is likely no way to enable JDBC codepath: 1. There is no planner config in CalciteConnectionProperty [1]. 2. In CalcitePrepareImpl, there is only one createPlannerFactory with one createPlanner, in which VolcanoPlanner is used [2]. If my u

Re: Implementation of DISTINCT and ORDER BY

2020-08-03 Thread Rui Wang
I did a test locally and I found your example syntax is correct. I used "select distinct deptno, empno from emp order by empno desc" and Calcite generates this plan: LogicalSort(sort0=[$1], dir0=[DESC]) LogicalAggregate(group=[{0, 1}]) LogicalProject(DEPTNO=[$7], EMPNO=[$0]) LogicalTa

Re: Re: Implementation of DISTINCT and ORDER BY

2020-08-04 Thread Rui Wang
[name_last] DESC > > So, just wanted to make sure that this is the right approach. > > Regards, > Hrudaya > > -Original Message- > From: Rui Wang > Sent: Monday, August 3, 2020 10:08 PM > To: dev@calcite.apache.org > Subject: [EXTERNAL] Re: Implementation of DISTI

Re: [DISCUSS] Towards Calcite 1.25.0

2020-08-08 Thread Rui Wang
Is the highlight section a right place for [1] as it has bigger impact? [1] https://github.com/apache/calcite/commit/19edf52c76c6a1507721f5bd37f2a33497aa0c4c -Rui On Fri, Aug 7, 2020 at 9:48 PM Andrei Sereda wrote: > Hello, > > Does anybody want to mention any highlights for 1.25 besides remo

  1   2   3   >