Re: [DISCUSS] Table API / SQL internal timestamp handling

2017-08-01 Thread Fabian Hueske
gt; > orthogonal >> > > > > > to >> > > > > > > this discussion, IMO. >> > > > > > > 4) From my point of view, proc-time is a purely virtual column >> > and >> > > > not >> > > >

Re: [DISCUSS] Table API / SQL internal timestamp handling

2017-08-01 Thread Fabian Hueske
ma > > > > > > > definition of the table. Processing time queries can never > > compute > > > > the > > > > > > same > > > > > > > results as batch queries but their semantics should be aligned > > with > > > >

Re: [DISCUSS] Table API / SQL internal timestamp handling

2017-07-31 Thread Xingcan Cui
hanks for the remarks. I have a question > regarding > > > your > > > > > > > suggestion not to consider to create proctime window in a > regular > > > > > > column. I > > > > > > > think this would be useful though. First

Re: [DISCUSS] Table API / SQL internal timestamp handling

2017-07-31 Thread Shaoxuan Wang
> > provenance, traceability ...). Secondly - I do not think it is > > > > > > contradicting with the semantics in batch SQL as in SQL you have > > the > > > > > > function "now()" ...which pretty much carry the same semantics as &g

Re: [DISCUSS] Table API / SQL internal timestamp handling

2017-07-31 Thread Xingcan Cui
in example, > > > which > > > > > > could be reordered of course). All we have to do is to ensure > that > > > all > > > > > > timestamps are aligned with the watermarks. > > > > > > > > > > > > Radu 2: > > > > &

Re: [DISCUSS] Table API / SQL internal timestamp handling

2017-07-31 Thread Fabian Hueske
> Riesstrasse 25, 80992 München > > > > > > > > E-mail: radu.tudo...@huawei.com > > > > Mobile: +49 15209084330 > > > > Telephone: +49 891588344173 > > > > > > > > HUAWEI TECHNOLOGIES Duesseldorf GmbH > > > > Hansaallee 205, 40549 Düsseldorf, Germ

Re: [DISCUSS] Table API / SQL internal timestamp handling

2017-07-31 Thread Jark Wu
gister Court Düsseldorf, HRB 56063, > > > Managing Director: Bo PENG, Qiuen Peng, Shengli Wang > > > Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063, > > > Geschäftsführer: Bo PENG, Qiuen Peng, Shengli Wang > > > This e-mail and its attachments

Re: [DISCUSS] Table API / SQL internal timestamp handling

2017-07-27 Thread Xingcan Cui
bove. Any use of the information contained herein in any way > > (including, but not limited to, total or partial disclosure, > reproduction, > > or dissemination) by persons other than the intended recipient(s) is > > prohibited. If you receive this e-mail in error, please notify the se

Re: [DISCUSS] Table API / SQL internal timestamp handling

2017-07-27 Thread Fabian Hueske
main > > > stream (...although I am not sure which one is the main stream in > > > the > > case > > > of a full join:) ) > > > > > > > > > Dr. Radu Tudoran > > > Staff Research Engineer - Big Data Expert IT R&D Division > >

RE: [DISCUSS] Table API / SQL internal timestamp handling

2017-07-27 Thread Radu Tudoran
sage- From: Shaoxuan Wang [mailto:shaox...@apache.org] Sent: Thursday, July 27, 2017 6:00 AM To: Dev Subject: Re: [DISCUSS] Table API / SQL internal timestamp handling Hi Everyone, I like this proposal. The problem we used to have is that we have treated eventtime column as a special timest

Re: [DISCUSS] Table API / SQL internal timestamp handling

2017-07-26 Thread Shaoxuan Wang
Amtsgericht Düsseldorf, HRB 56063, > > Geschäftsführer: Bo PENG, Qiuen Peng, Shengli Wang > > This e-mail and its attachments contain confidential information from > > HUAWEI, which is intended only for the person or entity whose address is > > listed above. Any use of the informat

Re: [DISCUSS] Table API / SQL internal timestamp handling

2017-07-26 Thread Fabian Hueske
way > (including, but not limited to, total or partial disclosure, reproduction, > or dissemination) by persons other than the intended recipient(s) is > prohibited. If you receive this e-mail in error, please notify the sender > by phone or email immediately and delete it! > > --

RE: [DISCUSS] Table API / SQL internal timestamp handling

2017-07-26 Thread Radu Tudoran
tely and delete it! -Original Message- From: Jark Wu [mailto:j...@apache.org] Sent: Wednesday, July 26, 2017 8:29 AM To: dev@flink.apache.org Subject: Re: [DISCUSS] Table API / SQL internal timestamp handling Hi Xingcan, IMO, I don't think event-time of join results could be auto

Re: [DISCUSS] Table API / SQL internal timestamp handling

2017-07-25 Thread Jark Wu
Hi Xingcan, IMO, I don't think event-time of join results could be automatically decided by system. Considering batch tables, if users want a event time window aggregation after join, user must specify the time field explicitly (T1.rowtime or T2.rowtime or the computed result of them). So in the c

Re: [DISCUSS] Table API / SQL internal timestamp handling

2017-07-25 Thread Xingcan Cui
Hi all, @Fabian, thanks for raising this. @Radu and Jark, personally I think the timestamp field is critical for query processing and thus should be declared as (or supposed to be) NOT NULL. In addition, I think the event-time semantic of the join results should be automatically decided by the sy

Re: [DISCUSS] Table API / SQL internal timestamp handling

2017-07-25 Thread Jark Wu
Hi, Radu's concerns make sense to me, especially the null value timestamp and multi-proctime. I have also something in my mind. I would like to propose some time indicator built-in functions, e.g. ROW_TIME(Timestamp ts) will generate a event time logical attribute, PROC_TIME() will generate a pro

RE: [DISCUSS] Table API / SQL internal timestamp handling

2017-07-25 Thread Radu Tudoran
Hi, I think this is an interesting discussion and I would like to add some issues and give some feedback. - For supporting the join we do not only need to think of the time but also on the null values. For example if you have a LEFT (or RIGHT) JOIN between items of 2 input streams, and the sec