Hi Jark,
Thanks for reply, yes, I think it's a good feature, it can improve the
NRT scenarios
as you mentioned in the FLIP. Also, I think it can improve the
streaming SQL greatly,
it can support richer window operations in flink SQL and bring great
convenience to users.
(we are now only supported group window in flink).
Regarding the SESSION window, I think it's especially useful for user
behavior analysis(e.g.
counting user visits on a news website or social platform), but I agree
that we can keep it
out of the FLIP now to catch up 1.12.
Recently, I've done some work on the stream planner with the TVFs, and
I'm willing to contribute
to this part. Is it in the plan of this FLIP?
Best,
PengchengLiu
在 2020/9/26 下午11:09，“Jark Wu”<imj...@gmail.com> 写入:
Hi pengcheng,
That's great to see you also have the need of window join.
You are right, the windowing TVF is a powerful feature which can support
more operations in the future.
I think it as of the date time "partition" selection in batch SQL jobs,
with this new syntax, I think it is possible
to migrate traditional batch SQL jobs to Flink SQL by changing a few lines.
Regarding the SESSION window, this is on purpose to keep it out of the
FLIP, because we want to keep the
FLIP small to catch up 1.12 and SESSION TVF is rarely useful (e.g. session
window join?).
Best,
Jark
On Fri, 25 Sep 2020 at 22:59, liupengcheng <pengchengliucr...@gmail.com>
wrote:
> Hi, Jark,
> I'm very interested in this feature, and I'm also working on this
> recently.
> I just have a glance at the FLIP, it's good, but I found that
> there is no plan to add SESSION windows.
> Also, I think there can be more things we can do based on this new
> syntax. For example,
> - window sort support
> - window union/intersect/minus support
> - Improve dimension table join
> We can have more deep discussion on this new feature later .
> I've also opened an jira that is related to this feature recently:
> https://issues.apache.org/jira/browse/FLINK-18830
>
> Best!
> PengchengLiu
>
> 在 2020/9/25 下午10:30，“Jark Wu”<imj...@gmail.com> 写入:
>
> Hi everyone,
>
> I want to start a FLIP about supporting windowing table-valued
> functions
> (TVF).
> The main purpose of this FLIP is to improve the near real-time (NRT)
> experience of Flink.
>
> FLIP-145:
>
>
https://cwiki.apache.org/confluence/display/FLINK/FLIP-145%3A+Support+SQL+windowing+table-valued+function
>
> We want to introduce TUMBLE, HOP, CUMULATE windowing TVFs, the
> CUMULATE is
> a new kind of window.
> With the windowing TVFs, we can support richer operations on windows,
> including window join, window TopN and so on.
> This makes things simple: we only need to assign windows at the
> beginning
> of the query, and then apply operations after that like traditional
> batch
> SQL.
> We hope it can help to reduce the learning curve of windows, improve
> NRT
> for Flink, and attract more batch users.
>
> A simple code snippet for 10 minutes tumbling window aggregate:
>
> SELECT window_start, window_end, SUM(price)
> FROM TABLE(
> TUMBLE(TABLE Bid, DESCRIPTOR(bidtime), INTERVAL '10' MINUTES))
> GROUP BY window_start, window_end;
>
> I'm looking forward to your feedback.
>
> Best,
> Jark
>
>
>
