Hi Jungtaek:
We also meet this problem during migration of streaming application to
Structure Streaming in Baidu practice, we solved this in our folk and just
steady running in product.
As the initial plan we are doing the code clean work and preparing give a
SPIP in Oct, happy to see your proposal. Hope we share some spots together.
Here’s the PR and doc:
https://github.com/apache/spark/pull/22583
<https://github.com/apache/spark/pull/22583>
https://docs.google.com/document/d/1zeAc7QKSO7J4-Yk06kc76kvldl-QHLCDJuu04d7k2bg/edit?usp=sharing
<https://docs.google.com/document/d/1zeAc7QKSO7J4-Yk06kc76kvldl-QHLCDJuu04d7k2bg/edit?usp=sharing>
Thanks,
Yuanjian Li
> 在 2018年9月28日，06:22，Jungtaek Lim <kabh...@gmail.com
> <mailto:kabh...@gmail.com>> 写道：
>
> Hi all,
>
> I would like to initiate discussion thread to discuss "Native support of
> session window".
> Origin issue is filed to SPARK-10816 [1] but I can file another one for
> representing SPIP if necessary. WIP but working PR is available as well, so
> we can even test it directly or see the difference if some of us feel more
> convenient to go through the source code instead of doc.
>
> I've attached PDF version of SPIP in SPARK-10816, but adding Google Docs link
> [2] for who feel convenient to comment in doc.
>
> Please let me know if we would like to see also technical design for this. I
> avoid to go too deep on SPIP doc so anyone could review and see the benefit
> of adopting this.
>
> Looking forward to hear your feedback.
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
> 1. https://issues.apache.org/jira/browse/SPARK-10816
> <https://issues.apache.org/jira/browse/SPARK-10816>
> 2.
> https://docs.google.com/document/d/1_rMLmUSyGzb62RnP2A3WX6D6uRxox8Q_7WcoI_HrTw4/edit?usp=sharing
>
> <https://docs.google.com/document/d/1_rMLmUSyGzb62RnP2A3WX6D6uRxox8Q_7WcoI_HrTw4/edit?usp=sharing>
> 3. https://github.com/apache/spark/pull/22482
> <https://github.com/apache/spark/pull/22482>
>
>
