Thanks for the great write up! I think this s very good starting point for a detailed discussion about features, syntax and semantics of streaming SQL. I'll comment on the document in the next days and describe Flink's current status, our approaches (or planned approaches) and ask a couple of questions.
Thanks, Fabian 2017-07-28 3:05 GMT+02:00 Julian Hyde <jh...@apache.org>: > Tyler, > > Thanks for this. I am reading the document thoroughly and will give my > feedback in a day or two. > > Julian > > > On Jul 25, 2017, at 12:54 PM, Pramod Immaneni <pra...@datatorrent.com> > wrote: > > > > Thanks for the invitation Tyler. I am sure folks who worked on the > calcite > > integration and others would be interested. > > > > On Tue, Jul 25, 2017 at 12:12 PM, Tyler Akidau > <taki...@google.com.invalid> > > wrote: > > > >> +d...@apex.apache.org, since I'm told Apex has a Calcite integration as > >> well. If anyone on the Apex side wants to join in on the fun, your input > >> would be welcomed! > >> > >> -Tyler > >> > >> > >> On Mon, Jul 24, 2017 at 4:34 PM Tyler Akidau <taki...@apache.org> > wrote: > >> > >>> Hello Flink, Calcite, and Beam dev lists! > >>> > >>> Linked below is the second document I promised way back in April > >> regarding > >>> a collaborative spec for streaming SQL in Beam/Calcite/Flink (& > apologies > >>> for the delay; I thought I was nearly done a while back and then > temporal > >>> joins expanded to something much larger than expected). > >>> > >>> To repeat what it says in the doc, my hope is that it can serve various > >>> purposes over it's lifetime: > >>> > >>> - > >>> - A discussion ground for ironing out any remaining features > necessary > >>> for supporting robust streaming semantics in Calcite SQL. > >>> > >>> - A rough, high-level source of truth for tracking efforts underway > in > >>> support of this, currently spanning the Calcite, Flink, and Beam > >> projects. > >>> > >>> - A written specification of the changes that were made, for the sake > >>> of understanding the delta after the fact. > >>> > >>> The first and third points are, IMO, the most important. AFAIK, there > are > >>> a few features missing still that need to be defined (e.g., triggers > >>> equivalents via EMIT, robust temporal join support). I'm also > proposing a > >>> clear distinction of streams and tables, which I think is important, > but > >>> which I believe is not the approach most folks have been taking in this > >>> area. Sorting out these open issues and then having a concise record of > >> the > >>> solutions adopted will be important for providing a solid streaming > >>> experience and teaching folks how to use it. > >>> > >>> At any rate, I would much appreciate it if anyone with an interest in > >> this > >>> stuff could please take a look and add comments/suggestions/references > >> to > >>> related work in flight/etc as appropriate. For now please use > >>> comments/suggestions, but if you really want to dive in with edit > access, > >>> let me know. > >>> > >>> The doc: http://s.apache.org/streaming-sql-spec > >>> > >>> -Tyler > >>> > >>> > >>> > >> > >
