Hi Aljoscha, Thanks for the nice summary, this is a very good initiative.
I added some comments to the respective sections (where I didnt fully agree :).). At some point I think it would be good to have a public hangout session on this, which could make a more dynamic discussion. Cheers, Gyula Aljoscha Krettek <aljos...@apache.org> ezt írta (időpont: 2015. jún. 22., H, 21:34): > Hi, > with people proposing changes to the streaming part I also wanted to throw > my hat into the ring. :D > > During the last few months, while I was getting acquainted with the > streaming system, I wrote down some thoughts I had about how things could > be improved. Hopefully, they are in somewhat coherent shape now, so please > have a look if you are interested in this: > > https://docs.google.com/document/d/1rSoHyhUhm2IE30o5tkR8GEetjFvMRMNxvsCfoPsW6_4/edit?usp=sharing > > This mostly covers: > - Timestamps assigned at sources > - Out-of-order processing of elements in window operators > - API design > > Please let me know what you think. Comment in the document or here in the > mailing list. > > I have a PR in the makings that would introduce source timestamps and > watermarks for keeping track of them. I also hacked a proof-of-concept of a > windowing system that is able to process out-of-order elements using a > FlatMap operator. (It uses panes to perform efficient pre-aggregations.) > > Cheers, > Aljoscha >
