Without throttle, it will eventually ran out of memory. I think this is a very common use case for Flink users during stream replay or re-process. Do we have anything feature planed for it? Would like to contribute on the initiative.
On Wed, May 2, 2018 at 2:43 AM, Fabian Hueske <fhue...@gmail.com> wrote: > Hi Tao, > > The watermarks of operators that consume from two (or more) streams are > always synced to the lowest watermark. > This behavior guarantees that data won't be late (unless it was late when > watermarks were assigned). However, the operator will most likely need to > buffer more events from the "faster" streams. > > Right now, it is not possible to throttle faster streams to the pace of > the slowest stream. > > Best, Fabian > > 2018-04-27 1:05 GMT+02:00 Tao Xia <t...@udacity.com>: > >> Hi All, >> I am trying to reply events from 3 different sources and hopefully in >> time sequence, say Stream1, Stream2, Stream3. Since their size vary a lot, >> the watermarks on one stream is much faster than other streams. Is there >> any way to coordinate the watermarks between different input streams. >> Thanks, >> Tao >> > >
