Re: Watermark in broadcast

2017-12-14 Thread Tzu-Li (Gordon) Tai
enter, 46th Floor, New York, NY 10007
     From: Seth Wiesman Date: Wednesday, December 13, 2017 at 12:04 PM To: Timo Walther , "user@flink.apache.org" Subject: Re: Watermark in broadcast   Hi Timo,   I think you are correct. This stream is consumed from Kafka and the number of

Re: Watermark in broadcast

2017-12-14 Thread Fabian Hueske
Wiesman > *Date: *Wednesday, December 13, 2017 at 12:04 PM > *To: *Timo Walther , "user@flink.apache.org" < > user@flink.apache.org> > *Subject: *Re: Watermark in broadcast > > > > Hi Timo, > > > > I think you are correct. This stream is consumed

Re: Watermark in broadcast

2017-12-13 Thread Seth Wiesman
iesman Date: Wednesday, December 13, 2017 at 12:04 PM To: Timo Walther , "user@flink.apache.org" Subject: Re: Watermark in broadcast Hi Timo, I think you are correct. This stream is consumed from Kafka and the number of partitions is much less than the parallelism of the program so ther

Re: Watermark in broadcast

2017-12-13 Thread Seth Wiesman
@01D3740A.880106E0]<http://www.mediamath.com/> Seth Wiesman | Software Engineer, Data
 4 World Trade Center, 46th Floor, New York, NY 10007
 From: Timo Walther Date: Wednesday, December 13, 2017 at 11:46 AM To: "user@flink.apache.org" Subject: Re: Watermark in broadcast Hi Set

Re: Watermark in broadcast

2017-12-13 Thread Timo Walther
Hi Seth, are you sure that all partitions of the broadcasted stream send a watermark? processWatermark is only called if a minimum watermark arrived from all partitions. Regards, Timo Am 12/13/17 um 5:10 PM schrieb Seth Wiesman: Hi, How are watermarks propagated during a broadcast partiti

Watermark in broadcast

2017-12-13 Thread Seth Wiesman
Hi, How are watermarks propagated during a broadcast partition? I have a TwoInputStreamTransformation that takes a broadcast stream as one of its inputs. Both streams are assigned timestamps and watermarks before being connected however I only ever see watermarks from my non-broadcast stream. I