Thanks Max -
I will advance watermarks when no event arrives for a while. But when
using Kafka is it a good practice to assign events to partitions randomly
instead say device id or region id where the devices are located. What I
noticed is if devices sending to one of the partitions stop sending
Hi Sameer,
If you use Event Time you should make sure to assign Watermarks and
Timestamps at the source. As you already observed, Flink may get stuck
otherwise because it waits for Watermarks to progress in time.
There is no timeout for windows. However, you can implement that logic
in your Water
Sorry for replying to my own messages but this is super confusing and
logical at the same time to me :-).
If I have Kafka Topic with 10 partitions. If I partition by device id when
I write to the Topic, and use Event Time, my pipeline freezes (if fewer
than 10 devices are active initially). Becaus
And this is happening in my local environment. As soon as I set the
parallelism to 1 it all works fine.
Sameer
On Wed, Aug 10, 2016 at 3:11 PM, Sameer W wrote:
> Hi,
>
> I am noticing this behavior with Event Time processing-
>
> I have a Kafka topic with 10 partitions. Each Event Source sends
Hi,
I am noticing this behavior with Event Time processing-
I have a Kafka topic with 10 partitions. Each Event Source sends data to
any one of the partitions. Say I have only 1 event source active at this
moment, which means only one partition is receiving data.
None of my windows will fire now
