And this is happening in my local environment. As soon as I set the parallelism to 1 it all works fine.
Sameer On Wed, Aug 10, 2016 at 3:11 PM, Sameer W <sam...@axiomine.com> wrote: > Hi, > > I am noticing this behavior with Event Time processing- > > I have a Kafka topic with 10 partitions. Each Event Source sends data to > any one of the partitions. Say I have only 1 event source active at this > moment, which means only one partition is receiving data. > > None of my windows will fire now because the 9 partitions (source function > instances) are not sending any watermarks and Flink waits forever. > > I go to topic with 1 partition but leave default parallelism intact. Only > one Mapper instance contributes to the subsequent keyBy operation but other > 7 (assuming 8 of default parallelism) are idle. I assign watermarks after > the map function. Again the same behavior because the 7 other mappers are > not sending watermarks. > > How do I handle this? Not all of my partitions are going to be receiving > data at all times using this partitioning strategy. Or I have to use random > partitioning which will also work. > > Thanks, > Sameer >
