Re: FlinkKafkaConsumer subscribes to partitions in restoredState only.

2017-07-26 Thread ninad
Got it. Thanks Gordon. -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/FlinkKafkaConsumer-subscribes-to-partitions-in-restoredState-only-tp14233p14484.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble

Re: FlinkKafkaConsumer subscribes to partitions in restoredState only.

2017-07-24 Thread Tzu-Li (Gordon) Tai
One other note: the functionality is actually already merged to the master branch. You can also take a look at the feature documentation here [1]. [1]  https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/connectors/kafka.html#kafka-consumers-partition-discovery On 25 July 2017 at 1:22

Re: FlinkKafkaConsumer subscribes to partitions in restoredState only.

2017-07-24 Thread Tzu-Li (Gordon) Tai
Hi, Sorry for not replying to this earlier, it seems like this thread hadn’t been noticed earlier. What you are experiencing is expected behavior. In Flink 1.3, new partitions will not be picked up, only partitions that are in checkpoints state will be subscribed to on restore runs. One main r

Re: FlinkKafkaConsumer subscribes to partitions in restoredState only.

2017-07-24 Thread ninad
Any update on this guys? -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/FlinkKafkaConsumer-subscribes-to-partitions-in-restoredState-only-tp14233p14410.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabbl