Re: Synchronized Kafka sources

2017-08-10 Thread 魏偉哲
Hi Yunus, I'm not sure if there is a way to synchronize two Kafka sources in Flink, but I have another opinion on this question. How about adjust number of shards and parallelism of consumers on A and B? For example, making A have higher parallelism and B have lower parallelism so that you can ma

Synchronized Kafka sources

2017-08-04 Thread Yunus Olgun
Hi, Is it possible to synchronize two kafka sources? So they can consume from different Kafka topics in close enough event times. My use case is, I have two Kafka topics: A(very large) and B(large). There is a mapping of one to one or zero between A and B. Topology is simply join A and B in a tum