Re: kafka partition assignment

2016-06-21 Thread Michal Hariš
mplex topologies > there will most likely be shuffle downstream anyways. > > Regards, > Gordon > > > [1] > https://cwiki.apache.org/confluence/display/FLINK/Data+exchange+between+tasks > > > On June 21, 2016 at 4:02:01 AM, Michal Hariš (michal.har...@gmail.com)

Re: kafka partition assignment

2016-06-20 Thread Michal Hariš
/flink-docs-master/apis/streaming/#physical-partitioning > . > > Hope this helps! > > Regards, > Gordon > > On June 16, 2016 at 7:50:05 PM, Michal Hariš (michal.har...@gmail.com) > wrote: > > Hi, I was recently looking into a kafka connector issue (FLINK-4023 / > FLIN

kafka partition assignment

2016-06-16 Thread Michal Hariš
Hi, I was recently looking into a kafka connector issue (FLINK-4023 / FLINK-4069), when it was pointed out that partition assignment will not be deterministic if the partition discovery is imply moved to the open() method. In the assignPartitions of FlinkKafkaConsumerBase a modulo on the __index__