Re: Use a round-robin kafka partitioner

2017-10-26 Thread kla
Thanks for your comment. If I write the KafkaPartitioner anyway I have to somehow pass the *kafka.producer.Partitioner* which is not so easy. So I have found the easiest solution for this, you have just pass /null/ value: outputStream.addSink(new FlinkKafkaProducer010<>(producerProperties.getProp

Re: Use a round-robin kafka partitioner

2017-10-25 Thread Chesnay Schepler
So you want to use the kafka partitioner directly? How about an adapter? public class KafkaPartitionerWrapper extends KafkaPartitioner implements Serializable { private final kafka.producer.Partitionerpartitioner; public KafkaPartitionerWrapper(kafka.producer.Partitioner partitioner) {

Re: Use a round-robin kafka partitioner

2017-10-25 Thread kla
Exactly, I did like this, the only thing is that I am using 1.2.0 version of Flink and in this version the class name is KafkaPartitioner. But the problem is that I would not like to "fork" the Kafka's source code. (Please check my first comment) Thanks, Konstantin -- Sent from: http://apache-

Re: Use a round-robin kafka partitioner

2017-10-25 Thread Chesnay Schepler
Hi! you will have to modify your partitioner to implement the FlinkKafkaPartitioner interface instead. You

Re: Use a round-robin kafka partitioner

2017-10-24 Thread kla
Hi Chesnay, Thanks for your reply. I would like to use the partitioner within the Kafka Sink operation. By default kafka sink is using FixedPartitioner: public FlinkKafkaProducer010(String topicId, KeyedSerializationSchema serializationSchema, Properties producerConfig) {

Re: Use a round-robin kafka partitioner

2017-10-24 Thread Chesnay Schepler
Could you expand a bit more on what you want to achieve? (In particular /where/ you want to use this partitioner; as an operation before a sink or within a kafka sink) On 24.10.2017 09:24, kla wrote: Hey, I would like to use a round-robin kafka partitioner in the apache flink. (the default on