Re: Getting key from keyed stream

2017-01-12 Thread Paul Joireman
Thanks Jamie, Just figured that out after some digging and a little trial and error, that works great. Paul From: Jamie Grier Sent: Thursday, January 12, 2017 4:59:43 PM To: user@flink.apache.org Subject: Re: Getting key from keyed stream A simpler and

Re: Getting key from keyed stream

2017-01-12 Thread Jamie Grier
A simpler and more efficient approach would simply be the following: val stream = env.addSource(new FlinkKafkaConsumer(...)) stream .addSink(new FlinkKafkaProducer(new MyKeyedSerializationSchema(...))) env.execute() In MyKeyedSerializationSchema just override the getTargetTopic() method. Tha

Getting key from keyed stream

2017-01-12 Thread Paul Joireman
Hi all, Is there a simple way to read the key from a KeyedStream. Very simply I'm trying to read a message from Kafka, separate the incoming messages by a field in the message and write the original message back to Kafka using that field as a new topic. I chose to partition the incoming str