DAG processing in Kafka Streams

2017-07-16 Thread Sameer Kumar
Currently, we don't have DAG processing in Kafka Streams. Having a DAG has its own share of advantages in that, it can optimize code on its own and come up with a optimized execution plan. Are we exploring in this direction, do we have this in our current roadmap. -Sameer.

Re: How can we re-key and re-partition records based on that key

2017-07-16 Thread Matthias J. Sax
If you only want to change the key, you can use #selectKey() -- if you want to change key and value, you can use #map(). Stream will automatically repartition the data afterwards if required (ie, if you do a group-by or join). If you want to force repartitioning, you can just call #through() after

How can we re-key and re-partition records based on that key

2017-07-16 Thread Sachin Mittal
Hi, I have a topic of records (K1, V1). This topic is partitioned based on the K1. I use multiple streams instances to process these records based on that key so that each instance process certain partitions only. Now I want to re-key the records in that topic with a format (K2, V2) and push them