Hi,

I am very exited about all of this in general. Sadly I haven’t had the time to really take a deep look. One thing that is/was always a difficult topic to resolve many to many relationships in table x table x table joins is the repartitioning that has to happen at some point.

From the documentation I saw this:

"The *keys* of data records determine the partitioning of data in both Kafka and Kafka Streams, i.e. how data is routed to specific partitions within topics."

This feels unnecessarily restrictive as i can't currently imagin how to resolve many to many relationships with this. One can also emmit every record to many partitions to make up for no read replicas in kafka aswell as partitioning schemes that don't work like this (Shards processing overlapping key spaces).

I would really love to hear your thoughts on these topics. Great work! Google grade technologies for everyone!
I <3 logs



On 10.03.2016 22:26, Jay Kreps wrote:
Hey all,

Lot's of people have probably seen the ongoing work on Kafka Streams
happening. There is no real way to design a system like this in a vacuum,
so we put up a blog, some snapshot docs, and something you can download and
use easily to get feedback:

http://www.confluent.io/blog/introducing-kafka-streams-stream-processing-made-simple

We'd love comments or thoughts from anyone...

-Jay


Reply via email to