RE: EXTERNAL: Re: Separate Kafka partitioning from key compaction

2020-05-12 Thread Young, Ben
I'm not sure that's feasible in this case, but I'll have a look! Thanks, Ben -Original Message- From: Liam Clarke-Hutchinson Sent: 06 May 2020 19:47 To: users@kafka.apache.org Subject: EXTERNAL: Re: Separate Kafka partitioning from key compaction Could you deploy a Ka

Re: Separate Kafka partitioning from key compaction

2020-05-06 Thread Liam Clarke-Hutchinson
Could you deploy a Kafka Streams app that implemented your desired partitioning? Obviously this would require a duplication in topics between those produced to initially, and those partitioned the way you'd like, but it would solve the issue you're having. On Wed, 6 May 2020, 10:25 pm Young, Ben

Separate Kafka partitioning from key compaction

2020-05-06 Thread Young, Ben
Hi, We have a use case where we'd like the partition a key is hashed to, to be a subset of the keys that are used for compaction. It would be really cool if there was a built in hashing strategy that could help us as we're potentially using Kafka from multiple languages and it could be hard to