Re: Yet Another Repartitioning Question About Kafka Streams

2021-04-11 Thread Gareth Collins
g/confluence/display/KAFKA/KIP-535%3A+Allow+state+stores+to+serve+stale+reads+during+rebalance > > > Guozhang > > On Sun, Apr 4, 2021 at 12:41 PM Gareth Collins > > wrote: > > > Hi, > > > > Thanks very much for answers to my previous questions here. > > &g

Yet Another Repartitioning Question About Kafka Streams

2021-04-04 Thread Gareth Collins
Hi, Thanks very much for answers to my previous questions here. I had a couple more questions about repartitioning and I just want to confirm my understanding. (1) Given the following scenario: (a) I have a cluster of Kafka stream nodes with partitions assigned to each. (b) One node goes down.

Re: Kafka Streams And Partitioning

2021-03-20 Thread Gareth Collins
t; > > > > If the source topics are user input topics and you have control over > the > > > production to these topics, > > > then just use a custom partitioner to produce to them. If you don't > have > > > control over them, you can > > > ins

Re: Kafka Streams And Partitioning

2021-03-17 Thread Gareth Collins
d operator, it accepts a Repartitioned > which itself accepts a > StreamPartitioner that you can use to control the partitioning. > > You can check out the class WindowedStreamPartitioner for an example: this > is how we handle the > WindowStore case that you pointed out. &g

Right Use Case For Kafka Streams?

2021-03-15 Thread Gareth Collins
Kafka Streams provides if I don't have to. Will the Kafka Streams RocksDB solution be so much faster than a distributed read that it won't be the bottleneck even with huge amounts of data? Any info/opinions would be greatly appreciated. thanks in advance, Gareth Collins

Kafka Streams And Partitioning

2021-03-15 Thread Gareth Collins
advance, Gareth Collins