Re: Streams constantly reblancing,

2020-09-11 Thread Sophie Blee-Goldman
That definitely sounds pretty odd if no new logs are showing up on the Streams side. This is just a guess, but is there any chance the source topics aren't being found during startup? I was just reminded of a bug in an old-ish version where Streams would gracefully shutdown but remain in the REBALA

Re: Scaling issues with MSK on AWS

2020-09-11 Thread Arti Pande
Hi Ben, Thanks for quick reply. As you mentioned, we are seeing the write throughput per broker being capped at 250 MB/s. We tried multiple scenarios as below 1) Msg size 1 KB, 3 publishers, 3 brokers, with 3 parritions and RF = 2 In this, as mentioned in earlier emai, with replication on Tot

Re: Mirror Maker 2.0 NOT generating checkpoints for consumers running in assign mode

2020-09-11 Thread Ananya Sen
I am still facing the same issue. On Fri, Sep 11, 2020, 4:46 PM wrote: > Hi Ananya > Are you able to resolve this issue ,I'm also facing same issue . > > What parameter should be pass here if I'm doing failover from cluster A > ---> B > > Map newOffsets = > RemoteClusterUtils.transla

Re: Kafka stream error - Consumer is not subscribed to any topics or assigned any partitions

2020-09-11 Thread Sophie Blee-Goldman
You should upgrade to 2.5.1, it contains a fix for this. Technically the "fix" is just to automatically set the num.stream.threads to 0 when a global-only topology is detected, so setting this manually would accomplish the same thing. But the fix also includes a tweak of the KafkaStreams state mac

Kafka stream error - Consumer is not subscribed to any topics or assigned any partitions

2020-09-11 Thread Pushkar Deole
Hi All, I upgraded from Kafka streams 2.4 to 2.5.0 and one of the applications suddenly stopped working with the error message: Exception in thread "DsiApplication-0fcde033-dab2-431c-9d82-76e85fcb4c91-StreamThread-1" java.lang.IllegalStateException: Consumer is not subscribed to any topics or ass

Re: Mirror Maker 2.0 NOT generating checkpoints for consumers running in assign mode

2020-09-11 Thread Manoj.Agrawal2
Hi Ananya Are you able to resolve this issue ,I'm also facing same issue . What parameter should be pass here if I'm doing failover from cluster A ---> B Map newOffsets = RemoteClusterUtils.translateOffsets(properties, "A", "TestTopic-123", Duration.ofMillis(5500)); Properties= Bo