Hey everyone,
We ran across a little bit of a landmine in Kafka Streams 0.11.0.1.
We decided to flip num.standby.replicas to 1 so that we had a hot copy of
our state stores on other hosts to facilitate fast failover. However, one
of the applications we tried this on currently uses an in memory store.
When we flipped the setting on, we then started to restart the running
instances.
When we restarted the first instance of the application, it came up fine.
When we restarted the second instance, both instances crashed with the
following exception:
java.lang.IllegalStateException: Consumer is not subscribed to any topics
or assigned any partitions
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1037)
at
org.apache.kafka.streams.processor.internals.StreamThread.maybeUpdateStandbyTasks(StreamThread.java:752)
at
org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:524)
at
org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:480)
at
org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:457)
Changing the application to use a persistent store seems to have resolved
the issue.
A few questions:
1. This feels like a bug... is this a bug? :D
2. If so, is this already filed somewhere? I did some searching on JIRA but
didn't turn up much.
Happy to file an issue and (potentially) tinker with a fix if this is
indeed a bug.
Thanks,
Matt Farmer
