Matt Farmer created KAFKA-6214:
----------------------------------
Summary: Using standby replicas with an in memory state store
causes Streams to crash
Key: KAFKA-6214
URL: https://issues.apache.org/jira/browse/KAFKA-6214
Project: Kafka
Issue Type: Bug
Components: streams
Affects Versions: 0.11.0.1
Reporter: Matt Farmer
We decided to start experimenting with Standby Replicas of our State Stores by
setting the following configuration setting:
{code}
num.standby.replicas=1
{code}
Most applications did okay with this except for one that used an in memory
state store instead of a persistent state store. With the new configuration,
the first instance of this application booted fine. When the second instance
came up, both instances crashed with the following exception:
{code}
java.lang.IllegalStateException: Consumer is not subscribed to any topics or
assigned any partitions
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1037)
at
org.apache.kafka.streams.processor.internals.StreamThread.maybeUpdateStandbyTasks(StreamThread.java:752)
at
org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:524)
at
org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:480)
at
org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:457)
{code}
Monit attempted to restart both instances but they would just continue to crash
over and over again. The state store in our problematic application is declared
like so:
{code}
Stores
.create("TheStateStore")
.withStringKeys()
.withStringValues()
.inMemory()
.build()
{code}
Luckily we had a config switch in place that could turn on an alternate,
persistent state store. As soon as we flipped to the persistent state store,
things started working as we expected.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
