[
https://issues.apache.org/jira/browse/KAFKA-6976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539565#comment-16539565
]
Ashish Surana commented on KAFKA-6976:
--------------------------------------
Created below ticket to address this issue the right way:
https://issues.apache.org/jira/browse/KAFKA-7149: Reduce assignment data size
to improve kafka streams scalability
> Kafka Streams instances going in to DEAD state
> ----------------------------------------------
>
> Key: KAFKA-6976
> URL: https://issues.apache.org/jira/browse/KAFKA-6976
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 0.10.2.0
> Reporter: Deepak Goyal
> Priority: Blocker
> Attachments: kafkaStreamsDeadState.log
>
>
> We are using Kafka 0.10.2.0, Kafka-Streams 1.1.0. We have Kafka Cluster of 16
> machines, and topic that is being consumed by Kafka Streams has 256
> partitions. We spawned 400 machines of Kakfa Streams application. We see that
> all of the StreamThreads go in to DEAD state.
> {quote}{{[2018-05-25 05:59:29,282] INFO stream-thread
> [ksapp-19f923d7-5f9e-4137-b79f-ee20945a7dd7-StreamThread-1] State transition
> from PENDING_SHUTDOWN to DEAD
> (org.apache.kafka.streams.processor.internals.StreamThread) [2018-05-25
> 05:59:29,282] INFO stream-client [ksapp-19f923d7-5f9e-4137-b79f-ee20945a7dd7]
> State transition from REBALANCING to ERROR
> (org.apache.kafka.streams.KafkaStreams) [2018-05-25 05:59:29,282] WARN
> stream-client [ksapp-19f923d7-5f9e-4137-b79f-ee20945a7dd7] All stream threads
> have died. The instance will be in error state and should be closed.
> (org.apache.kafka.streams.KafkaStreams) [2018-05-25 05:59:29,282] INFO
> stream-thread [ksapp-19f923d7-5f9e-4137-b79f-ee20945a7dd7-StreamThread-1]
> Shutdown complete
> (org.apache.kafka.streams.processor.internals.StreamThread)}}
> {quote}
> Please note that when we only have 100 kafka-streams application machines,
> things are working as expected. We see that instances are consuming messages
> from topic.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
