Hello, We noticed that a kafka streams app is stuck in rebalance state with below error. Two instance of the app were running fine until a rebalace was triggered(possibly due to network issue). Both app instance are running(no app restart) App itself doesn't create/use state store. NUM_STREAM_THREADS_CONFIG=2
I did see several tickets with similar errors that are marked as fixed. I'm using version 0.10.2.1. 17/12/04 18:34:57 WARN StreamThread: Could not create task 0_2. Will retry. org.apache.kafka.streams.errors.LockException: task [0_2] Failed to lock the state directory for task 0_2 at org.apache.kafka.streams.processor.internals.ProcessorStateManager.<init>(ProcessorStateManager.java:100) at org.apache.kafka.streams.processor.internals.AbstractTask.<init>(AbstractTask.java:73) at org.apache.kafka.streams.processor.internals.StreamTask.<init>(StreamTask.java:108) at org.apache.kafka.streams.processor.internals.StreamThread.createStreamTask(StreamThread.java:864) at org.apache.kafka.streams.processor.internals.StreamThread$TaskCreator.createTask(StreamThread.java:1237) at org.apache.kafka.streams.processor.internals.StreamThread$AbstractTaskCreator.retryWithBackoff(StreamThread.java:1210) at org.apache.kafka.streams.processor.internals.StreamThread.addStreamTasks(StreamThread.java:967) at org.apache.kafka.streams.processor.internals.StreamThread.access$600(StreamThread.java:69) at org.apache.kafka.streams.processor.internals.StreamThread$1.onPartitionsAssigned(StreamThread.java:234) at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinComplete(ConsumerCoordinator.java:259) at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:352) at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:303) at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:290) at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1029) at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:995) at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:592) at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:361) 17/12/04 18:34:58 WARN StreamThread: Could not create task 0_8. Will retry. org.apache.kafka.streams.errors.LockException: task [0_8] Failed to lock the state directory for task 0_8 kafka-consumer-groups --new-consumer --bootstrap-server <...> --describe --group GeoTest Note: This will only show information about consumers that use the Java consumer API (non-ZooKeeper-based consumers). Warning: Consumer group 'GeoTest' is rebalancing. I keep seeing the above lock exception continuously and app is not making any progress. Any idea why it is stuck? I read a few suggestions that required me to manually delete state directory. I'd like to avoid that. Thanks, Srikanth
