Michal Borowiecki created KAFKA-4366:
----------------------------------------
Summary: KafkaStreams.close() blocks indefinitely
Key: KAFKA-4366
URL: https://issues.apache.org/jira/browse/KAFKA-4366
Project: Kafka
Issue Type: Bug
Components: streams
Affects Versions: 0.10.0.1, 0.10.1.0
Reporter: Michal Borowiecki
Assignee: Guozhang Wang
KafkaStreams.close() method calls join on all its threads without a timeout,
meaning indefinitely, which makes it prone to deadlocks and unfit to be used in
shutdown hooks.
(KafkaStreams::close is used in numerous examples by confluent:
https://github.com/confluentinc/examples/tree/kafka-0.10.0.1-cp-3.0.1/kafka-streams/src/main/java/io/confluent/examples/streams
and
https://www.confluent.io/blog/introducing-kafka-streams-stream-processing-made-simple/
so we assumed it to be recommended practice)
A deadlock happens, for instance, if System.exit() is called from within the
uncaughtExceptionHandler. (We need to call System.exit() from the
uncaughtExceptionHandler because KAFKA-4355 issue shuts down the StreamThread
and to recover we want the process to exit, as our infrastructure will then
start it up again.)
The System.exit call (from the uncaughtExceptionHandler, which runs in the
StreamThread) will execute the shutdown hook in a new thread and wait for that
thread to join. If the shutdown hook calls KafkaStreams.close, it will in turn
block waiting for the StreamThread to join, hence the deadlock.
Runtime.addShutdownHook javadocs state:
{quote}
Shutdown hooks run at a delicate time in the life cycle of a virtual machine
and should therefore be coded defensively. They should, in particular, be
written to be thread-safe and to avoid deadlocks insofar as possible
{quote}
and
{quote}
Shutdown hooks should also finish their work quickly.
{quote}
Therefore the current implementation of KafkaStreams.close() which waits
forever for threads to join is completely unsuitable for use in a shutdown
hook.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
