On our current project we want to have a single shell script that: first starts kafka with zookeeper and second strarts tomcat server with our webapp. We run kafka and tomcat on single machine, use kafka 0.8 We set num.partitions=4 in server.properties. In webapp we start 4 threads that do kafka.consumer.Consumer.createJavaConsumerConnector to create consumer connectors.
The logic of script is following: 1. run zookeeper-server-start.sh and wait until zk process appears(with linux pgrep command). 2. run kafka-server-start.sh and wait until kafka process appears. 3. sleep a fixed timeout(say 10 seconds) to let kafka completely initialize 4. start tomcat If timeout in step#3 is smaller we get following warning and consumers do not receive messages: WARN [kafka.consumer.ZookeeperConsumerConnector] [test-1381405301679-580f9706], No broker partitions consumed by consumer thread test-1381405301679-580f9706-3 for topic test-topic Is there a way to avoid this hardcoded timeout? How to check that partitions are initialized? Thanks, Viktor
