Todd Palino created KAFKA-1648:
----------------------------------
Summary: Round robin consumer balance throws an NPE when there are
no topics
Key: KAFKA-1648
URL: https://issues.apache.org/jira/browse/KAFKA-1648
Project: Kafka
Issue Type: Bug
Components: consumer
Reporter: Todd Palino
Assignee: Neha Narkhede
If you use the roundrobin rebalance method with a wildcard consumer, and there
are no topics in the cluster, rebalance throws a NullPointerException in the
consumer and fails. It retries the rebalance, but will continue to throw the
NPE.
2014/09/23 17:51:16.147 [ZookeeperConsumerConnector]
[kafka-audit_lva1-app0007.corp-1411494404908-4e620544], Cleared all relevant
queues for this fetcher
2014/09/23 17:51:16.147 [ZookeeperConsumerConnector]
[kafka-audit_lva1-app0007.corp-1411494404908-4e620544], Cleared the data chunks
in all the consumer message iterators
2014/09/23 17:51:16.148 [ZookeeperConsumerConnector]
[kafka-audit_lva1-app0007.corp-1411494404908-4e620544], Committing all offsets
after clearing the fetcher queues
2014/09/23 17:51:46.148 [ZookeeperConsumerConnector]
[kafka-audit_lva1-app0007.corp-1411494404908-4e620544], begin rebalancing
consumer kafka-audit_lva1-app0007.corp-1411494404908-4e620544 try #0
2014/09/23 17:51:46.148 ERROR [OffspringServletRuntime] [main]
[kafka-console-audit] [] Boot listener
com.linkedin.kafkaconsoleaudit.KafkaConsoleAuditBootListener failed
kafka.common.ConsumerRebalanceFailedException:
kafka-audit_lva1-app0007.corp-1411494404908-4e620544 can't rebalance after 10
retries
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:630)
at
kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:897)
at
kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:931)
at
kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:159)
at
kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:101)
at
com.linkedin.tracker.consumer.TrackingConsumerImpl.initWildcardIterators(TrackingConsumerImpl.java:88)
at
com.linkedin.tracker.consumer.TrackingConsumerImpl.getWildcardIterators(TrackingConsumerImpl.java:116)
at
com.linkedin.kafkaconsoleaudit.KafkaConsoleAudit.createAuditThreads(KafkaConsoleAudit.java:59)
at
com.linkedin.kafkaconsoleaudit.KafkaConsoleAudit.initializeAudit(KafkaConsoleAudit.java:50)
at
com.linkedin.kafkaconsoleaudit.KafkaConsoleAuditFactory.createInstance(KafkaConsoleAuditFactory.java:125)
at
com.linkedin.kafkaconsoleaudit.KafkaConsoleAuditFactory.createInstance(KafkaConsoleAuditFactory.java:20)
at
com.linkedin.util.factory.SimpleSingletonFactory.createInstance(SimpleSingletonFactory.java:20)
at
com.linkedin.util.factory.SimpleSingletonFactory.createInstance(SimpleSingletonFactory.java:14)
at com.linkedin.util.factory.Generator.doGetBean(Generator.java:337)
at com.linkedin.util.factory.Generator.getBean(Generator.java:270)
at
com.linkedin.kafkaconsoleaudit.KafkaConsoleAuditBootListener.onBoot(KafkaConsoleAuditBootListener.java:16)
at
com.linkedin.offspring.servlet.OffspringServletRuntime.startGenerator(OffspringServletRuntime.java:147)
at
com.linkedin.offspring.servlet.OffspringServletRuntime.start(OffspringServletRuntime.java:73)
at
com.linkedin.offspring.servlet.OffspringServletContextListener.contextInitialized(OffspringServletContextListener.java:28)
at
org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:771)
at
org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:424)
at
org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:763)
at
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:249)
at
org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1250)
at
org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:706)
at
org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:492)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at
com.linkedin.emweb.ContextBasedHandlerImpl.doStart(ContextBasedHandlerImpl.java:105)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at
com.linkedin.emweb.WebappDeployerImpl.start(WebappDeployerImpl.java:333)
at
com.linkedin.emweb.WebappDeployerImpl.deploy(WebappDeployerImpl.java:187)
at
com.linkedin.emweb.StateKeeperWebappDeployer.deploy(StateKeeperWebappDeployer.java:75)
at
com.linkedin.emweb.StateKeeperWebappDeployer.restore(StateKeeperWebappDeployer.java:62)
at
com.linkedin.emweb.WebappRunner.restoreServerState(WebappRunner.java:171)
at com.linkedin.emweb.BaseRunner.start(BaseRunner.java:96)
at
com.linkedin.spring.cmdline.ServerCmdLineApp.start(ServerCmdLineApp.java:86)
at
com.linkedin.spring.cmdline.ServerCmdLineApp.doRun(ServerCmdLineApp.java:102)
at
com.linkedin.spring.cmdline.CmdLineAppRunner.run(CmdLineAppRunner.java:246)
at
com.linkedin.spring.cmdline.CmdLineAppRunner.main(CmdLineAppRunner.java:480)
(note - the com.linkedin stuff in the stack trace is from the container we run
our applications in and does not affect the operation of the rebalance)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
