backup(dump) and restore environment
Hi I'm looking for the best way to "dump" current system state, and recreate it on the new, autonomic environment. Lets say I'd like to create a copy of Production, and based on that, create new, separate environment for testing. Can You suggest some solutions? greetings Jakub
Re: backup(dump) and restore environment
Hello Jakub, Maybe it will work for you to combine MirrorMaker https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=27846330 and Burrow: https://github.com/linkedin/Burrow See recent announcement for Burrow http://mail-archives.apache.org/mod_mbox/kafka-users/201506.mbox/%3CCACrdVJpS3k1ZxCVHGqs0H5d7gfUUsQXdZ66DMRUEAccPrCOzvg%40mail.gmail.com%3E which mentions how it helps with monitoring MirrorMaker. Kind regards, Stevo Slavic. On Tue, Jun 9, 2015 at 11:00 AM, Jakub Muszynski wrote: > Hi > > I'm looking for the best way to "dump" current system state, and recreate > it on the new, autonomic environment. > Lets say I'd like to create a copy of Production, and based on that, create > new, separate environment for testing. > > Can You suggest some solutions? > > greetings > Jakub >
Re: backup(dump) and restore environment
Thanks for the quick anwser. The only think I'm not happy about is _no_ env separation. As far as I do understand, this is "online" operation, am I right? So the best would be to create mirror, and then cut it off form Production. greetings Jakub 2015-06-09 11:17 GMT+02:00 Stevo Slavić : > Hello Jakub, > > Maybe it will work for you to combine MirrorMaker > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=27846330 > and Burrow: https://github.com/linkedin/Burrow > See recent announcement for Burrow > > http://mail-archives.apache.org/mod_mbox/kafka-users/201506.mbox/%3CCACrdVJpS3k1ZxCVHGqs0H5d7gfUUsQXdZ66DMRUEAccPrCOzvg%40mail.gmail.com%3E > which mentions how it helps with monitoring MirrorMaker. > > Kind regards, > Stevo Slavic. > > On Tue, Jun 9, 2015 at 11:00 AM, Jakub Muszynski > wrote: > > > Hi > > > > I'm looking for the best way to "dump" current system state, and recreate > > it on the new, autonomic environment. > > Lets say I'd like to create a copy of Production, and based on that, > create > > new, separate environment for testing. > > > > Can You suggest some solutions? > > > > greetings > > Jakub > > >
Query on kafka broker and topic metadata
Hi, I am new to kafka and I have a doubt. How to read specified broker and topic statistics from kafka server? I want to read below parameters about existing topic from kafka. 1) How many activeMessages 2) How many activeSubscriptions 3) How many totalMessages 4) How many totalSubscriptions 5) How mnay deliveryFaults 6) How many pendingDelivery Pls do the needful. Thanks & Regards, Pavan
Kafka topic registration documentation.
Hi, I was wondering if the process of new topic registration is described somewhere. I'm using Go client to talk to kafka cluster and it would be awesome to ensure topic existence. I know there is auto creation configuration flag, but using this there is no way to use custom configuration for every topic. Is there some documentation what has to be done in order to ensure topic existence? Right now I'm using command line tool shipped with kafka binary, but I would prefer to be able to do this without jvm requirement.
Mirror maker doesn't rebalance after getting ZkNoNodeException
Hi, I have two mirror makers A and B both subscripting to the same whitelist. During topic rebalancing one of the mirror maker A encountered ZkNoNodeException and then stopped all connections. but mirror maker B didn't pick up the topics that were consumed by A and left some of the topics unassigned. I think this is due to A not releasing ownership of those topics. My question is why A didn't release ownership upon receiving error? Here is the stack trace [2015-06-08 15:55:46,379] INFO [test_some-ip-1433797157389-247c1fc4], exception during rebalance (kafka.consumer.ZookeeperConsumerConnector) org.I0Itec.zkclient.exception.ZkNoNodeException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /consumers/test/ids/test_some-ip-1433797157389-247c1fc4 at org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:47) at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685) at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:766) at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:761) at kafka.utils.ZkUtils$.readData(ZkUtils.scala:473) at kafka.consumer.TopicCount$.constructTopicCount(TopicCount.scala:61) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.kafka$consumer$ZookeeperConsumerConnector$ZKRebalancerListener$$rebalance(ZookeeperConsumerConnector.scala:657) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1$$anonfun$apply$mcV$sp$1.apply$mcVI$sp(ZookeeperConsumerConnector.scala:629) at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcV$sp(ZookeeperConsumerConnector.scala:620) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply(ZookeeperConsumerConnector.scala:620) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply(ZookeeperConsumerConnector.scala:620) at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:619) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anon$1.run(ZookeeperConsumerConnector.scala:572) Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /consumers/test/ids/test_some-ip-1433797157389-247c1fc4 at org.apache.zookeeper.KeeperException.create(KeeperException.java:111) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155) at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1184) at org.I0Itec.zkclient.ZkConnection.readData(ZkConnection.java:103) at org.I0Itec.zkclient.ZkClient$9.call(ZkClient.java:770) at org.I0Itec.zkclient.ZkClient$9.call(ZkClient.java:766) at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675) ... 13 more Here is the last part of the log [2015-06-08 15:55:49,848] INFO [ConsumerFetcherManager-1433797157399] Stopping leader finder thread (kafka.consumer.ConsumerFetcherManager) [2015-06-08 15:55:49,848] INFO [ConsumerFetcherManager-1433797157399] Stopping all fetchers (kafka.consumer.ConsumerFetcherManager) [2015-06-08 15:55:49,848] INFO [ConsumerFetcherManager-1433797157399] All connections stopped (kafka.consumer.ConsumerFetcherManager) [2015-06-08 15:55:49,848] INFO [test_some-ip-1433797157389-247c1fc4], Cleared all relevant queues for this fetcher (kafka.consumer.ZookeeperConsumerConnector) [2015-06-08 15:55:49,849] INFO [test_some-ip-1433797157389-247c1fc4], Cleared the data chunks in all the consumer message iterators (kafka.consumer.ZookeeperConsumerConnector) [2015-06-08 15:55:49,849] INFO [test_some-ip-1433797157389-247c1fc4], Invoking rebalance listener before relasing partition ownerships. (kafka.consumer.ZookeeperConsumerConnector) As seen in the log Mirror maker A didn't release ownership and it didn't attempt to trigger another round of rebalancing either. I checked zk. the node that was reported missing actually existed and it was created at the same time the error was thrown. I use the latest trunk code
ConsumerFetcherManager MaxLag
Hi - I am using kafka 0.8.2.1 I was hoping to monitor overall lag (for all partitions) for a high level consumer via JMX, but ConsumerFetcherManager MaxLag always returns value 0, despite the fact that I can see consumer offsets updating in zk (consumer is connected properly), and kafka.tools.ConsumerOffsetChecker is reporting accurate lag information. I have tried reading the value via jconsole, as well as using java jmx api (via MBeanServer). I’m sure I can get this information directly from ZK, but was hoping to just read the MBean attribute and be done. Do others see non-zero values for MaxLag reported to JMX? I noticed this conversation http://grokbase.com/t/kafka/users/14cb99wavh/jmx-mbean-of-gauge-long-is-a-java-lang-object about this mbean using Gauge to report the attribute value, and needing to cast to Long, but I don’t think this is the problem I’m seeing. Thanks Tyson
Re: Query on kafka topic metadata
Hi Pavan, 1) you cannot read the current number of messages for a topic, but you can read the number of bytes for a topic in Kafka metrics. 2) you cannot read the current number of subscribers directly. However you can read from ZK and parse the results to get the subscriber counts. 3) / 4) I am not sure what you mean by "active" messages and subscription. 5) / 6) You can find this in the Kafka metrics. You can take a look at the list of metrics here: http://kafka.apache.org/documentation.html#monitoring Guozhang On Fri, Jun 5, 2015 at 11:02 AM, Pavan Chenduluru wrote: > Hi, > > I am new to kafka and I have a doubt. > > How to read specified topic statistics from kafka server? > > I want to read below parameters about existing topic from kafka. > > 1) How many activeMessages > 2) How many activeSubscriptions > 3) How many totalMessages > 4) How many totalSubscriptions > 5) How mnay deliveryFaults > 6) How many pendingDelivery > > Pls do the needful. > > Thanks & Regards, > Pavan > -- -- Guozhang
Re: Mirror maker doesn't rebalance after getting ZkNoNodeException
Which version of MM are you running? On 6/9/15, 4:49 AM, "tao xiao" wrote: >Hi, > >I have two mirror makers A and B both subscripting to the same whitelist. >During topic rebalancing one of the mirror maker A encountered >ZkNoNodeException and then stopped all connections. but mirror maker B >didn't pick up the topics that were consumed by A and left some of the >topics unassigned. I think this is due to A not releasing ownership of >those topics. My question is why A didn't release ownership upon receiving >error? > >Here is the stack trace > >[2015-06-08 15:55:46,379] INFO [test_some-ip-1433797157389-247c1fc4], >exception during rebalance (kafka.consumer.ZookeeperConsumerConnector) >org.I0Itec.zkclient.exception.ZkNoNodeException: >org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = >NoNode for /consumers/test/ids/test_some-ip-1433797157389-247c1fc4 >at >org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:47) >at >org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685) >at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:766) >at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:761) >at kafka.utils.ZkUtils$.readData(ZkUtils.scala:473) >at >kafka.consumer.TopicCount$.constructTopicCount(TopicCount.scala:61) >at >kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.kafka$consu >mer$ZookeeperConsumerConnector$ZKRebalancerListener$$rebalance(ZookeeperCo >nsumerConnector.scala:657) >at >kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$sy >ncedRebalance$1$$anonfun$apply$mcV$sp$1.apply$mcVI$sp(ZookeeperConsumerCon >nector.scala:629) >at >scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141) >at >kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$sy >ncedRebalance$1.apply$mcV$sp(ZookeeperConsumerConnector.scala:620) >at >kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$sy >ncedRebalance$1.apply(ZookeeperConsumerConnector.scala:620) >at >kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$sy >ncedRebalance$1.apply(ZookeeperConsumerConnector.scala:620) >at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33) >at >kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebal >ance(ZookeeperConsumerConnector.scala:619) >at >kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anon$1.run >(ZookeeperConsumerConnector.scala:572) >Caused by: org.apache.zookeeper.KeeperException$NoNodeException: >KeeperErrorCode = NoNode for >/consumers/test/ids/test_some-ip-1433797157389-247c1fc4 >at >org.apache.zookeeper.KeeperException.create(KeeperException.java:111) >at >org.apache.zookeeper.KeeperException.create(KeeperException.java:51) >at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155) >at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1184) >at >org.I0Itec.zkclient.ZkConnection.readData(ZkConnection.java:103) >at org.I0Itec.zkclient.ZkClient$9.call(ZkClient.java:770) >at org.I0Itec.zkclient.ZkClient$9.call(ZkClient.java:766) >at >org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675) >... 13 more > > > >Here is the last part of the log > >[2015-06-08 15:55:49,848] INFO [ConsumerFetcherManager-1433797157399] >Stopping leader finder thread (kafka.consumer.ConsumerFetcherManager) > >[2015-06-08 15:55:49,848] INFO [ConsumerFetcherManager-1433797157399] >Stopping all fetchers (kafka.consumer.ConsumerFetcherManager) > >[2015-06-08 15:55:49,848] INFO [ConsumerFetcherManager-1433797157399] All >connections stopped (kafka.consumer.ConsumerFetcherManager) > >[2015-06-08 15:55:49,848] INFO [test_some-ip-1433797157389-247c1fc4], >Cleared all relevant queues for this fetcher >(kafka.consumer.ZookeeperConsumerConnector) > >[2015-06-08 15:55:49,849] INFO [test_some-ip-1433797157389-247c1fc4], >Cleared the data chunks in all the consumer message iterators >(kafka.consumer.ZookeeperConsumerConnector) > >[2015-06-08 15:55:49,849] INFO [test_some-ip-1433797157389-247c1fc4], >Invoking rebalance listener before relasing partition ownerships. >(kafka.consumer.ZookeeperConsumerConnector) > > >As seen in the log Mirror maker A didn't release ownership and it didn't >attempt to trigger another round of rebalancing either. I checked zk. the >node that was reported missing actually existed and it was created at the >same time the error was thrown. > > >I use the latest trunk code
offsets.storage=kafka, dual.commit.enabled=false still requires ZK
We are setting up a new Kafka project (0.8.2.1) and are trying to go straight to consumer offsets stored in Kafka. Unfortunately it looks like the Java consumer will try to connect to ZooKeeper regardless of the settings. Will/When will this dependency go away completely? It would simplify our deployments if our consumers didn't have to connect to ZooKeeper at all. P.S. I've asked this on Stack Overflow, if you would like to answer there for posterity: http://stackoverflow.com/questions/30719331/kafka-0-8-2-1-offsets-storage-kafka-still-requires-zookeeper
Re: offsets.storage=kafka, dual.commit.enabled=false still requires ZK
The existing consumer uses Zookeeper both to commit offsets and to assign partitions to different consumers and threads in the same consumer group. While offsets can be committed to Kafka in 0.8.2 releases, which greatly reduces the load on Zookeeper, the consumer still requires Zookeeper to manage group membership and partition ownership. The new consumer (which we hope to have ready for 0.8.3 release) will completely remove the Zookeeper dependency, managing both offsets and partition ownership within Kafka itself. Gwen On Tue, Jun 9, 2015 at 10:26 AM, noah wrote: > We are setting up a new Kafka project (0.8.2.1) and are trying to go > straight to consumer offsets stored in Kafka. Unfortunately it looks like > the Java consumer will try to connect to ZooKeeper regardless of the > settings. > > Will/When will this dependency go away completely? It would simplify our > deployments if our consumers didn't have to connect to ZooKeeper at all. > > P.S. I've asked this on Stack Overflow, if you would like to answer there > for posterity: > http://stackoverflow.com/questions/30719331/kafka-0-8-2-1-offsets-storage-kafka-still-requires-zookeeper
Stale topic for existing producer
BackgroundWe have a few consumer that *used* to consume a particular topic. This topic still exist and the consumers still exist, however, these consumer no longer subscribe to the topic. When we check the offset for the Consumer Group, kafka.tools.ConsumerOffsetChecker, ZK believes this tier is falling behind even though it's no longer attempting to consume this topic. Kafka StackKafka 0.8.1 with Zookeeper 3.3.4; Simple Consumer Question We keep track of the lag using the kafka.tools.ConsumerOffsetChecker to ensure we don't have a systematic issue we need to investigate. Is there any way to expire stale, or no longer consumed, topics per consumer group? Casey
Re: offsets.storage=kafka, dual.commit.enabled=false still requires ZK
The new consumer implementation, which should be included in 0.8.3, only needs a bootstrap.servers setting and does not use a zookeeper connection. On Tue, Jun 9, 2015 at 1:26 PM, noah wrote: > We are setting up a new Kafka project (0.8.2.1) and are trying to go > straight to consumer offsets stored in Kafka. Unfortunately it looks like > the Java consumer will try to connect to ZooKeeper regardless of the > settings. > > Will/When will this dependency go away completely? It would simplify our > deployments if our consumers didn't have to connect to ZooKeeper at all. > > P.S. I've asked this on Stack Overflow, if you would like to answer there > for posterity: > > http://stackoverflow.com/questions/30719331/kafka-0-8-2-1-offsets-storage-kafka-still-requires-zookeeper > -- Thanks, Ewen
Re: [ANNOUNCE] Burrow - Consumer Lag Monitoring as a Service
Hi Todd, Thanks for open sourcing this, I'm excited to take a look. It looks like it's specific to offsets stored in kafka (and not zookeeper) correct? I assume by that that LinkedIn is using the kafka storage now in production? Jason On Thu, Jun 4, 2015 at 9:43 PM, Todd Palino wrote: > I am very happy to introduce Burrow, an application to provide Kafka > consumer status as a service. Burrow is different than just a "lag > checker": > > * Multiple Kafka cluster support - Burrow supports any number of Kafka > clusters in a single instance. You can also run multiple copies of Burrow > in parallel and only one of them will send out notifications. > > * All consumers, all partitions - If the consumer is committing offsets to > Kafka (not Zookeeper), it will be available in Burrow automatically. Every > partition it consumes will be monitored simultaneously, avoiding the trap > of just watching the worst partition (MaxLag) or spot checking individual > topics. > > * Status can be checked via HTTP request - There's an internal HTTP server > that provides topic and consumer lists, can give you the latest offsets for > a topic either from the brokers or from the consumer, and lets you check > consumer status. > > * Continuously monitor groups with output via email or a call to an > external HTTP endpoint - Configure emails to send for bad groups, checked > continuously. Or you can have Burrow call an HTTP endpoint into another > system for handling alerts. > > * No thresholds - Status is determined over a sliding window and does not > rely on a fixed limit. When a consumer is checked, it has a status > indicator that tells whether it is OK, a warning, or an error, and the > partitions that caused it to be bad are provided. > > > Burrow was created to address specific problems that LinkedIn has with > monitoring consumers, in particular wildcard consumers like mirror makers > and our audit consumers. Instead of checking offsets for specific consumers > periodically, it monitors the stream of all committed offsets > (__consumer_offsets) and continually calculates lag over a sliding window. > > We welcome all feedback, comments, and contributors. This project is very > much under active development for us (we're using it in some of our > environments now, and working on getting it running everywhere to replace > our previous monitoring system). > > Burrow is written in Go, published under the Apache License, and hosted on > GitHub at: > https://github.com/linkedin/Burrow > > Documentation is on the GitHub wiki at: > https://github.com/linkedin/Burrow/wiki > > -Todd >
producer metadata behavior when topic not created
Hi, I am talking about the 0.8.2 java producer. In our deployment, we disables auto topic creation, because we would like to control the precise number of partitions created for each topic and the placement of partitions (e.g. zone-aware). I did some experimentation and checked the code. metadata request to broker (for non-exist topic) will got a successful response. should broker return failure or partial failure if queried topic doesn't exist? can we add metric at broker side for querying non-exist topics? The net behavior is that there are more metadata queries from producer, throttled by the backoff config (default is 100ms). can we add a metric for metadata request and response rate? rate should normally be very low during steady state, as default refresh interval is 5 mins. basically, I am trying to detect this scenario (non-exist topic) and be able to alert on some metrics. any other suggestions? Thanks, Steven
Re: [ANNOUNCE] Burrow - Consumer Lag Monitoring as a Service
For mirror maker and our audit application, we've been using Kafka-committed offsets for some time now. We've got a few other consumers who are using it, but we haven't actively worked on moving the bulk of them over. It's been less critical since we put the ZK transaction logs on SSD. And yeah, this is specific for kafka-committed offsets. I'm looking at some options for handling Zookeeper as well, but since our goal with this was to monitor our own infrastructure applications and move forwards, it hasn't gotten a lot of my attention yet. -Todd On Tue, Jun 9, 2015 at 11:53 AM, Jason Rosenberg wrote: > Hi Todd, > > Thanks for open sourcing this, I'm excited to take a look. > > It looks like it's specific to offsets stored in kafka (and not zookeeper) > correct? I assume by that that LinkedIn is using the kafka storage now in > production? > > Jason > > On Thu, Jun 4, 2015 at 9:43 PM, Todd Palino wrote: > > > I am very happy to introduce Burrow, an application to provide Kafka > > consumer status as a service. Burrow is different than just a "lag > > checker": > > > > * Multiple Kafka cluster support - Burrow supports any number of Kafka > > clusters in a single instance. You can also run multiple copies of Burrow > > in parallel and only one of them will send out notifications. > > > > * All consumers, all partitions - If the consumer is committing offsets > to > > Kafka (not Zookeeper), it will be available in Burrow automatically. > Every > > partition it consumes will be monitored simultaneously, avoiding the trap > > of just watching the worst partition (MaxLag) or spot checking individual > > topics. > > > > * Status can be checked via HTTP request - There's an internal HTTP > server > > that provides topic and consumer lists, can give you the latest offsets > for > > a topic either from the brokers or from the consumer, and lets you check > > consumer status. > > > > * Continuously monitor groups with output via email or a call to an > > external HTTP endpoint - Configure emails to send for bad groups, checked > > continuously. Or you can have Burrow call an HTTP endpoint into another > > system for handling alerts. > > > > * No thresholds - Status is determined over a sliding window and does not > > rely on a fixed limit. When a consumer is checked, it has a status > > indicator that tells whether it is OK, a warning, or an error, and the > > partitions that caused it to be bad are provided. > > > > > > Burrow was created to address specific problems that LinkedIn has with > > monitoring consumers, in particular wildcard consumers like mirror makers > > and our audit consumers. Instead of checking offsets for specific > consumers > > periodically, it monitors the stream of all committed offsets > > (__consumer_offsets) and continually calculates lag over a sliding > window. > > > > We welcome all feedback, comments, and contributors. This project is very > > much under active development for us (we're using it in some of our > > environments now, and working on getting it running everywhere to replace > > our previous monitoring system). > > > > Burrow is written in Go, published under the Apache License, and hosted > on > > GitHub at: > > https://github.com/linkedin/Burrow > > > > Documentation is on the GitHub wiki at: > > https://github.com/linkedin/Burrow/wiki > > > > -Todd > > >
Re: Mirror maker doesn't rebalance after getting ZkNoNodeException
I use commit 9e894aa0173b14d64a900bcf780d6b7809368384 from trunk code On Wed, 10 Jun 2015 at 01:09 Jiangjie Qin wrote: > Which version of MM are you running? > > On 6/9/15, 4:49 AM, "tao xiao" wrote: > > >Hi, > > > >I have two mirror makers A and B both subscripting to the same whitelist. > >During topic rebalancing one of the mirror maker A encountered > >ZkNoNodeException and then stopped all connections. but mirror maker B > >didn't pick up the topics that were consumed by A and left some of the > >topics unassigned. I think this is due to A not releasing ownership of > >those topics. My question is why A didn't release ownership upon receiving > >error? > > > >Here is the stack trace > > > >[2015-06-08 15:55:46,379] INFO [test_some-ip-1433797157389-247c1fc4], > >exception during rebalance (kafka.consumer.ZookeeperConsumerConnector) > >org.I0Itec.zkclient.exception.ZkNoNodeException: > >org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = > >NoNode for /consumers/test/ids/test_some-ip-1433797157389-247c1fc4 > >at > >org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:47) > >at > >org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685) > >at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:766) > >at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:761) > >at kafka.utils.ZkUtils$.readData(ZkUtils.scala:473) > >at > >kafka.consumer.TopicCount$.constructTopicCount(TopicCount.scala:61) > >at > >kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.kafka$consu > >mer$ZookeeperConsumerConnector$ZKRebalancerListener$$rebalance(ZookeeperCo > >nsumerConnector.scala:657) > >at > >kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$sy > >ncedRebalance$1$$anonfun$apply$mcV$sp$1.apply$mcVI$sp(ZookeeperConsumerCon > >nector.scala:629) > >at > >scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141) > >at > >kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$sy > >ncedRebalance$1.apply$mcV$sp(ZookeeperConsumerConnector.scala:620) > >at > >kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$sy > >ncedRebalance$1.apply(ZookeeperConsumerConnector.scala:620) > >at > >kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$sy > >ncedRebalance$1.apply(ZookeeperConsumerConnector.scala:620) > >at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33) > >at > >kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebal > >ance(ZookeeperConsumerConnector.scala:619) > >at > >kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anon$1.run > >(ZookeeperConsumerConnector.scala:572) > >Caused by: org.apache.zookeeper.KeeperException$NoNodeException: > >KeeperErrorCode = NoNode for > >/consumers/test/ids/test_some-ip-1433797157389-247c1fc4 > >at > >org.apache.zookeeper.KeeperException.create(KeeperException.java:111) > >at > >org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > >at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155) > >at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1184) > >at > >org.I0Itec.zkclient.ZkConnection.readData(ZkConnection.java:103) > >at org.I0Itec.zkclient.ZkClient$9.call(ZkClient.java:770) > >at org.I0Itec.zkclient.ZkClient$9.call(ZkClient.java:766) > >at > >org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675) > >... 13 more > > > > > > > >Here is the last part of the log > > > >[2015-06-08 15:55:49,848] INFO [ConsumerFetcherManager-1433797157399] > >Stopping leader finder thread (kafka.consumer.ConsumerFetcherManager) > > > >[2015-06-08 15:55:49,848] INFO [ConsumerFetcherManager-1433797157399] > >Stopping all fetchers (kafka.consumer.ConsumerFetcherManager) > > > >[2015-06-08 15:55:49,848] INFO [ConsumerFetcherManager-1433797157399] All > >connections stopped (kafka.consumer.ConsumerFetcherManager) > > > >[2015-06-08 15:55:49,848] INFO [test_some-ip-1433797157389-247c1fc4], > >Cleared all relevant queues for this fetcher > >(kafka.consumer.ZookeeperConsumerConnector) > > > >[2015-06-08 15:55:49,849] INFO [test_some-ip-1433797157389-247c1fc4], > >Cleared the data chunks in all the consumer message iterators > >(kafka.consumer.ZookeeperConsumerConnector) > > > >[2015-06-08 15:55:49,849] INFO [test_some-ip-1433797157389-247c1fc4], > >Invoking rebalance listener before relasing partition ownerships. > >(kafka.consumer.ZookeeperConsumerConnector) > > > > > >As seen in the log Mirror maker A didn't release ownership and it didn't > >attempt to trigger another round of rebalancing either. I checked zk. the > >node that was reported missing actually existed and it was created at the > >same time the error was thrown. > > > > > >I use the latest trunk code > >
Usage of Kafka Mirror Maker
Hi Team, I would like to know under which all circumstances I can use MirrorMaker process ? i read on a website that : "*it is not really intended as a fault-tolerance mechanism*" , so curious to know where all i can think of using MirrorMaker. Also, how to know the version of MirrorMaker process running in one of my datacenter? I see in below link that a better version is proposed https://cwiki.apache.org/confluence/display/KAFKA/KIP-3+-+Mirror+Maker+Enhancement Regards, Nitin Kumar Sharma.
