This is probably your issue: https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-IamusingtheZK-basedproducerin0.7andIseedataonlyproducedonsomeofthebrokers,butnotall,why ?
Thanks, Jun On Mon, Feb 10, 2014 at 4:04 PM, Tomas Nunez <nu...@pythian.com> wrote: > I don't see anything in kafabe03. In fact, I just restarted it, and I > saw in the logs a lot of other topics registering. First, lines like > this: > (...) > [2014-02-10 20:30:07,195] INFO Loading log 'topic2-0' > (kafka.log.LogManager) > [2014-02-10 20:30:07,201] INFO Loading the last segment > /var/kafka/topic2-0/00000000000559732292.kafka in mutable mode, > recovery false (kafka.log.Log) > (...) > [2014-02-10 20:30:07,464] INFO Registering broker /brokers/ids/3 > (kafka.server.KafkaZooKeeper) > [2014-02-10 20:30:07,486] INFO Registering broker /brokers/ids/3 > succeeded with > id:3,creatorId:kafkabe03-1392064207464,host:kafkabe03,port:9092 > (kafka.server.KafkaZooKeeper) > [2014-02-10 20:30:07,516] INFO Begin registering broker topic > /brokers/topics/topic2/3 with 1 partitions > (kafka.server.KafkaZooKeeper) > [2014-02-10 20:30:07,522] INFO End registering broker topic > /brokers/topics/topic2/3 (kafka.server.KafkaZooKeeper) > (...) > > But no trace of this particular topic, topic1. > > Any idea why this particular topic isn't registering? Where can I look? > > Thanks again > Tomàs > > > One of the be brokers (3) is not registered in ZK. Do you see ZK session > expiration (potentially due to GC) in that broker? > > Thanks, > > Jun > > > > > On Mon, Feb 10, 2014 at 4:28 PM, Tomas Nunez <nu...@pythian.com> wrote: > > > Hi > > > > I'm new around here and I'm dealing with a problem, and reading the > > documentation I don't know where else to look. > > > > I have a cross-dc mirrormaker setup: Mirrormaker is consuming from 5 > > frontend servers in each DC (10 in total) and 5 backend servers are > > consuming from mirrormaker. That's working for most of the topics, but > some > > of them are not being consumed from all backend servers. > > > > as an example, browsing with zkcli I find this: > > > > kafka-0.7.2-incubating-src/bin$ /usr/share/zookeeper/bin/zkCli.sh -server > > zookeeperbe01:2181 ls /brokers/topics/topic1 > > Connecting to zookeeperbe01:2181 > > > > WATCHER:: > > > > WatchedEvent state:SyncConnected type:None path:null > > [2, 1, 5, 4] > > kafka-0.7.2-incubating-src/bin$ /usr/share/zookeeper/bin/zkCli.sh -server > > zookeeperfe01:2181 ls /brokers/topics/topic1 > > Connecting to zookeeperfe01:2181 > > > > WATCHER:: > > > > WatchedEvent state:SyncConnected type:None path:null > > [3, 2, 1, 5, 4] > > > > What can be wrong? Where can I look to get more info to troubleshoot > this? > > Any hint? > > > > Kafka version is 0.7.2, in mirromaker config I have "--num.streams 5 > > --num.producers 5". Zookeeper version is 3.3.5, > > > > Here you can see kafka-console-consumer.sh connecting to both fe and be > > servers: > > https://gist.github.com/pythiannunez/1623934cb538678f053e > > > > Thanks! > > > > > > -- > > > -- > > > >
