Message loss due to zookeeper ensemble doesn't work

Hi, I have the offset saved in zookeeper. Because zookeeper quorum doesn't work for a short time(leader is down and new leader election).Then there is a chance that the offset doesn't write to the Zookeeper, which will lose data. I would ask whether Kafka provides some mechism for this kind of

replication factor: 1 larger than available brokers: 0 issue

Hello, We have been using Kafka 0.8.1 for a few months and this morning we started seeing a lot of these types of errors. We started trying to publish messages to a few additional topics, but topic creation (automatic creation mode) always fails with the following type of error: kafka.common.

Kafka rebalancing and general questions

Hi all, I've very recently become interested in Kafka and have studied it a great deal but find myself confused on some of the more basic design aspects of the technology and despite considering various online resources still haven't found the answer I was looking for -- hoping this community coul

Re: Does kafka write key to broker?

Thanks Jason. Hi Liquan, I was doing this initially using KeyedMessage without setting the key. But if the key is null, the custom partitioner won't be used and kafka will write to random partition which is cached for some time within the producer. I have a custom partitioner to have fairness bet

Re: Best Practices for Java Consumers

+1 On this idea. On Tue, Jun 23, 2015 at 5:55 PM, Gwen Shapira wrote: > I don't know of any such resource, but I'll be happy to help > contribute from my experience. > I'm sure others would too. > > Do you want to start one? > > Gwen > > On Tue, Jun 23, 2015 at 2:03 PM, Tom McKenzie > wrote: >

Re: data loss - replicas

i ran the following in both servers * kafka-run-class.sh kafka.tools.DumpLogSegments --files /tmp/kafka-logs/jun8-6/21764229.log* It seems you might have run that on the last log segment. Can you run it on 21764229.log on both brokers and compare? I'm

Re: How to run Kafka in background

You can pass "-daemon" config property to kafka startup script. ./kafka-server-start.sh -daemon ../config/server.1.properties On Wed, Jun 24, 2015 at 4:14 PM, bit1...@163.com wrote: > Hi, > > I am using kafak 0.8.2.1 , and when I startup Kafka with the script: > ./kafka-server-start.sh ../config

Re: How to run Kafka in background

try it as nohup ./kafka-server-start.sh ../config/server.1.properties & On Wed, Jun 24, 2015 at 4:14 PM, bit1...@163.com wrote: > Hi, > > I am using kafak 0.8.2.1 , and when I startup Kafka with the script: > ./kafka-server-start.sh ../config/server.1.properties & > > I think it will run as bac

How to run Kafka in background

Hi, I am using kafak 0.8.2.1 , and when I startup Kafka with the script: ./kafka-server-start.sh ../config/server.1.properties & I think it will run as background process, but when i close the terminal, the server is shutdown, which looks that it doesn't run in background Then how can I run

Re: Re: No key specified when sending the message to Kafka

Thank you,Ewen. I undertand it now. bit1...@163.com From: Ewen Cheslack-Postava Date: 2015-06-24 11:53 To: users@kafka.apache.org Subject: Re: No key specified when sending the message to Kafka It does balance data, but is sticky over short periods of time (for some definition of short...).