Re: kafka sending duplicate content to consumer

ssages > that it just got. If you handle 500 of them and crash without committing > the offsets somewhere (either to kafka or in some other system). When you > restart the you will start your fetch again from the last committed offset. > Kafka has no notion of an already consumed message

kafka sending duplicate content to consumer

Hi, i want know in which situation does kafka send same event multiple times to consumer. Is there a consumer side configuration to tell kafka to send only once and stop retries? -- Regards Tousif Khazi

Re: kafka brokers going down within 24 hrs

any help ? On Mon, Jan 19, 2015 at 11:43 AM, Tousif wrote: > Here are the logs from broker id 0 and 1 and it was captured when broker > 1 went down. > > http://paste.ubuntu.com/9782553/ > http://paste.ubuntu.com/9782554/ > > > i'm using netty in s

Re: kafka brokers going down within 24 hrs

ty.Context" storm.messaging.netty.buffer_size: 209715200 storm.messaging.netty.max_retries: 10 storm.messaging.netty.max_wait_ms: 5000 storm.messaging.netty.min_wait_ms: 1 On Sat, Jan 17, 2015 at 1:24 AM, Harsha wrote: > Tousif, > I meant to say if kafka broker is going dow

Re: kafka brokers going down within 24 hrs

yteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54) at kafka.network.Processor.read(SocketServer.scala:347) at kafka.network.Processor.run(SocketServer.scala:245) at java.lang.Thread.run(Thread.java:662) On Thu, Jan 15, 2015 at 7:49 PM, Harsha wrote: > Tousif, >Which version of kafka

Re: kafka brokers going down within 24 hrs

g (kafka.server.ReplicaFetcherThread) [2015-01-15 09:58:14,156] INFO Closing socket connection to /10.0.0.11. (kafka.network.Processor) On Thu, Jan 15, 2015 at 12:01 PM, Chia-Chun Shih wrote: > You can use tools (e.g., VisialVM) to diagnose OOM problem. > > 2015-01-15 14:15 GMT+08:00

Re: kafka brokers going down within 24 hrs

, Tousif wrote: > Thanks harsha for quick response. > I don't see any other error. I used to see replica fetcher error but seems > to be disappeared after setting replica fetcher threads to 2 as I have 2 > partitions. Some times I see zookeeper session expiration. > On Jan 14, 2

Re: kafka brokers going down within 24 hrs

Thanks harsha for quick response. I don't see any other error. I used to see replica fetcher error but seems to be disappeared after setting replica fetcher threads to 2 as I have 2 partitions. Some times I see zookeeper session expiration. On Jan 14, 2015 8:02 PM, "Harsha" w

kafka brokers going down within 24 hrs

.0.11. (kafka.network.Processor) Error occurred during initialization of VM Could not reserve enough space for object heap Error occurred during initialization of VM Could not reserve enough space for object heap -- Regards, Tousif +918050227279 -- Regards Tousif Khazi