Re: Consumer behavior when Kafka rolls the old log file

2018-01-02 Thread Brett Rann
The old segment isn't rolled, a new one is. eg, take this partition directory: drwxr-xr-x2 kafka kafka 4096 Jan 1 10:31 . drwxr-xr-x 1483 kafka kafka94208 Jan 3 02:11 .. -rw-r--r--1 kafka kafka15504 Jan 1 10:31 04050355.index -rw-r--r--1 kafka kafka 8210576

Re: Kafka Streams 0.11 consumers losing offsets for all group.ids

2018-01-02 Thread Adam Gurson
Thank you for the response! The offsets.topic.replication.factor is set to 2 for Cluster A (the size of the cluster). It is 3 for Cluster B, but the number of in-sync replicas was manually increased to 4 (cluster size) for the the __consumer_offsets topic after the cluster was created. In addition

Re: Unable to start 1st broker in a 3 node configuration

2018-01-02 Thread Ted Yu
bq. zookeeper.connect = localhost:2181,eg2-pp-ifs-245: 2181,eg2-pp-ifs-219:*9092* Why did 9092 appear in zookeeper setting ? Cheers On Tue, Jan 2, 2018 at 2:18 AM, M. Manna wrote: > Hi All, > > Firstly a very Happy New Year! > > I set up my 3 node configuration where each of the broke

Unable to start 1st broker in a 3 node configuration

2018-01-02 Thread M. Manna
Hi All, Firstly a very Happy New Year! I set up my 3 node configuration where each of the broker is set to have identical configurations. They are in in three different servers, but within the same domain. I have got a very simply Windows script that does the following: 1) Starts each zookeeper

Consumer behavior when Kafka rolls the old log file

2018-01-02 Thread Debraj Manna
Can someone let me know how does consumer behaves when Kafka rolls the old log file (e.g. i.e. INFO Rolled new log segment for 'topic-{partition}') while the consumer is still consuming the log segment ?