Re: data corruption like behavior

2015-02-19 Thread Karts
[2015-02-05 14:21:09,708] ERROR [ReplicaFetcherThread-2-1], Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 147301; ClientId: ReplicaFetcherThread-2-1; ReplicaId: 3; MaxWait: 500 ms; MinBytes: 1 bytes; RequestInfo: [site.db.people,6] -> PartitionFetchInfo(0,1048576),[site.db.main,4] -

Re: data corruption like behavior

2015-02-19 Thread Jun Rao
Is there any error in the producer log? Is there any pattern in the messages being lost? Thanks, Jun On Thu, Feb 19, 2015 at 4:20 PM, Karts wrote: > yes i did. > > On Thu, Feb 19, 2015 at 2:42 PM, Jun Rao wrote: > > > Did you consume the messages from the beginning of the log? > > > > Thanks,

Re: data corruption like behavior

2015-02-19 Thread Karts
actually i take that back. it reads from where the last offset left off. On Thu, Feb 19, 2015 at 4:20 PM, Karts wrote: > yes i did. > > On Thu, Feb 19, 2015 at 2:42 PM, Jun Rao wrote: > >> Did you consume the messages from the beginning of the log? >> >> Thanks, >> >> Jun >> >> On Thu, Feb 19,

Re: data corruption like behavior

2015-02-19 Thread Karts
yes i did. On Thu, Feb 19, 2015 at 2:42 PM, Jun Rao wrote: > Did you consume the messages from the beginning of the log? > > Thanks, > > Jun > > On Thu, Feb 19, 2015 at 12:18 PM, Karts wrote: > > > but they have always been up. I mean when i was testing, all the > zookeepers > > were up. and al

Re: data corruption like behavior

2015-02-19 Thread Jun Rao
Did you consume the messages from the beginning of the log? Thanks, Jun On Thu, Feb 19, 2015 at 12:18 PM, Karts wrote: > but they have always been up. I mean when i was testing, all the zookeepers > were up. and all the kafka nodes were up. its just that I changed the > number of zookeeper nod

Re: data corruption like behavior

2015-02-19 Thread Karts
but they have always been up. I mean when i was testing, all the zookeepers were up. and all the kafka nodes were up. its just that I changed the number of zookeeper nodes in my first test iteration. second and third were still the same. not sure why the topics were losing some messages. On Thu, F

Re: data corruption like behavior

2015-02-19 Thread Jun Rao
Zookeeper requires a majority of the nodes to be up for the service to be available. Kafka relies on Zookeeper to be always available. Thanks, Jun On Thu, Feb 19, 2015 at 11:15 AM, Karts wrote: > I have noticed some strange patterns when testing with the 0.8.1 build and > the 0.8.2 builds, and