Are these messages sent to a newly created topic? Guozhang
On Mon, Feb 10, 2014 at 7:38 AM, Jun Rao <jun...@gmail.com> wrote: > If you reset the consumer offset and try to consume those messages again, > do you see the same drop? > > Thanks, > > Jun > > > On Mon, Feb 10, 2014 at 1:21 AM, A A <ek.tha....@hotmail.com> wrote: > > > Hi > > > > We have been using Kafka(0.8) for the past few months with the following > > setup > > Kafka Broker - 1Zookeepers Ensemble - 3Partitions per topic - 3 > > > > Yesterday, while running Stress tests in one of the QA machines , we > > observed that a few messages which were produced within a couple of > > milliseconds of each other did not reach the Kafka consumer. ie There was > > no trace of that message at the consumer end. > > > > We decided to check whether we had any errors at our side or there was a > > network issue. We did not find any issue. We then decided to check > whether > > we can find that message in one of the Kafka partitions. The message was > > found in one of the topic partitions. > > > > We are not sure why Kafka did not notify any consumers about the message. > > Are there any special cases where Kafka silently drops a message ? > > > > We also found a delay in the notifications/watches triggered from > > zookeeper. We are not sure whether these are related ? It will be > difficult > > to reproduce as the test probably took a few days to complete. But surely > > we did lose approximately 5% of the messages. We have logs of messages > > being produced at the producer side and corresponding entries in Kafka > > partitions logs. But nothing at the consumer side. The only repeating > > pattern was that the messages were probably produced within the same > > millisecond. So if you have a sequence of messages which was produced in > > the same millisecond like M0, M1, M2, M3 ie 4 messages. We probably have > > M0,M1,M3 but not M2. This is puzzling as to how only message is dropped > out > > of the given 4. > > > > > > We use the High Level Kafka Producer and Consumer. Both are single > > threaded(at our end). > > > > Does kafka need its own dedicated zookeeper ensemble ? We also use the > > same zookeeper ensemble as our configuration service. > > > > Unfortunately, we did not have DEBUG messages at the server enabled > during > > the setup. Although, NO error messages were observed during the same time > > period. > > > > > > Before we try running the same Tests again, can someone please shed more > > light as to the reasons why kafka dropped a few messages ? > > > > Kat > > > -- -- Guozhang
