If you reset the consumer offset and try to consume those messages again, do you see the same drop?
Thanks, Jun On Mon, Feb 10, 2014 at 1:21 AM, A A <ek.tha....@hotmail.com> wrote: > Hi > > We have been using Kafka(0.8) for the past few months with the following > setup > Kafka Broker - 1Zookeepers Ensemble - 3Partitions per topic - 3 > > Yesterday, while running Stress tests in one of the QA machines , we > observed that a few messages which were produced within a couple of > milliseconds of each other did not reach the Kafka consumer. ie There was > no trace of that message at the consumer end. > > We decided to check whether we had any errors at our side or there was a > network issue. We did not find any issue. We then decided to check whether > we can find that message in one of the Kafka partitions. The message was > found in one of the topic partitions. > > We are not sure why Kafka did not notify any consumers about the message. > Are there any special cases where Kafka silently drops a message ? > > We also found a delay in the notifications/watches triggered from > zookeeper. We are not sure whether these are related ? It will be difficult > to reproduce as the test probably took a few days to complete. But surely > we did lose approximately 5% of the messages. We have logs of messages > being produced at the producer side and corresponding entries in Kafka > partitions logs. But nothing at the consumer side. The only repeating > pattern was that the messages were probably produced within the same > millisecond. So if you have a sequence of messages which was produced in > the same millisecond like M0, M1, M2, M3 ie 4 messages. We probably have > M0,M1,M3 but not M2. This is puzzling as to how only message is dropped out > of the given 4. > > > We use the High Level Kafka Producer and Consumer. Both are single > threaded(at our end). > > Does kafka need its own dedicated zookeeper ensemble ? We also use the > same zookeeper ensemble as our configuration service. > > Unfortunately, we did not have DEBUG messages at the server enabled during > the setup. Although, NO error messages were observed during the same time > period. > > > Before we try running the same Tests again, can someone please shed more > light as to the reasons why kafka dropped a few messages ? > > Kat >
