Interestingly, just noticed - at the exact time the commit request should have arrived at the server (15:44:14,365) there are a few moments of silence in both kafka-request.log and server.log:
kafka-request.log: [2015-09-18 15:44:14,355] TRACE 500 bytes read from /192.168.221.91:60967 (kafka.network.Processor) [2015-09-18 15:44:14,432] TRACE Processor id 2 selection time = 300358143 ns (kafka.network.Processor) server.log: [2015-09-18 15:44:14,355] DEBUG [KafkaApi-0] Putting fetch request with correlation id 33264 from client ReplicaFetcherThread-0-0 into purgatory (kafka.server.KafkaApis) [2015-09-18 15:44:14,432] TRACE Processor id 2 selection time = 300358143 ns (kafka.network.Processor) could this silence from 355 to 432 be a sign of trouble? (no significant GC activity around this time) On Fri, Sep 18, 2015 at 4:07 PM, Vadim Bobrov <vadimbob...@gmail.com> wrote: > Hi, > > I am trying to commit offsets manually to kafka and it sometimes works > sometimes not. What I found in kafka-request.log for the successful attempt > is this: > > [2015-09-18 15:22:38,031] TRACE [KafkaApi-0] Handling request: Name: > OffsetCommitRequest; Version: 1; CorrelationId: 0; ClientId: chaos-test; > GroupId: chaos-test; GroupGenerationId: -1; Con > sumerId: ; RequestInfo: [weather,0] -> > OffsetAndMetadata[112,NO_METADATA],[msmt,0] -> > OffsetAndMetadata[79749693,NO_METADATA] from client: > > > however when it fails there is nothing like this in kafka-request.log. > There are other requests, but no OffsetCommitRequest. Can anybody help? > > Thanks > Vadim > P.S. just for reference - here is the error I am getting on the consumer: > > DEBUG 2015-09-18 19:44:14,373 [chaos-akka.actor.jms-dispatcher-49 ] > kafka.consumer.ZookeeperConsumerConnector - > [chaos-test_ds2.outsmartinc.com-1442604229831-6bdb81c7], Connected to > offset ma > nager raq0.outsmartinc.com:9092. > TRACE 2015-09-18 19:44:14,376 [chaos-akka.actor.jms-dispatcher-49 ] > kafka.network.BoundedByteBufferSend - 113 bytes written. > ERROR 2015-09-18 19:44:14,380 [chaos-akka.actor.jms-dispatcher-49 ] > kafka.consumer.ZookeeperConsumerConnector - > [chaos-test_ds2.outsmartinc.com-1442604229831-6bdb81c7], Error while > committing > offsets. > java.io.EOFException: Received -1 when reading from channel, socket has > likely been closed. > at kafka.utils.Utils$.read(Utils.scala:381) > at > kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54) > at > kafka.network.Receive$class.readCompletely(Transmission.scala:56) > at > kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29) > at kafka.network.BlockingChannel.receive(BlockingChannel.scala:111) > at > kafka.consumer.ZookeeperConsumerConnector.liftedTree2$1(ZookeeperConsumerConnector.scala:313) > at > kafka.consumer.ZookeeperConsumerConnector.commitOffsets(ZookeeperConsumerConnector.scala:310) > at > kafka.javaapi.consumer.ZookeeperConsumerConnector.commitOffsets(ZookeeperConsumerConnector.scala: > > >
