[
https://issues.apache.org/jira/browse/KAFKA-7044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16553496#comment-16553496
]
Vahid Hashemian edited comment on KAFKA-7044 at 7/23/18 11:57 PM:
------------------------------------------------------------------
[~ejpearson], thanks for providing the additional info. It seems that the call
to {{endOffsets(...)}} leads to a call to {{Fetcher.fetchOffsetsByTimes(...)}}
that involves a timeout. The default timeout that is currently in place is
quite long (30 secs), but I wonder if that somehow kicks in causing the call to
finding the end offset for the given partitions return with a partial list of
end offsets. Is it possible for you to increase the timeout and observe if
things will change on your side (for example, you can use
{{getConsumer.endOffsets(topicPartitions.asJava, 60000)}} to double the
timeout)?
Is it always the same partition 11 that causes the NPE for you? Is there
anything (size, lag, ...) different about that partition?
I'm also interested to know what comes back from this call in
{{Fetcher.sendListOffsetsRequest(...)}}:
{code:java}
RequestFuture<ListOffsetResult> future =
sendListOffsetRequest(entry.getKey(), entry.getValue(),
requireTimestamps);{code}
BTW, this seems to be similar to the issue reported in
[KAFKA-6662|https://issues.apache.org/jira/browse/KAFKA-6662].
was (Author: vahid):
[~ejpearson], thanks for providing the additional info. It seems that the call
to {{endOffsets(...)}} leads to a call to {{Fetcher.fetchOffsetsByTimes(...)}}
that involves a timeout. The default timeout that is currently in place is
quite long (30 secs), but I wonder if that somehow kicks in causing the call to
finding the end offset for the given partitions return with a partial list of
end offsets. Is it possible for you to increase the timeout and observe if
things will change on your side (for example, you can use
{{getConsumer.endOffsets(topicPartitions.asJava, 60000)}} to double the
timeout)?
Is it always the same partition 11 that causes the NPE for you? Is there
anything (size, lag, ...) different about that partition?
I'm also interested to know what comes back from this call in
{{Fetcher.sendListOffsetsRequest(...)}}:
{code:java}
RequestFuture<ListOffsetResult> future =
sendListOffsetRequest(entry.getKey(), entry.getValue(),
requireTimestamps);{code}
> kafka-consumer-groups.sh NullPointerException describing round robin or
> sticky assignors
> ----------------------------------------------------------------------------------------
>
> Key: KAFKA-7044
> URL: https://issues.apache.org/jira/browse/KAFKA-7044
> Project: Kafka
> Issue Type: Bug
> Components: tools
> Affects Versions: 1.1.0
> Environment: CentOS 7.4, Oracle JDK 1.8
> Reporter: Jeff Field
> Assignee: Vahid Hashemian
> Priority: Minor
>
> We've recently moved to using the round robin assignor for one of our
> consumer groups, and started testing the sticky assignor. In both cases,
> using Kafka 1.1.0 we get a null pointer exception *unless* the group being
> described is rebalancing:
> {code:java}
> kafka-consumer-groups --bootstrap-server fqdn:9092 --describe --group
> groupname-for-consumer
> Error: Executing consumer group command failed due to null
> [2018-06-12 01:32:34,179] DEBUG Exception in consumer group command
> (kafka.admin.ConsumerGroupCommand$)
> java.lang.NullPointerException
> at scala.Predef$.Long2long(Predef.scala:363)
> at
> kafka.admin.ConsumerGroupCommand$KafkaConsumerGroupService$$anonfun$getLogEndOffsets$2.apply(ConsumerGroupCommand.scala:612)
> at
> kafka.admin.ConsumerGroupCommand$KafkaConsumerGroupService$$anonfun$getLogEndOffsets$2.apply(ConsumerGroupCommand.scala:610)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.immutable.List.foreach(List.scala:392)
> at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
> at scala.collection.immutable.List.map(List.scala:296)
> at
> kafka.admin.ConsumerGroupCommand$KafkaConsumerGroupService.getLogEndOffsets(ConsumerGroupCommand.scala:610)
> at
> kafka.admin.ConsumerGroupCommand$ConsumerGroupService$class.describePartitions(ConsumerGroupCommand.scala:328)
> at
> kafka.admin.ConsumerGroupCommand$ConsumerGroupService$class.collectConsumerAssignment(ConsumerGroupCommand.scala:308)
> at
> kafka.admin.ConsumerGroupCommand$KafkaConsumerGroupService.collectConsumerAssignment(ConsumerGroupCommand.scala:544)
> at
> kafka.admin.ConsumerGroupCommand$KafkaConsumerGroupService$$anonfun$10$$anonfun$13.apply(ConsumerGroupCommand.scala:571)
> at
> kafka.admin.ConsumerGroupCommand$KafkaConsumerGroupService$$anonfun$10$$anonfun$13.apply(ConsumerGroupCommand.scala:565)
> at scala.collection.immutable.List.flatMap(List.scala:338)
> at
> kafka.admin.ConsumerGroupCommand$KafkaConsumerGroupService$$anonfun$10.apply(ConsumerGroupCommand.scala:565)
> at
> kafka.admin.ConsumerGroupCommand$KafkaConsumerGroupService$$anonfun$10.apply(ConsumerGroupCommand.scala:558)
> at scala.Option.map(Option.scala:146)
> at
> kafka.admin.ConsumerGroupCommand$KafkaConsumerGroupService.collectGroupOffsets(ConsumerGroupCommand.scala:558)
> at
> kafka.admin.ConsumerGroupCommand$ConsumerGroupService$class.describeGroup(ConsumerGroupCommand.scala:271)
> at
> kafka.admin.ConsumerGroupCommand$KafkaConsumerGroupService.describeGroup(ConsumerGroupCommand.scala:544)
> at kafka.admin.ConsumerGroupCommand$.main(ConsumerGroupCommand.scala:77)
> at kafka.admin.ConsumerGroupCommand.main(ConsumerGroupCommand.scala)
> [2018-06-12 01:32:34,255] DEBUG Removed sensor with name connections-closed:
> (org.apache.kafka.common.metrics.Metrics){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
