[
https://issues.apache.org/jira/browse/KAFKA-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexandru Ionita updated KAFKA-4246:
------------------------------------
Description:
Trying to manually assign partition/topics to a consumer will not work
correctly. The consumer will be able to fetch records from the given
partitions, but the first commit will fail with the following message:
{code}
2016-10-03 13:44:50.673 DEBUG 11757 --- [pool-9-thread-1]
o.a.k.c.c.internals.ConsumerCoordinator : Offset commit for group XXXXXX
failed: The coordinator is not aware of this member.
2016-10-03 13:44:50.673 WARN 11757 --- [pool-9-thread-1]
o.a.k.c.c.internals.ConsumerCoordinator : Auto offset commit failed for group
XXXXXX: Commit cannot be completed since the group has already rebalanced and
assigned the partitions to another member. This means that the time between
subsequent calls to poll() was longer than the configured session.timeout.ms,
which typically implies that the poll loop is spending too much time message
processing. You can address this either by increasing the session timeout or by
reducing the maximum size of batches returned in poll() with max.poll.records.
{code}.
All this while the consumer will continue to poll records from the kafka
cluster, but every commit will fail with the same message.
I tried setting the {{session.timeout.ms}} to values like 50000, but I was
getting the same outcome => no successfull commits.
If I only switch from {{consumer.assign( subscribedPartitions )}} to
{{consumer.subscribe( topics )}}, everything works as expected. No other client
configurations should be changed to make it work.
Am I missing something here?
was:
Trying to manually assign partition/topics to a consumer will not work
correctly. The consumer will be able to fetch records from the given
partitions, but the first commit will fail with the following message:
{code}
2016-10-03 13:44:50.673 DEBUG 11757 --- [pool-9-thread-1]
o.a.k.c.c.internals.ConsumerCoordinator : Offset commit for group XXXXXX
failed: The coordinator is not aware of this member.
2016-10-03 13:44:50.673 WARN 11757 --- [pool-9-thread-1]
o.a.k.c.c.internals.ConsumerCoordinator : Auto offset commit failed for group
XXXXXX: Commit cannot be completed since the group has already rebalanced and
assigned the partitions to another member. This means that the time between
subsequent calls to poll() was longer than the configured session.timeout.ms,
which typically implies that the poll loop is spending too much time message
processing. You can address this either by increasing the session timeout or by
reducing the maximum size of batches returned in poll() with max.poll.records.
{code}.
All this while the consumer will continue to poll records from the kafka
cluster, but every commit will fail with the same message.
I tried setting the {{session.timeout.ms}} to values like 50000, but I was
getting the same outcome => no successfull commits.
If I only switch from {{consumer.assign( subscribedPartitions ) }} to
{{consumer.subscribe( topics ) }}, everything works as expected. No other
client configurations should be changed to make it work.
Am I missing something here?
> Discretionary partition assignment on the consumer side not functional
> ----------------------------------------------------------------------
>
> Key: KAFKA-4246
> URL: https://issues.apache.org/jira/browse/KAFKA-4246
> Project: Kafka
> Issue Type: Bug
> Components: clients
> Affects Versions: 0.10.0.1
> Reporter: Alexandru Ionita
>
> Trying to manually assign partition/topics to a consumer will not work
> correctly. The consumer will be able to fetch records from the given
> partitions, but the first commit will fail with the following message:
> {code}
> 2016-10-03 13:44:50.673 DEBUG 11757 --- [pool-9-thread-1]
> o.a.k.c.c.internals.ConsumerCoordinator : Offset commit for group XXXXXX
> failed: The coordinator is not aware of this member.
> 2016-10-03 13:44:50.673 WARN 11757 --- [pool-9-thread-1]
> o.a.k.c.c.internals.ConsumerCoordinator : Auto offset commit failed for
> group XXXXXX: Commit cannot be completed since the group has already
> rebalanced and assigned the partitions to another member. This means that the
> time between subsequent calls to poll() was longer than the configured
> session.timeout.ms, which typically implies that the poll loop is spending
> too much time message processing. You can address this either by increasing
> the session timeout or by reducing the maximum size of batches returned in
> poll() with max.poll.records.
> {code}.
> All this while the consumer will continue to poll records from the kafka
> cluster, but every commit will fail with the same message.
> I tried setting the {{session.timeout.ms}} to values like 50000, but I was
> getting the same outcome => no successfull commits.
> If I only switch from {{consumer.assign( subscribedPartitions )}} to
> {{consumer.subscribe( topics )}}, everything works as expected. No other
> client configurations should be changed to make it work.
> Am I missing something here?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
