Guido Raimondi created KAFKA-12589:
--------------------------------------
Summary: There are insufficient bytes available to read assignment
from the sync-group response (actual byte size 0) , this is not expected;
Key: KAFKA-12589
URL: https://issues.apache.org/jira/browse/KAFKA-12589
Project: Kafka
Issue Type: Bug
Components: clients, consumer
Affects Versions: 2.6.1
Environment: OS Windows 10 x64
Kafka server 2.6.0 / 2.6.1
Kafka Client 2.5.1 / 2.6.1
Reporter: Guido Raimondi
Hi,
when i try to subscribe my consumers i get this error :
java.lang.IllegalStateException: There are insufficient bytes available to read
assignment from the sync-group response (actual byte size 0) , this is not
expected; it is possible that the leader's assign function is buggy and did not
return any assignment for this member, or because static member is configured
and the protocol is buggy hence did not get the assignment for this
memberjava.lang.IllegalStateException: There are insufficient bytes available
to read assignment from the sync-group response (actual byte size 0) , this is
not expected; it is possible that the leader's assign function is buggy and did
not return any assignment for this member, or because static member is
configured and the protocol is buggy hence did not get the assignment for this
member at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinComplete(ConsumerCoordinator.java:367)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:440)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:359)
at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:513)
at
org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1268)
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1230)
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1210)
I have read the contents of the KAFKA-9801 and KAFKA-10772 bugs but have not
found any solutions.
My scenario scenario is this :
A . I'm writing a custom ConsumerPartitionAssignor. Very simple for this test.
B. i tried with the following versions of Kafka:
* Kafka Server ver 2.6.0 and client 2.5.1
* Kafka Server ver 2.6.1 and client ver 2.6.1
Steps :
1. I create through Spring Boot the topic "TEST_TOPIC" with 5 partitions ( 0-4)
2. I create a consumer and i subscribe it to TEST_TOPIC for all available
partitions. In my ConsumerPartitionAssignor implementation in the method
"assign" i have a Cluster data as follow :
metadata : Cluster(id = H0b9_2PXTdq_cfDZUKKN8w, nodes = [localhost:9092 (id: 0
rack: null)], partitions = [Partition(topic = TEST_TOPIC, partition = 2, leader
= 0, replicas = [0], isr = [0], offlineReplicas = []), Partition(topic =
TEST_TOPIC, partition = 1, leader = 0, replicas = [0], isr = [0],
offlineReplicas = []), Partition(topic = TEST_TOPIC, partition = 4, leader = 0,
replicas = [0], isr = [0], offlineReplicas = []), Partition(topic = TEST_TOPIC,
partition = 3, leader = 0, replicas = [0], isr = [0], offlineReplicas = []),
Partition(topic = TEST_TOPIC, partition = 0, leader = 0, replicas = [0], isr =
[0], offlineReplicas = [])],
3. The assign method returns the follow assignments :
Assignments \{TEST_TOPIC=Assignment(partitions=[TEST_TOPIC-4, TEST_TOPIC-2,
TEST_TOPIC-1, TEST_TOPIC-0, TEST_TOPIC-3])}
Soon after on the client i get :
[Thread-1] INFO org.apache.kafka.clients.consumer.internals.ConsumerCoordinator
- [Consumer clientId=DESKTOP-SV62B3Hd0ef0cf9-b51c-4659-992e-7eb7f98198a2,
groupId=TEST-MY-ASSIGNOR] Finished assignment for group at generation 1:
\{TEST_TOPIC=Assignment(partitions=[TEST_TOPIC-4, TEST_TOPIC-2, TEST_TOPIC-1,
TEST_TOPIC-0, TEST_TOPIC-3])} [Thread-1] INFO
org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer
clientId=DESKTOP-SV62B3Hd0ef0cf9-b51c-4659-992e-7eb7f98198a2,
groupId=TEST-MY-ASSIGNOR] Successfully synced group in generation
Generation\{generationId=1,
memberId='DESKTOP-SV62B3Hd0ef0cf9-b51c-4659-992e-7eb7f98198a2-0904f76a-67b4-4d21-bb7d-aa342ee830d1',
protocol='MYTEST'}
java.lang.IllegalStateException: There are insufficient bytes available to read
assignment from the sync-group response (actual byte size 0) , this is not
expected; it is possible that the leader's assign function is buggy and did not
return any assignment for this member, or because static member is configured
and the protocol is buggy hence did not get the assignment for this
memberjava.lang.IllegalStateException: There are insufficient bytes available
to read assignment from the sync-group response (actual byte size 0) , this is
not expected; it is possible that the leader's assign function is buggy and did
not return any assignment for this member, or because static member is
configured and the protocol is buggy hence did not get the assignment for this
member at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinComplete(ConsumerCoordinator.java:367)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:451)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:367)
at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:508)
at
org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1261)
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1230)
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1210)
While on the server :
[2021-03-31 13:25:46,151] DEBUG [GroupCoordinator 0]: Dynamic member with
unknown member id joins group TEST-MY-ASSIGNOR in Empty state. Created a new
member id
DESKTOP-SV62B3He45a46da-3ede-43d6-9599-aef5698a18a9-f2e02d3f-b79e-4382-91b3-a8c54e90fd49
and request the member to rejoin with this id.
(kafka.coordinator.group.GroupCoordinator) [2021-03-31 13:25:46,159] DEBUG
[GroupCoordinator 0]: Dynamic Member with specific member id
DESKTOP-SV62B3He45a46da-3ede-43d6-9599-aef5698a18a9-f2e02d3f-b79e-4382-91b3-a8c54e90fd49
joins group TEST-MY-ASSIGNOR in Empty state. Adding to the group now.
(kafka.coordinator.group.GroupCoordinator) [2021-03-31 13:25:46,160] DEBUG
Request key
MemberKey(DESKTOP-SV62B3Hd0ef0cf9-b51c-4659-992e-7eb7f98198a2-0904f76a-67b4-4d21-bb7d-aa342ee830d1)
unblocked 1 Heartbeat operations (kafka.server.DelayedOperationPurgatory)
[2021-03-31 13:25:46,168] INFO [GroupCoordinator 0]: Preparing to rebalance
group TEST-MY-ASSIGNOR in state PreparingRebalance with old generation 0
(__consumer_offsets-35) (reason: Adding new member
DESKTOP-SV62B3He45a46da-3ede-43d6-9599-aef5698a18a9-f2e02d3f-b79e-4382-91b3-a8c54e90fd49
with group instance id None) (kafka.coordinator.group.GroupCoordinator)
[2021-03-31 13:25:46,168] DEBUG Request key GroupKey(TEST-MY-ASSIGNOR)
unblocked 0 Rebalance operations (kafka.server.DelayedOperationPurgatory)
[2021-03-31 13:25:46,169] INFO [GroupCoordinator 0]: Stabilized group
TEST-MY-ASSIGNOR generation 1 (__consumer_offsets-35)
(kafka.coordinator.group.GroupCoordinator) [2021-03-31 13:25:46,169] DEBUG
Request key
MemberKey(DESKTOP-SV62B3Hd0ef0cf9-b51c-4659-992e-7eb7f98198a2-0904f76a-67b4-4d21-bb7d-aa342ee830d1)
unblocked 1 Heartbeat operations (kafka.server.DelayedOperationPurgatory)
[2021-03-31 13:25:46,388] INFO [GroupCoordinator 0]: Assignment received from
leader for group TEST-MY-ASSIGNOR for generation 1
(kafka.coordinator.group.GroupCoordinator) [2021-03-31 13:25:46,389] WARN
[GroupCoordinator 0]: Setting empty assignments for members
Set(DESKTOP-SV62B3He45a46da-3ede-43d6-9599-aef5698a18a9-f2e02d3f-b79e-4382-91b3-a8c54e90fd49)
of TEST-MY-ASSIGNOR for generation 1
(kafka.coordinator.group.GroupCoordinator) [2021-03-31 13:25:46,391] DEBUG
[Partition __consumer_offsets-35 broker=0] High watermark updated from
(offset=0 segment=[0:0]) to (offset=1 segment=[0:421])
(kafka.cluster.Partition) [2021-03-31 13:25:46,392] DEBUG Request key
TopicPartitionOperationKey(__consumer_offsets,35) unblocked 0 Fetch operations
(kafka.server.DelayedOperationPurgatory) [2021-03-31 13:25:46,392] DEBUG
Request key TopicPartitionOperationKey(__consumer_offsets,35) unblocked 0
Produce operations (kafka.server.DelayedOperationPurgatory) [2021-03-31
13:25:46,392] DEBUG Request key
TopicPartitionOperationKey(__consumer_offsets,35) unblocked 0 DeleteRecords
operations (kafka.server.DelayedOperationPurgatory) [2021-03-31 13:25:46,392]
DEBUG [ReplicaManager broker=0] Produce to local log in 2 ms
(kafka.server.ReplicaManager) [2021-03-31 13:25:46,393] WARN [GroupCoordinator
0]: Sending empty assignment to member
DESKTOP-SV62B3He45a46da-3ede-43d6-9599-aef5698a18a9-f2e02d3f-b79e-4382-91b3-a8c54e90fd49
of TEST-MY-ASSIGNOR for generation 1 with no errors
(kafka.coordinator.group.GroupCoordinator) [2021-03-31 13:25:46,394] DEBUG
Request key
MemberKey(DESKTOP-SV62B3Hd0ef0cf9-b51c-4659-992e-7eb7f98198a2-0904f76a-67b4-4d21-bb7d-aa342ee830d1)
unblocked 1 Heartbeat operations (kafka.server.DelayedOperationPurgatory)
Is it a bug or am I doing something wrong?
Thanks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
