[
https://issues.apache.org/jira/browse/KAFKA-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370143#comment-17370143
]
kiran commented on KAFKA-4246:
------------------------------
Hello Experts,
I am using kafka 2.12 and spark-sql-kafka-0-10_2.12
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_${scala.tools.version}</artifactId>
<version>${kafka.version}</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql-kafka-0-10
-->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql-kafka-0-10_2.12</artifactId>
<version>${spark.version}</version>
<scope>test</scope>
</dependency>
I have been getting this error below. Would appreciate any inputs what exactly
I need to change/add? In my case the consumer is a spark streaming application
using DStreams
2021-06-25 04:05:21 ERROR AbstractCoordinator.java:950 - [Consumer
clientId=consumer-kafkaToCassandraWriterGroup-2,
groupId=kafkaToCassandraWriterGroup] LeaveGroup request failed with error: The
coordinator is not aware of this member.
2021-06-25 04:10:21 ERROR AbstractCoordinator.java:950 - [Consumer
clientId=consumer-kafkaToCassandraWriterGroup-3,
groupId=kafkaToCassandraWriterGroup] LeaveGroup request failed with error: The
coordinator is not aware of this member.
> Discretionary partition assignment on the consumer side not functional
> ----------------------------------------------------------------------
>
> Key: KAFKA-4246
> URL: https://issues.apache.org/jira/browse/KAFKA-4246
> Project: Kafka
> Issue Type: Bug
> Components: clients
> Affects Versions: 0.10.0.1
> Reporter: Alexandru Ionita
> Priority: Major
>
> Trying to manually assign partition/topics to a consumer will not work
> correctly. The consumer will be able to fetch records from the given
> partitions, but the first commit will fail with the following message:
> {code}
> 2016-10-03 13:44:50.673 DEBUG 11757 --- [pool-9-thread-1]
> o.a.k.c.c.internals.ConsumerCoordinator : Offset commit for group XXXXXX
> failed: The coordinator is not aware of this member.
> 2016-10-03 13:44:50.673 WARN 11757 --- [pool-9-thread-1]
> o.a.k.c.c.internals.ConsumerCoordinator : Auto offset commit failed for
> group XXXXXX: Commit cannot be completed since the group has already
> rebalanced and assigned the partitions to another member. This means that the
> time between subsequent calls to poll() was longer than the configured
> session.timeout.ms, which typically implies that the poll loop is spending
> too much time message processing. You can address this either by increasing
> the session timeout or by reducing the maximum size of batches returned in
> poll() with max.poll.records.
> {code}.
> All this while the consumer will continue to poll records from the kafka
> cluster, but every commit will fail with the same message.
> I tried setting the {{session.timeout.ms}} to values like 50000, but I was
> getting the same outcome => no successfull commits.
> If I only switch from {{consumer.assign( subscribedPartitions )}} to
> {{consumer.subscribe( topics )}}, everything works as expected. No other
> client configurations should be changed to make it work.
> Am I missing something here?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
