[
https://issues.apache.org/jira/browse/KAFKA-17696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17887266#comment-17887266
]
黃竣陽 commented on KAFKA-17696:
-----------------------------
Hello [~lianetm], I want to confirm that my understanding is correct: Here are
the two issues:
# The `{{{}ConsumerNetworkThread`{}}} will not be able to send an
`{{{}ErrorEvent`{}}}, because in this case (Unauthorized topic), it does not
fall within the scope of {{metadata. `maybeThrowAnyException()`}} which handles
`{{{}recoverableException()`{}}}. Additionally, in the {{position}} method of
`{{{}AsyncKafkaConsumer`{}}}, there is no mechanism to handle
`{{{}ErrorEvent`{}}}.
# The `{{{}ConsumerNetworkThread{}}} `will keep polling until it times out.
The issue here is how to interrupt this behavior when encountering such cases
(e.g., Unauthorized topic).
In this cases if we send an ErrorEvent from {{ConsumerNetworkThread, it will be
fail by `}}
updateFetchPositions`due to it is blocking in
`ConsumerUtils.getResult(event.future())`.
I don't have more idea about this way about
OffsetsRequestManager.onMetadataError.
I will dig more in this issue
> New consumer background operations unaware of metadata errors
> -------------------------------------------------------------
>
> Key: KAFKA-17696
> URL: https://issues.apache.org/jira/browse/KAFKA-17696
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Reporter: Lianet Magrans
> Assignee: 黃竣陽
> Priority: Critical
> Labels: consumer-threading-refactor
> Fix For: 4.0.0
>
>
> When a metadata error happens (ie. Unauthorized topic), the network layer is
> the one to detect it and it just propagates it to the app thread via en
> ErrorEvent.
> [https://github.com/apache/kafka/blob/0edf5dbd204df9eb62bfea1b56993e95737df5a3/clients/src/main/java/org/apache/kafka/clients/consumer/internals/NetworkClientDelegate.java#L153]
> That allows api calls that processBackgroundEvents to throw the error in the
> app thread (ex. poll, unsubscribe and close, which are the only api calls
> that currently processBackgroundEvents).
> This means that all other api calls that do not processBackgroundEvent will
> never know about errors like Unauthorized topics. Moreover, it really means
> that the background operations are not notified/aborted when a metadata error
> happens (auth error). Ex. call to position block waiting for the
> updateFetchPositions
> ([here|https://github.com/apache/kafka/blob/0edf5dbd204df9eb62bfea1b56993e95737df5a3/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java#L1586]),
> will leave a
> pendingOffsetFetchEvent waiting to complete, even when the background already
> got an Unauthorized exception (but it only passed it to the app thread via
> ErrorEvent)
> I wonder if we should ensure that metadata errors are not only propagated to
> the app thread via ErrorEvents, but also ensure that we notify all request
> managers in the background (so that they can decide if completeExceptionally
> their outstanding events). Ex. OffsetsRequestManager.onMetadataError should
> completeExceptionally the pendingOffsetFetchEvent (just first thought, there
> could be other approaches, but note that calling processBackgroundEvent in
> api calls like positions will not do because we would block first on the
> CheckAndUpdatePositions, then processBackgroundEvents that would only happen
> after the CheckAndUpdate)
>
> This behaviour can be repro with the integration test
> AuthorizerIntegrationTest.testOffsetFetchWithNoAccess with the new consumer
> enabled (discovered with [https://github.com/apache/kafka/pull/17107] )
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
