[
https://issues.apache.org/jira/browse/KAFKA-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744124#comment-17744124
]
Abhijeet Kumar commented on KAFKA-15181:
----------------------------------------
As far as I know, the ReplicaFetcher keeps retrying and should eventually be
able to find the metadata in the RLMM. We should not be blocking while waiting
for the consumers to catch up as the ReplicaFetcher acquires a global lock and
it will prevent other ReplicaFetchers from running as well as
LeaderAndIsrRequests.
We implemented a latching mechanism to address this scenario. I can raise a PR
to pull those changes to trunk.
> Race condition on partition assigned to TopicBasedRemoteLogMetadataManager
> ---------------------------------------------------------------------------
>
> Key: KAFKA-15181
> URL: https://issues.apache.org/jira/browse/KAFKA-15181
> Project: Kafka
> Issue Type: Sub-task
> Components: core
> Reporter: Jorge Esteban Quilcate Otoya
> Assignee: Jorge Esteban Quilcate Otoya
> Priority: Major
> Labels: tiered-storage
>
> TopicBasedRemoteLogMetadataManager (TBRLMM) uses a cache to be prepared
> whever partitions are assigned.
> When partitions are assigned to the TBRLMM instance, a consumer is started to
> keep the cache up to date.
> If the cache hasn't finalized to build, TBRLMM fails to return remote
> metadata about partitions that are store on the backing topic. TBRLMM may not
> recover from this failing state.
> A proposal to fix this issue would be wait after a partition is assigned for
> the consumer to catch up. A similar logic is used at the moment when TBRLMM
> writes to the topic, and uses send callback to wait for consumer to catch up.
> This logic can be reused whever a partition is assigned, so when TBRLMM is
> marked as initialized, cache is ready to serve requests.
> Reference: https://github.com/aiven/kafka/issues/33
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
