Hasil Sharma created KAFKA-19385: ------------------------------------ Summary: Unexpectdly stuck consumption on tiered storage enabled topics Key: KAFKA-19385 URL: https://issues.apache.org/jira/browse/KAFKA-19385 Project: Kafka Issue Type: Bug Components: Tiered-Storage Affects Versions: 3.8.1 Reporter: Hasil Sharma
Problem: We were unable to consume data from beginning on remote storage enabled topic partitions. Context: We have enabled fetch from nearest replica enabled on the cluster, and running Kafka version 3.8.1 Repro: We ran the command below for several minutes and received no records. Re-running the same command with different client.rack returns multiple records (as expected). ``` $ /pay/kafka/install/current/bin/kafka-console-consumer.sh --bootstrap-server some-broker:9094 --topic archiver.default.v2 --partition 90 --from-beginning --consumer-property client.rack=us-east-2a ^CProcessed a total of 0 messages ``` For the duration of the command execution, the broker in us-east-2a with replica for topic partition showed thousands of DEBUG log lines (and no ERROR) below: ``` [2025-06-06 01:02:03.568099] DEBUG RemoteLogReader:81 - Finished reading records from remote storage for topic partition archiver.default.v2-90 [2025-06-06 01:02:03.568056] DEBUG RemoteLogReader:68 - Reading records from remote storage for topic partition archiver.default.v2-90 ``` Looks like `rlm.read` is throwing exception OffsetOutOfRangeException. -- This message was sent by Atlassian Jira (v8.20.10#820010)