fuyusakaiori opened a new issue, #9006: URL: https://github.com/apache/rocketmq/issues/9006
### Before Creating the Bug Report - [X] I found a bug, not just asking a question, which should be created in [GitHub Discussions](https://github.com/apache/rocketmq/discussions). - [X] I have searched the [GitHub Issues](https://github.com/apache/rocketmq/issues) and [GitHub Discussions](https://github.com/apache/rocketmq/discussions) of this repository and believe that this is not a duplicate. - [X] I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ. ### Runtime platform environment CentOS Linux 8 ### RocketMQ version version: 5.1.4 ### JDK Version java version "1.8.0_221" Java(TM) SE Runtime Environment (build 1.8.0_221-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.221-b11, mixed mode) ### Describe the Bug 1) 延时消息的延时时间超过本地存储的过期时间，达到过期时间时就会被清理掉，而此时因为延时消息没有被投递到真实的 Topic，所以就会发现多级存储也没有对应的延时消息，最后消费者就无法消费这条延时消息 2) 这样就会导致即使配置了多级存储，本地存储的过期时间还会受制于延时消息的延时时间，这样就还是没有办法减少本地存储的过期时间 (1) If the delay time of a delayed message exceeds the default storage expiration time, the message will be cleaned up when it expires. However, since the delayed message hasn’t been delivered to the actual topic yet, there will be no corresponding delayed message in the tiered storage. As a result, the consumer won’t be able to consume this delayed message. (2)This means that even though tiered storage is configured, the expiration time of the default storage is still affected by the delay time of the message. Therefore, it's not possible to reduce the expiration time of the local storage. ### Steps to Reproduce 1) 本地存储设置过期时间为 1H，多级存储设置过期时间为 6H，删除过期消息的时间为 1H 后 2) 分别发送 1 条延时消息等级为 17 (1H) 和等级为 18 (2H) 的延时消息到延时消息的 Topic 3) 然后持续发送大量普通消息到普通消息的 Topic，确保会生成多个 CommitLog，确保过期时可以删除延时消息 4) 等待 1H 后启动消费者消费延时消息，会发现延时等级为 17 的消息可以消费，但是延时等级为 18 的消息无法消费 (1) The default storage expiration time is set to 1 hour, while the tiered storage expiration time is set to 6 hours. Expired messages are deleted 1 hour after expiration. (2) A delayed message with level 17 (1 hour delay) and another with level 18 (2 hours delay) are sent to the delayed message topic. (3) At the same time, a large number of regular messages are sent to the normal message topic to ensure multiple CommitLogs are generated, ensuring that expired delayed messages can be deleted. (4) After waiting for 1 hour, a consumer is started to consume the delayed messages. It is observed that the delayed message with level 17 (1 hour delay) can be consumed, but the message with level 18 (2 hours delay) cannot be consumed. ### What Did You Expect to See? 1) 梳理源码发现是延时消息的对应的系统 Topic 没有上传到多级存储，最终导致本地延时消息过期后，也没法把延时消息投递真实的 Topic 2) 此外，在投递真正的延时消息时只会在本地存储查询，不会去多级存储查，所以只将系统 Topic 的消息上传到多级存储是不够的，还需要支持在投递延时消息时从多级存储拉取 (1) After reviewing the source code, it was found that the system topic corresponding to the delayed message was not uploaded to the tiered storage. As a result, when the local delayed message expires, it cannot be delivered to the real topic. (2) Additionally, when delivering actual delayed messages, the system only queries the default storage and does not check the tiered storage. Therefore, it’s not enough to simply upload the system topic messages to tiered storage; there also needs to be support for fetching delayed messages from tiered storage during delivery. ### What Did You See Instead? 1) 这个 Bug 会导致即使配置了多级存储，本地存储的过期时间还会受制于延时消息的延时时间，这样就还是没有办法减少本地存储的过期时间 (1) This bug causes the default storage expiration time to still be dependent on the delay time of the delayed messages, even if tiered storage is configured. As a result, it is not possible to reduce the expiration time of the default storage. ### Additional Context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
