bxfjb opened a new issue, #8408: URL: https://github.com/apache/rocketmq/issues/8408
### Before Creating the Enhancement Request - [X] I have confirmed that this should be classified as an enhancement rather than a bug/feature. ### Summary When tiered storage enabled, high GC pressure was shown under tens of thousands pub/sub TPS, which may cause full GC occasionally. ### Motivation Uncontrollable and unpredictable full GC may cause STW and disable the service. ### Describe the Solution You'd Like 1. Use off-heap cache instead of in-heap cache as read-ahead cache. Our gc logs show that the current cache eviction strategy causes a large number of objects to remain in the old generation. From the discussion in this [link](https://stackoverflow.com/questions/72667144/caching-objects-causes-frequent-major-gc), caffiene cache is not designed for such high-load scenarios by Ben Manes himself. 2. Another possible optimization may in the process of uploading indexfile. In the current design, the size of each compressed index is about 570MB. If the object storage SDK used to upload this much data at one time, it will inevitably be copied into the heap, which will also bring great pressure to JVM. It might be a good idea to upload the file in many parts. ### Describe Alternatives You've Considered 1. Use off-heap cache as read-ahead cache. 2. Split index file compaction's result into many parts. ### Additional Context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
