Yukang-Lian opened a new issue, #41753: URL: https://github.com/apache/doris/issues/41753
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description We propose several improvements to the compaction process, focusing on better profiling, logging, and error reporting. Below are the specific tasks: 1. **Add compaction profiling**: - Profile the I/O and CPU time during compaction. - Reflect the data volume read from and written to cache/S3 during compaction. 2. **Enhance compaction status with cumulative error codes**: - Add cumulative status error codes to track errors that occur during compaction more effectively. 3. **Add detailed parameters to compaction status**: - Provide detailed information about recent compaction scheduling, including specific data about the last few compactions. 4. **Add logging at the compaction scheduling phase**: - Improve logging at the compaction scheduling stage to make it easier to troubleshoot issues. 5. **Track progress for large compactions**: - Add progress tracking for large-scale compactions to monitor their status. 6. **Optimize compaction logs**: - Reduce unnecessary warning logs in both cloud and community environments to avoid log clutter. ### Solution _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
