freemandealer opened a new issue, #12609: URL: https://github.com/apache/doris/issues/12609
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description -238 (OLAP_ERR_TOO_MANY_SEGMENTS) is a common issue when loading large data. Things get worse when data has low compression ratio or memtables flushed due to exceeding memory limit. Because too mush small segments are created. see: https://github.com/apache/doris/issues?q=-238 ### Solution Rowsetwise compaction happens when load (transaction) is done. Maybe we should do compaction DURING load -- segmentwise compaction. ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
