spaces-X opened a new issue #8656: URL: https://github.com/apache/incubator-doris/issues/8656
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description The current version( #8561 ) of vectorized stream load is using skip list to aggregate values, which is a row-structured constant ordered data result. Considering that we just do pre-aggregation in memtable and queries will not go to memtable, this implementation of maintaining constant order is too expensive. I plan to refactor this part of the code and replace the existing skip list with other solutions based on pr #8561. ### Solution Solution 1: First, sort of incoming-data-block. Then, merge the sorted-data-block. Then, append merged-data-block to final-block. At last, do a finalize(sort + merge) of the final-block to flush. Solution 2: First, aggregate the incoming-data-block by hash table. At last, sort the whole aggregated-block. ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
