[GitHub] [incubator-doris] zbtzbtzbt opened a new pull request #8633: [Opt] Optimize hotspot functions in compaction

[GitBox]

zbtzbtzbt opened a new pull request #8633:
URL: https://github.com/apache/incubator-doris/pull/8633


   ## Proposed changes
   **Compaction** is very cpu consuming, 
   after test, I found partial bottleneck is in function 
`rowset_with_max_version` 
   so made following targeted performance tuning:
   
   - better time cost O(1) function`max_version()` rather than previous heavy 
and hotspot O(n)
   - use faster `phmap::flat_hash_map` replace `rs_version_map` and 
`_stale_rs_version_map` data structure
   
   currently, these changes seems do not affect the speed of data load.
   
   1FE 3BE load 5GB data for agg table:
   
   Previous:
   test1: start | end  ==> 14:38:22|14:40:16 ==> 114 s
   test2: start | end   ==> 14:47:46|14:49:38 ==> 112 s
   
   Enhancement:
   test1: start | end  ==> 15:33:02|15:34:58 ==> 116 s
   test2: start | end ==> 16:54:20|16:56:22 ==> 122s
   
   
   ### Previous: 
   `Tablet::can_do_compaction` 40%, like Health Kit in China, red means 
unhealth.
   
   <img width="1331" alt="previous" 
src="https://user-images.githubusercontent.com/35688959/159883132-ce6f2409-b9b3-4f68-b486-8996f5c54106.png";>
   
   ### Enhancement:
   `Tablet::can_do_compaction` 20%, like Health Kit in China, green means 
health.
   
   <img width="1347" alt="max_version_V2" 
src="https://user-images.githubusercontent.com/35688959/159883088-19b9c97c-3ecd-49f5-9524-a662b40c0f29.png";>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org