[GitHub] [doris] Accelarator opened a new issue, #14101: [Bug] duplicate表做简单的group by时耗时1min51s

[GitBox]

Accelarator opened a new issue, #14101:
URL: https://github.com/apache/doris/issues/14101

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Version
   
   1.1.3
   
   ### What's Wrong?
   
   **基础背景：**
   集群1FE3BE，duplicate表，数据量1天8亿左右，47个字段，60G存储。集群规模华为云服务器32core256G SSD 270M/s
   **问题：**
   group by耗时1min51s。后台查看SQL执行过程中doris be节点的执行进程磁盘IO没有波动
   **建表语句：**
   ``` SQL
   CREATE TABLE IF NOT EXISTS online_duration_data_detail(                 
      `totaldate` datetime COMMENT '服务端时间 格式’yyyy-mm-dd hh:mi:ss’',    
      `appid` STRING COMMENT '应用ID',  
      `uid` bigint COMMENT '用户ID',              
      `client_timestamp` datetime COMMENT '客户端时间',
      ... 其余43个字段
   )
   ENGINE=olap 
   DUPLICATE KEY(totaldate) 
   COMMENT "latest" 
   PARTITION BY RANGE(totaldate)() 
   DISTRIBUTED BY HASH(client_timestamp) BUCKETS 32 
   PROPERTIES (     
       'replication_num' = '1',
       'dynamic_partition.enable' = 'true',
       'dynamic_partition.time_unit' = 'day',
       'dynamic_partition.end' = '1',
       'dynamic_partition.prefix' = 'p',
       'dynamic_partition.buckets' = '32',
       'dynamic_partition.create_history_partition' = 'true',
       'dynamic_partition.history_partition_num' = '5'
   );
   ```
   **查询SQL：**
   ``` SQL
   select 
       appid
       ,uid 
   from 
       app_online_duration_detail
   where 
       totaldate between '2022-11-02 00:00:00' and '2022-11-03 00:00:00'
   group by 
       appid
       ,uid
   limit 10
   ````
   **第一层profile：**
   
![image](https://user-images.githubusercontent.com/18416349/200720517-467f20a9-37a9-40a9-94b5-c4471591f562.png)
   
   **第二层profile：**
   
![image](https://user-images.githubusercontent.com/18416349/200720619-5ee72cca-4b26-4820-9f08-44f274139c2e.png)
   
   **第三层profile：**
   
![image](https://user-images.githubusercontent.com/18416349/200720665-c5e5b232-ee0d-4f09-bb0f-f236b2b0946b.png)
   
   
   
   
   ### What You Expected?
   
   耗时基本在预聚合之后的VDataStreamSender上，为啥会出现这种情况？是集群问题还是部署问题？如何优化？
   
   ### How to Reproduce?
   
   如问题描述所述
   
   ### Anything Else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org