Re: [I] Slowdown in ClickBench Q36-Q37 between DataFusion 43.0.0 and 44.0.0 [datafusion]

[via GitHub]

alamb commented on issue #14481:
URL: https://github.com/apache/datafusion/issues/14481#issuecomment-2633735903

   I got flamegraphs from them using https://github.com/flamegraph-rs/flamegraph
   
   
   ## Q36
   ```shell
   ./datafusion-cli-44 -c "SELECT \"URL\", COUNT(*) AS PageViews FROM 
'hits.parquet' WHERE \"CounterID\" = 62 AND \"EventDate\"::INT::DATE >= 
'2013-07-01' AND \"EventDate\"::INT::DATE <= '2013-07-31' AND \"DontCountHits\" 
= 0 AND \"IsRefresh\" = 0 AND \"URL\" <> '' GROUP BY \"URL\" ORDER BY PageViews 
DESC LIMIT 10; "
   ```
   
   And made flamegraphs with
   ```
   sudo flamegraph -- ./datafusion-cli-43 -c "SELECT \"URL\", COUNT(*) AS 
PageViews FROM 'hits.parquet' WHERE \"CounterID\" = 62 AND 
\"EventDate\"::INT::DATE >= '2013-07-01' AND \"EventDate\"::INT::DATE <= 
'2013-07-31' AND \"DontCountHits\" = 0 AND \"IsRefresh\" = 0 AND \"URL\" <> '' 
GROUP BY \"URL\" ORDER BY PageViews DESC LIMIT 10;
   ```
   
   Here is DataFusion 43: 
   
   
![Image](https://github.com/user-attachments/assets/a9ed1c18-bca3-4f6c-907e-c5630cbb3575)
   
   Here is DataFusion 44:
   
   
![Image](https://github.com/user-attachments/assets/67693d6a-f50e-492e-ba67-0b4b2da524bd)
   
   A largre amount of the time is spent decoding ParquetMetadata
   
![Image](https://github.com/user-attachments/assets/cf3457b2-b152-4d85-ab9b-97aeffb6ecec)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org