Re: [I] Exponential planning time (100s of seconds) with `UNION` and `ORDER BY` queries [datafusion]

[via GitHub]

alamb commented on issue #13748:
URL: https://github.com/apache/datafusion/issues/13748#issuecomment-2578712143

   Update: 
   I spent some time trying to avoid calling 
`OrderingEquivalenceClass::normalized_oeq_class` as much. This helped but not 
enough to really fix the problem:
   - https://github.com/apache/datafusion/pull/14049
   
   
   I made a flamegraph with https://github.com/apache/datafusion/pull/14049 and 
it looks like almost all the time now is in "remove_reundant_entries"
   
   ![Screenshot 2025-01-08 at 4 31 59 
PM](https://github.com/user-attachments/assets/b71bd2d9-c4ab-4812-9fee-23f359fbab76)
   
   (here is the raw flamegraph):
   
![flamegraph](https://github.com/user-attachments/assets/480b53c3-7b3b-474a-86d4-7467560ace0d)
   
   
   My next step will be to attempt to use some sort of Set rather than Vec for 
OrderingEquivalenceClass


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org