Rich-T-kid commented on issue #24822: URL: https://github.com/apache/datafusion/issues/24822#issuecomment-6786922676
Separate Parquet-side follow-up from #27252 / apache/arrow-rs#12161: use page-encoding stats during physical planning and only promote columns whose selected data pages remain dictionary encoded. Mixed columns like `URL` and `Title` otherwise spend much of the scan decoding, hashing, and rebuilding dictionaries. This will not change the in-memory repro here, but should avoid feeding that expensive shape into aggregation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
