Rich-T-kid commented on PR #25185: URL: https://github.com/apache/datafusion/pull/25185#issuecomment-5635962382
there is no perf difference entirely due to how the benchmarks are set up. in the benchmarks the values array are created fresh for each `intern()` call. in practice this isnt what happens, dictionary array values typically map to one parquet row-group which can be very large. if maintainers are interested in seeing what the performance gains this would mean for parquet files we may need to wait for https://github.com/apache/datafusion/pull/24227 to be over the finish line so we can run @adriangbot with the flag enabled. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
