alamb commented on issue #15582: URL: https://github.com/apache/datafusion/issues/15582#issuecomment-2781382025
> I would be happy to share / upstream any work I do on this if there is interest. Thanks @matthewmturner -- what I think would be really valuable is if you could prove/disprove the theory that caching parsed metadata improves local TPCH performance I think it is already pretty well understood that avoiding object store requests to remote object stores helps, and many downstream systems already do this (influx, pydantic, etc) . However, we haven't built in caching for metadata locally because I think we assumed the overhead wasn't very big compared to the complexity of a built in metadata cache -- maybe we should re-evalute that assesment -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
