alamb commented on issue #27103: URL: https://github.com/apache/datafusion/issues/27103#issuecomment-6637310979
> Move the crate to a new apache/datafusion-spark repository under the DataFusion PMC, like Comet, Ballista and datafusion-iceberg. Its CI can run real Spark across versions, and its contributors and reviewers would be people who can debug Spark. The pieces of core that only this crate uses would move with it: enable_ansi_mode, datafusion.spark.*, datafusion-cli --spark and the Spark handling in the sqllogictest runner. I am in favor of this option personally as I think it is cleaner and fits better with the DataFusion modular design Also, it would then set a clear precedent that we shouldn't add datafusion-json / datafusion-variant into the main repo (we can potentially bring them into apache governance, but they would live in separate repos) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
