2010YOUY01 commented on issue #11472: URL: https://github.com/apache/datafusion/issues/11472#issuecomment-2236334307
> Do you think it would still be desirable to be able to turn off file-level partioning for CSVs only (and/or to vary it on a per-type basis)? > > I'm not familiar enough with Parquet to understand if there would be a functional reason to disable it for that format? Yes, using the global option is only a quick fix. I think it's a good idea to add an extra option field for individual files, and later the planner can make better decisions based on per file's `newlines_in_values` property (like for now disable parallel execution) And it can be CSV-specific, parquet seems not relevant here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
