alihan-synnada opened a new pull request, #13250: URL: https://github.com/apache/datafusion/pull/13250
## Which issue does this PR close? No related issue ## Rationale for this change I came across the following obstacles trying to write some custom schema inference code: 1. There are cases where we need to know if the user explicitly set the `schema_infer_max_rec` config. 2. The stream passed into `read_to_delimited_chunks_from_stream` does not necessarily have to be `BoxStream<'static>` ## What changes are included in this PR? 1. Make `schema_infer_max_rec` an `Option` 2. Add lifetime parameter to functions in CSV and compression that makes the compiler complain ## Are these changes tested? Tested with existing code and tests. ## Are there any user-facing changes? 1. `CsvOptions::schema_infer_max_rec` and `JsonOptions::schema_infer_max_rec` are now `Option<usize>`s as opposed to `usize` (`with_schema_infer_max_rec` signature is unchanged) 2. `read_to_delimited_chunks`, `read_to_delimited_chunks_from_stream`, `convert_stream`, `convert_to_compress_stream` now take lifetime parameters I believe (1) is a breaking change but I'm not sure about (2) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
