tustvold commented on issue #15162: URL: https://github.com/apache/datafusion/issues/15162#issuecomment-2714900060
> Comet using the DataFusion physical plan expressions directly, there is no coercion phase and schema from Apache Spark for the same comes as IMO this is the issue, the spark code is not returning data with the same schema as is then used to construct the RecordBatch. Where does the schema provided to the RecordBatch constructor come from? > Adding a config to support relax name checking is a good solution to me This sounds great in theory, although I've yet to see a coherent approach to how this would work. Ultimately the field name is part of the schema in much the same way as StringViewArray and StringArray are different types, one can layer a logical schema type on top, but at some point something actually needs to coerce the types. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
