mbutrovich commented on code in PR #1106:
URL: https://github.com/apache/datafusion-comet/pull/1106#discussion_r1850724380
##########
spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala:
##########
@@ -2521,9 +2524,12 @@ object QueryPlanSerde extends Logging with
ShimQueryPlanSerde with CometExprShim
new
SparkToParquetSchemaConverter(conf).convert(scan.requiredSchema)
val dataSchemaParquet =
new
SparkToParquetSchemaConverter(conf).convert(scan.relation.dataSchema)
+ val partitionSchemaParquet =
+ new
SparkToParquetSchemaConverter(conf).convert(scan.relation.partitionSchema)
Review Comment:
#1103 discusses how the schemas have already lost necessary information at
this point. Should we construct a new partition schema from the true Parquet
schema rather than the partitionSchema that may have lost/converted type
information already?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
