yihua commented on code in PR #5379:
URL: https://github.com/apache/hudi/pull/5379#discussion_r854863511
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieBaseRelation.scala:
##########
@@ -336,7 +335,11 @@ abstract class HoodieBaseRelation(val sqlContext:
SQLContext,
}
protected final def appendMandatoryColumns(requestedColumns: Array[String]):
Array[String] = {
- val missing = mandatoryColumns.filter(col =>
!requestedColumns.contains(col))
+ // For a nested field in mandatory columns, we should first get the
root-level field, and then
+ // check for any missing column, as the requestedColumns should only
contain root-level fields
+ // We should only append root-level field as well
+ val missing = mandatoryColumns.map(col =>
HoodieAvroUtils.getRootLevelFieldName(col))
Review Comment:
If I understand it correctly, do you mean to say we can do the filtering of
`mandatoryColumns` upon initialization of the class? That's not possible since
we need to do on-the-fly filtering based on the passed-in `requestedColumns`
which may vary when `buildScan` is called.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
