CTTY commented on code in PR #5943:
URL: https://github.com/apache/hudi/pull/5943#discussion_r925171715
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieStorageConfig.java:
##########
@@ -130,6 +130,12 @@ public class HoodieStorageConfig extends HoodieConfig {
.defaultValue("TIMESTAMP_MICROS")
.withDocumentation("Sets spark.sql.parquet.outputTimestampType. Parquet
timestamp type to use when Spark writes data to Parquet files.");
+ public static final ConfigProperty<String> PARQUET_FIELD_ID_WRITE_ENABLED =
ConfigProperty
+ .key("hoodie.parquet.fieldId.write.enabled")
+ .defaultValue("true")
+ .withDocumentation("Sets spark.sql.parquet.fieldId.write.enabled. "
+ + "If enabled, Spark will write out parquet native field ids that
are stored inside StructField's metadata as parquet.field.id to parquet
files.");
+
Review Comment:
[SPARK-38094](https://github.com/apache/spark/pull/35385) Spark 3.3 checks
if this field is enabled. There would be NPE if not provided. default value is
true in Spark
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
