danny0405 commented on a change in pull request #4421:
URL: https://github.com/apache/hudi/pull/4421#discussion_r773698637
##########
File path:
hudi-flink/src/main/java/org/apache/hudi/util/AvroSchemaConverter.java
##########
@@ -244,10 +244,13 @@ public static Schema convertToSchema(LogicalType
logicalType, String rowName) {
return nullable ? nullableSchema(time) : time;
case DECIMAL:
DecimalType decimalType = (DecimalType) logicalType;
- // store BigDecimal as byte[]
+ // store BigDecimal as Fixed
+ // for spark compatibility.
Schema decimal =
LogicalTypes.decimal(decimalType.getPrecision(),
decimalType.getScale())
- .addToSchema(SchemaBuilder.builder().bytesType());
+ .addToSchema(SchemaBuilder
+ .fixed(String.format("%s.fixed", rowName))
+
.size(computeMinBytesForDecimlPrecision(decimalType.getPrecision())));
Review comment:
Some of the community users feedback that the hive 2 and 3 has different
physical type for decimal, the low version hive seems to have `bytes` as
parquet physical type and the high version hive has `fixed_bytes` as the
physical type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
