codope commented on code in PR #12006:
URL: https://github.com/apache/hudi/pull/12006#discussion_r1778489263
##########
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieArrayWritableAvroUtils.java:
##########
@@ -52,6 +52,7 @@ public static int[] getProjection(Schema from, Schema to) {
* and if the size changes the reader will fail
*/
public static UnaryOperator<ArrayWritable> projectRecord(Schema from, Schema
to) {
+ //TODO: [HUDI-8261] add casting to the projection
Review Comment:
if it's not a significant change, maybe you can do it in this PR itself?
##########
hudi-common/src/main/java/org/apache/hudi/avro/AvroSchemaUtils.java:
##########
@@ -206,6 +207,18 @@ private static boolean isProjectionOfInternal(Schema
sourceSchema,
return atomicTypeEqualityPredicate.apply(sourceSchema, targetSchema);
}
+ public static Option<Schema.Type> findNestedFieldType(Schema schema, String
fieldName) {
Review Comment:
Don't we already have a util to find nested field type? Can reuse some parts
of `HoodieAvroUtils.getNestedFieldSchemaFromRecord`. In case a new method is
needed, please also add a UT for the same.
##########
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HiveHoodieReaderContext.java:
##########
@@ -232,6 +232,12 @@ public UnaryOperator<ArrayWritable> projectRecord(Schema
from, Schema to, Map<St
return HoodieArrayWritableAvroUtils.projectRecord(from, to);
}
+ @Override
+ public Comparable castValue(Comparable value, Schema.Type newType) {
+ //TODO: [HUDI-8261] actually do casting here
Review Comment:
What's preventing from casting here?
##########
hudi-common/src/test/java/org/apache/hudi/common/table/read/TestHoodieFileGroupReaderBase.java:
##########
@@ -107,73 +104,6 @@ public void validateRecordsInFileGroup(String tablePath,
validateRecordsInFileGroup(tablePath, actualRecordList, schema, fileSlice,
false);
}
- public abstract Comparable getComparableUTF8String(String value);
-
- @Test
- public void testCompareToComparable() throws Exception {
Review Comment:
We are still comparing ordering values in file group record buffer, so why
remove this test? Or are we testing comparuson of different types somewhere
else?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
