aihuaxu commented on code in PR #3258:
URL: https://github.com/apache/parquet-java/pull/3258#discussion_r2243493263
##########
parquet-avro/src/main/java/org/apache/parquet/avro/AvroSchemaConverter.java:
##########
@@ -470,11 +470,8 @@ public Optional<Schema> visit(
@Override
public Optional<Schema> visit(
LogicalTypeAnnotation.VariantLogicalTypeAnnotation
variantLogicalType) {
- String name = parquetGroupType.getName();
- List<Schema.Field> fields = new ArrayList<>();
- fields.add(new Schema.Field("metadata",
Schema.create(Schema.Type.BYTES)));
- fields.add(new Schema.Field("value",
Schema.create(Schema.Type.BYTES)));
- return of(Schema.createRecord(name, null, namespace(name,
names), false, fields));
+ return of(
Review Comment:
@rdblue and @cashmand Here is another issue that I have noticed:
If the messageType is as follows, do we generate the schema including
everything like typed_value or we just include metadata and value? In Iceberg
code, right now we are expecting full schema, e.g., we will set the value to
`typed_value` field.
```
GroupType variantType =
Types.buildGroup(Type.Repetition.REQUIRED)
.as(LogicalTypeAnnotation.variantType((byte)1))
.id(2)
.required(PrimitiveTypeName.BINARY)
.named("metadata")
.addField(objectFields)
.named("var");
MessageType parquetSchema = parquetSchema(variantType);
GenericRecord recordA = record(fieldA, Map.of("value",
serialize(Variants.of((short) 1234))));
GenericRecord recordB = record(fieldB, Map.of("typed_value", "iceberg"));
```
```
message table {
required int32 id;
optional group var (VARIANT(1)) {
required binary metadata;
optional binary value;
optional group typed_value {
required group a {
optional binary value;
optional group typed_value (LIST) {
repeated group list {
required group element {
optional binary value;
optional binary typed_value (STRING);
}
}
}
}
}
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
