Hongshun Wang created FLINK-36609:
-------------------------------------
Summary: Protobuf Format not support proto3
Key: FLINK-36609
URL: https://issues.apache.org/jira/browse/FLINK-36609
Project: Flink
Issue Type: Improvement
Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
Affects Versions: 1.20.0
Reporter: Hongshun Wang
Fix For: 1.20.1
Attachments: image-2024-10-28-14-11-17-931.png
Currently, though ProtoToRowConverter try to handle PROTO3 as follows:
{code:java}
public ProtoToRowConverter(RowType rowType, PbFormatConfig formatConfig)
throws PbCodegenException {
try {
Descriptors.Descriptor descriptor =
PbFormatUtils.getDescriptor(formatConfig.getMessageClassName());
Class<?> messageClass =
Class.forName(
formatConfig.getMessageClassName(),
true,
Thread.currentThread().getContextClassLoader());
String fullMessageClassName = PbFormatUtils.getFullJavaName(descriptor);
boolean readDefaultValuesForPrimitiveTypes =
formatConfig.isReadDefaultValues();
if (descriptor.getFile().getSyntax() == Syntax.PROTO3) {
// pb3 always read default values for primitive types
readDefaultValuesForPrimitiveTypes = true;
}
//...
} {code}
However, protoc will compile message of PROTO3 as
FileDescriptor rather than Descriptor, thus the following code will throw
exception:
!https://alidocs.dingtalk.com/core/api/resources/img/5eecdaf48460cde59040feb26895f4d9b9874851c547dab758e70b814913bc360a414d3de9277d871abf3af1cbd75249a9fcba3598ff5ae2936e5b530f5e23a6779e2fa1453b2c8d24c0547dfa0c758ff70226a219283534fc653b69905bac42?tmpCode=ada2b509-fde1-48f9-81f7-3a57fefa6367!
{code:java}
public static Descriptors.Descriptor getDescriptor(String className) {
try {
Class<?> pbClass =
Class.forName(className, true,
Thread.currentThread().getContextClassLoader());
return (Descriptors.Descriptor)
pbClass.getMethod(PbConstant.PB_METHOD_GET_DESCRIPTOR).invoke(null);
} catch (Exception e) {
throw new IllegalArgumentException(
String.format("get %s descriptors error!", className), e);
}
} {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
