Artem created FLINK-34620: ----------------------------- Summary: Process recursive protobuf schemas in PbToRowTypeUtil Key: FLINK-34620 URL: https://issues.apache.org/jira/browse/FLINK-34620 Project: Flink Issue Type: Improvement Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile) Reporter: Artem
The current version of _PbToRowTypeUtil.generateRowType_ doesn't support recursive protobuf schemas like message RecursiveMessage \{ int32 id = 1; RecursiveMessage message = 2; } It throws _java.lang.StackOverflowError._ This could be implemented by trimming the recursion. Same approach is used in Apache Spark, for example. ([link|[https://github.com/apache/spark/blob/master/connector/protobuf/src/main/scala/org/apache/spark/sql/protobuf/utils/SchemaConverters.scala#L178])] then output Table schemas for the above proto-schema will be like this: recursiveFieldMaxDepth=0: message: ROW< id: INTEGER > recursiveFieldMaxDepth=1: message: ROW< id: INTEGER, message: ROW< id: INTEGER > > recursiveFieldMaxDepth=2: message: ROW< id: INTEGER, message: ROW< id: INTEGER, message: ROW< id: INTEGER > > > -- This message was sent by Atlassian Jira (v8.20.10#820010)