[
https://issues.apache.org/jira/browse/FLINK-18202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235531#comment-17235531
]
Suhan Mao commented on FLINK-18202:
-----------------------------------
[~twalthr] [~libenchao]
I have finished the first version of flink-pb. the code is located in
[https://github.com/maosuhan/flink-pb] for temporary use. You can go through
the code with the help of README. You can review the design and the code and
I'm very happy to hear your response.
Features:
# Codegen in deserialization. The performance is close to java native API.
# Fast serialization. The performance is 3 times faster than proto builder
API.(setter -> build -> toByte
Array())
# pb2 and pb3 is both supported. We recognize the syntax from proto file read
by table source/sink function.
# All protobuf data type is supported including simple type, message type, map
type, array type.
# Implement Dynamic Table Factory and RowData interface
# Support pb.message-class-name, pb.ignore-parse-errors and
pb.ignore-default-values connector params. Check detail in github.
Here is some of my questions, I'm new to contribute flink code:
# where to place the code in flink project. In flink-formats? Name this module
by "flink-protobuf" ?
# Should I give a more detailed design doc?
# Is the way I use codegen in deserialization part the right way in flink
project?
# Is test case enough and give me some advice.
# Is there any further actions we can move forward.
> Introduce Protobuf format
> -------------------------
>
> Key: FLINK-18202
> URL: https://issues.apache.org/jira/browse/FLINK-18202
> Project: Flink
> Issue Type: New Feature
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table
> SQL / API
> Reporter: Benchao Li
> Priority: Major
> Attachments: image-2020-06-15-17-18-03-182.png
>
>
> PB[1] is a very famous and wildly used (de)serialization framework. The ML[2]
> also has some discussions about this. It's a useful feature.
> This issue maybe needs some designs, or a FLIP.
> [1] [https://developers.google.com/protocol-buffers]
> [2] [http://apache-flink.147419.n8.nabble.com/Flink-SQL-UDF-td3725.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
