[
https://issues.apache.org/jira/browse/FLINK-38547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Khaled Hammouda updated FLINK-38547:
------------------------------------
Release Note:
Flink now uses protobuf-java 4.32.1 (corresponding to Protocol Buffers version
32), upgrading from
protobuf-java 3.21.7 (Protocol Buffers version 21). This major upgrade enables:
- **Protobuf Editions Support**: Full support for the new `edition = "2023"`
and `edition = "2024"`
syntax introduced in Protocol Buffers v27+. Editions provide a unified
approach that combines
proto2 and proto3 functionality with fine-grained feature control.
- **Improved Proto3 Field Presence**: Better handling of optional fields in
proto3 without the
limitations of older protobuf versions, eliminating the need to set
`protobuf.read-default-values`
to `true` for field presence checking.
- **Enhanced Performance**: Leverages performance improvements and bug fixes
from 11 Protocol
Buffers releases (versions 22-32).
- **Modern Protobuf Features**: Access to newer protobuf capabilities including
Edition 2024
features and improved runtime behavior.
Users with existing proto2 and proto3 `.proto` files will continue to work
without changes.
was:
Flink now uses protobuf-java 4.32.1 (corresponding to Protocol Buffers version
32), upgrading from
protobuf-java 3.21.7 (Protocol Buffers version 21). This major upgrade enables:
- **Protobuf Editions Support**: Full support for the new `edition = "2023"`
and `edition = "2024"`
syntax introduced in Protocol Buffers v27+. Editions provide a unified
approach that combines
proto2 and proto3 functionality with fine-grained feature control.
- **Improved Proto3 Field Presence**: Better handling of optional fields in
proto3 without the
limitations of older protobuf versions, eliminating the need to set
`protobuf.read-default-values`
to `true` for field presence checking.
- **Enhanced Performance**: Leverages performance improvements and bug fixes
from 11 Protocol
Buffers releases (versions 22-32).
- **Modern Protobuf Features**: Access to newer protobuf capabilities including
Edition 2024
features and improved runtime behavior.
Users with existing proto2 and proto3 `.proto` files will continue to work
without changes. For
those interested in adopting Protobuf Editions, see the updated
[Protobuf format
documentation](https://nightlies.apache.org/flink/flink-docs-release-2.1/docs/connectors/table/formats/protobuf/)
for examples and guidance.
> Upgrade protobuf-java from 3.x to 4.32.1 with compatibility patch for
> parquet-protobuf
> --------------------------------------------------------------------------------------
>
> Key: FLINK-38547
> URL: https://issues.apache.org/jira/browse/FLINK-38547
> Project: Flink
> Issue Type: Improvement
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
> Affects Versions: 2.1.0
> Reporter: Khaled Hammouda
> Priority: Major
> Labels: format, protobuf, pull-request-available, upgrade
>
> Flink currently uses protobuf-java 3.x, causing incompatibility issues with
> applications trying to use protobuf 4.x.
> In particular, {{flink-formats/flink-protobuf}} still uses a protobuf 3.x API
> that were removed (Descriptors.FileDescriptor.Syntax enum used for
> proto2/proto3 syntax detection).
> Also, the {{parquet-protobuf}} dependency (parquet-java 1.15.2) still uses
> the same API, and thus requires a local patch until the same protobuf version
> upgrade is completed upstream
> [https://github.com/apache/parquet-java/issues/3175|https://github.com/apache/parquet-java/issues/3175].
> I submitted a PR that implements this upgrade.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
