[
https://issues.apache.org/jira/browse/FLINK-33533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18126371#comment-18126371
]
ChenZhuo commented on FLINK-33533:
----------------------------------
I would like to take this documentation task. Could a committer assign it to me
(chenzhuo)?
I propose a short performance section in the English and Chinese Protobuf
format pages, distinguishing protoc-generated message methods from the Flink
conversion code already split by FLINK-32650. Current Flink uses protobuf-java
4.32.1, but that does not rewrite user-supplied generated classes. The linked
upstream issue #10247 remains open and PR #10367 was closed without merging.
OpenJDK HotSpot 11, 17 and 21 still define DontCompileHugeMethods=true and
HugeMethodLimit=8000 bytecodes (see the OpenJDK jdk11u runtime/globals.hpp and
jdk17u/jdk21u compiler/compiler_globals.hpp).
The note would explain that large generated methods can limit JIT compilation
and suggest benchmarking -XX:-DontCompileHugeMethods via
env.java.opts.taskmanager as an optional workaround on a supporting JVM,
preserving existing JVM options. It would explicitly avoid promising C2
compilation or a throughput improvement, since other compiler limits still
apply and compilation cost may increase. No runtime or default configuration
changes; validation would use the documentation Hugo build. Does this scope
sound appropriate?
> Add documentation about performance for huge ProtoBuf definations
> -----------------------------------------------------------------
>
> Key: FLINK-33533
> URL: https://issues.apache.org/jira/browse/FLINK-33533
> Project: Flink
> Issue Type: Improvement
> Components: Documentation, Formats (JSON, Avro, Parquet, ORC,
> SequenceFile)
> Reporter: Benchao Li
> Priority: Minor
>
> This is brought up in the discussion of FLINK-32650. For huge ProtoBuf
> definations, the generated java code would hive JIT C2 optimization
> threshold. To mitigate this problem temporarily, users can add
> {{-XX:-DontCompileHugeMethods}} to JVM args manually.
> See related issues in protobuf-java project[1][2]
> [1] https://github.com/protocolbuffers/protobuf/pull/10367
> [2] https://github.com/protocolbuffers/protobuf/issues/10247
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
