Aleksandr Iushmanov created FLINK-38603:
-------------------------------------------
Summary: FLIP-553 [Metrics] Robust OTel gRPC metric exporter
Key: FLINK-38603
URL: https://issues.apache.org/jira/browse/FLINK-38603
Project: Flink
Issue Type: New Feature
Reporter: Aleksandr Iushmanov
Jobs with large number of tasks and operator can result in significant payload
size. Open Telemetry exporter gRPC calls can be rejected due to too large
payload leading to loss of exported metric data in production environments.
Existing implementation presents the following limitations:
# While OTel exporter allows to enable gZip compression for requests, it is
not available in Flink Configuration.
# All metric data is exported in a single gRPC call, with no pagination which
impacts complex enough jobs.
# Metric attributes can be unexpectedly large, for example task_name attribute
can contain hundreds of operator names inside and be present in multiple
metrics having a significant impact on overall payload.
[Link to FLIP-553
page|https://cwiki.apache.org/confluence/display/FLINK/FLIP-553%3A+Robust+OTel+gRPC+metric+exporter]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
