[
https://issues.apache.org/jira/browse/FLINK-40966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-40966:
-----------------------------------
Labels: pull-request-available (was: )
> Reuse Fluss metadata clients across schema operations
> -----------------------------------------------------
>
> Key: FLINK-40966
> URL: https://issues.apache.org/jira/browse/FLINK-40966
> Project: Flink
> Issue Type: Improvement
> Components: Flink CDC
> Reporter: Xiaobing Fang
> Priority: Major
> Labels: pull-request-available
>
> h2. Description
>
> {{FlussMetaDataApplier}} creates and closes a {{Connection}} and {{Admin}}
> for every {{{}CreateTable{}}}, {{AddColumn}} and {{DropTable}} operation, and
> for every existing-table schema lookup. Closing the connection waits for the
> Netty event loop to shut down. With the default quiet period, this adds
> roughly two seconds per operation even when the metadata RPC itself is fast.
>
> Schema changes are processed serially. As a result, initializing many tables
> accumulates this shutdown cost. Existing-table expansion adds a separate
> schema lookup before {{CreateTable}} and can incur the cost twice per table,
> even when no columns need to be added.
>
> h2. Steps to reproduce
>
> # Start a pipeline with a Fluss sink and multiple tables.
> # Observe the intervals between schema operations, or capture the schema
> worker stack during initialization.
> # The worker repeatedly waits in {{FlussConnection.close}} /
> {{{}NettyClient.close{}}}.
>
> h2. Expected behavior
>
> Reuse one {{{}Connection{}}}/{{{}Admin{}}} pair per metadata applier and
> close it when the coordinator is disposed. Schema operations should not
> repeatedly shut down the network client.
>
> h2. Suggested fix
>
> Lazily initialize transient clients, reuse them for schema lookups and DDL,
> and close them together. Stop schema workers before releasing the metadata
> applier so shutdown cannot race with an in-flight operation. Preserve client
> isolation and serialization behavior.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
