[
https://issues.apache.org/jira/browse/FLINK-39940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bowen Li updated FLINK-39940:
-----------------------------
Fix Version/s: 2.3.0
Affects Version/s: 2.2.1
> DynamicKafkaSource should refresh restored cluster properties from
> KafkaMetadataService
> ---------------------------------------------------------------------------------------
>
> Key: FLINK-39940
> URL: https://issues.apache.org/jira/browse/FLINK-39940
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kafka
> Affects Versions: 2.2.1
> Reporter: Bowen Li
> Assignee: Bowen Li
> Priority: Major
> Fix For: 2.3.0
>
>
> {{DynamicKafkaSource}} restores {{KafkaStream}} metadata from the
> checkpointed enumerator state when a job is resumed from a checkpoint or
> savepoint. Before this change, restored sub-enumerators were rebuilt from the
> checkpointed cluster properties without first refreshing them from
> {{{}KafkaMetadataService{}}}. That means non-bootstrap Kafka client
> properties such as authentication or security settings can be stale at
> restore time, and the job can fail before the next periodic metadata refresh
> has a chance to reconcile the latest metadata.
> The restore path should use the latest metadata-service properties
> immediately, while still preserving the checkpointed {{bootstrap.servers}}
> for restored clusters so the enumerator reconnects to the same restored
> cluster identity. Restore-time metadata lookup failures should also fail fast
> instead of being deferred until a later discovery cycle.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
