[
https://issues.apache.org/jira/browse/FLINK-36296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hong Liang Teoh updated FLINK-36296:
------------------------------------
Fix Version/s: aws-connector-5.1.0
> Add incremental shard discovery for DDB Streams source
> ------------------------------------------------------
>
> Key: FLINK-36296
> URL: https://issues.apache.org/jira/browse/FLINK-36296
> Project: Flink
> Issue Type: Bug
> Components: Connectors / DynamoDB
> Reporter: Abhi Gupta
> Assignee: Abhi Gupta
> Priority: Major
> Labels: pull-request-available
> Fix For: aws-connector-5.1.0
>
>
> DDB Streams source does a full shard discovery every minute, on large
> streams, this discovery can take more than 20 minutes to discover all shards,
> we can optimize this to do an incremental shard discovery periodically while
> also doing periodic shard discovery. This makes having the children shards
> faster and reduces the millisBehindLatest
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
