[
https://issues.apache.org/jira/browse/FLINK-36270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hong Liang Teoh updated FLINK-36270:
------------------------------------
Fix Version/s: aws-connector-4.4.0
> DDB Streams Connector performance issue due to splitsAvailableForAssignment
> function
> ------------------------------------------------------------------------------------
>
> Key: FLINK-36270
> URL: https://issues.apache.org/jira/browse/FLINK-36270
> Project: Flink
> Issue Type: Bug
> Components: Connectors / DynamoDB
> Reporter: Abhi Gupta
> Assignee: Abhi Gupta
> Priority: Major
> Labels: pull-request-available
> Fix For: aws-connector-4.4.0
>
>
> In DDB Streams connector, while testing we found out that when we are
> spending a lot of time in markAsFinished function because we are calling
> splitsAvailableForAssignment which is O(N), and given n shards can be marked
> as finished concurrently, the algorithm becomes O(n^2). Change the algo to
> assign only child shards when a parent is finished. We can start tracking
> child shards of a shard in SplitTracker
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
