arunlakshman commented on code in PR #198:
URL:
https://github.com/apache/flink-connector-aws/pull/198#discussion_r2043091155
##########
flink-connector-aws/flink-connector-aws-kinesis-streams/src/main/java/org/apache/flink/connector/kinesis/source/split/KinesisShardSplitSerializer.java:
##########
@@ -149,21 +186,29 @@ public KinesisShardSplit deserialize(int version, byte[]
serialized) throws IOEx
String startingHashKey;
String endingHashKey;
- if (version == CURRENT_VERSION) {
+ // startingHashKey and endingHashKey were added in V1
+ if (version >= 1) {
startingHashKey = in.readUTF();
endingHashKey = in.readUTF();
} else {
startingHashKey = "-1";
endingHashKey = "0";
}
+ boolean isFinished = false;
+ // isFinished was added in V2
+ if (version >= 2) {
Review Comment:
Any version above 2 is expected to have the boolean. So didn't add the
`in.available()` check ?
##########
flink-connector-aws/flink-connector-aws-kinesis-streams/src/main/java/org/apache/flink/connector/kinesis/source/enumerator/tracker/SplitTracker.java:
##########
Review Comment:
`knownSplits` does not track finished splits. When a split is marked as
finished in `markAsFinished()`, it's removed from `knownSplits`
##########
flink-connector-aws/flink-connector-aws-kinesis-streams/src/main/java/org/apache/flink/connector/kinesis/source/enumerator/tracker/SplitTracker.java:
##########
Review Comment:
If you look at the Line 141 in the
[KinesisStreamsSourceEnumerator.java](https://github.com/apache/flink-connector-aws/pull/198/files#diff-056c7bf51143c7ea9ce5906b5a410ffc5ced69339fe06c6cbd619b358f9470de)
```
splitTracker.markAsFinished(splitsFinishedEvent.getFinishedSplitIds());
```
This call makes sure that the finished splits are removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
