Re: Connecting to kinesis with mfa

2020-12-16 Thread Cranmer, Danny
Hey Avi, I have reproduced and found a solution. The issue is not MFA, it is the BASIC credential provider is not using the token: https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/util/AWSUtil.java#L18

Re: Flink DynamoDB stream connector losing records

2020-09-15 Thread Cranmer, Danny
Hi Jiawei, I agree that the offset management mechanism uses the same code as Kinesis Stream Consumer and in theory should not lose exactly-once semantics. As Ying is alluding to, if your application is restarted and you have snapshotting disabled in AWS there is a chance that records can be lo