Reading from AWS Kinesis Stream Cross account

Hello there, Good morning. We are using Apache Beam (Java SDK 2.35.0) in our data pipeline to read from AWS Kinesis Stream using AWS KDA (Kinesis Data Analytics) and so far it’s working fine for few data pipelines. Now we have got a new requirement that AWS KDA (running an application impleme

Re: [KafkaIO] Use of sinkGroupId with Exactly Once Semantics

I haven't done this experimentally before, so take this with a grain of salt, but... Kafka Group Ids are essentially used to track where a logical (aka application level, not thread/machine level) producer / consumer is at. As such, I think it would be fine to use just one group id, even when writ

Re: [KafkaIO] Use of sinkGroupId with Exactly Once Semantics

The Kafka consumer offset key is (group, topic, partition) On Fri, Nov 11, 2022 at 7:58 AM John Casey via user wrote: > I haven't done this experimentally before, so take this with a grain of