Re: Some questions regarding operator IDs

2020-10-26 Thread Robert Metzger
Hey Kevin, setting the uid is not needed for exactly-once guarantees. It is used if you want to restore the operator state manually using a savepoint. This blog blog post (there are probably a lot more explaining this) could be helpful to understand how the checkpointing ensures exactly once desp

Some questions regarding operator IDs

2020-10-20 Thread Kevin Kwon
Hi team I'm subscribing 2 topics from Kafka Consumer, joining them and publishing back to a new topic via KafkaProducer (with Exactly Once semantic) As it's highly recommended to set uid for each operator, I'm curious how this works. For example, val topicASource = env .addSource(topicAConsume