Hello,
i'm wondering if Apache Beam is using the message guarantees of the
execution engines, that the pipeline is running on.
So if i use the SparkRunner the consistency guarantees are exactly-once?
Have a good day and thanks,
Hans Hartmann
I don’t have an exact answer why it’s so much slower for now (only some guesses
but it requires some profiling), though could you try to test the same Kafka
read but with “ConfluentSchemaRegistryDeserializerProvider” instead of
KafkaAvroDeserializer and AvroCoder?
More details and an example ho
