RE: Stream enrichment with ingest mode

Spark Streaming… :-) Best regards, Arnaud De : Lars Skjærven Envoyé : mercredi 14 février 2024 08:12 À : user Objet : Stream enrichment with ingest mode Dear all, A reoccurring challenge we have with stream enrichment in Flink is a robust mechanism to estimate that all messages of the source(s

Stream enrichment with ingest mode

Dear all, A reoccurring challenge we have with stream enrichment in Flink is a robust mechanism to estimate that all messages of the source(s) have been consumed/processed before output is collected. A simple example is two sources of catalogue metadata: - source A delivers products, - source B d