Great work! Overall the proposal looks good to me. But I think we are missing these topics: - how to enable this feature (new broker configuration flag?) - how to handle old consumer connecting to a system with watermarks (we should not deliver the watermark events?) - tooling to support operations (reset/inspect...)
Thanks for contributing this precious feature that is missing in Pulsar. Best regards Enrico Il Sab 2 Ott 2021, 22:41 Eron Wright <ewri...@streamnative.io.invalid> ha scritto: > Hi, > I've created PIP 102 to introduce the concept of watermarking into Pulsar, > to improve the correctness of stream processing applications that use > Pulsar topics as a source and/or sink. > > https://github.com/apache/pulsar/issues/12267 > > Some initial efforts were made during the 2021 Apache Pulsar Hackathon to > demonstrate the value proposition. Here's some relevant links: > > https://github.com/EronWright/pulsar-hackathon-2021-projects-watermarking > > > https://streamnative.io/blog/community/2021-06-22-pulsar-hackathon-2021-winners-announced/ > > I'm giving a talk at Pulsar Summit Europe on this idea, and will post more > materials as they become available: > https://pulsar-summit.org/en/event/europe-2021/sessions/pulsar-watermarking > > Looking forward to discussing this idea with the Pulsar community! > Eron Wright > StreamNative >
