Thanks for the feedback, I will address it along with feedback from the upcoming summit, and then post an updated draft. I also have details about evolving the Flink connector and Pulsar function framework to support this feature.
On Sun, Oct 3, 2021 at 4:57 AM Enrico Olivelli <eolive...@gmail.com> wrote: > Great work! > > Overall the proposal looks good to me. > But I think we are missing these topics: > - how to enable this feature (new broker configuration flag?) > - how to handle old consumer connecting to a system with watermarks (we > should not deliver the watermark events?) > - tooling to support operations (reset/inspect...) > > > Thanks for contributing this precious feature that is missing in Pulsar. > > Best regards > Enrico > > Il Sab 2 Ott 2021, 22:41 Eron Wright <ewri...@streamnative.io.invalid> ha > scritto: > > > Hi, > > I've created PIP 102 to introduce the concept of watermarking into > Pulsar, > > to improve the correctness of stream processing applications that use > > Pulsar topics as a source and/or sink. > > > > https://github.com/apache/pulsar/issues/12267 > > > > Some initial efforts were made during the 2021 Apache Pulsar Hackathon to > > demonstrate the value proposition. Here's some relevant links: > > > > > https://github.com/EronWright/pulsar-hackathon-2021-projects-watermarking > > > > > > > https://streamnative.io/blog/community/2021-06-22-pulsar-hackathon-2021-winners-announced/ > > > > I'm giving a talk at Pulsar Summit Europe on this idea, and will post > more > > materials as they become available: > > > https://pulsar-summit.org/en/event/europe-2021/sessions/pulsar-watermarking > > > > Looking forward to discussing this idea with the Pulsar community! > > Eron Wright > > StreamNative > > >
