It's highly use-case dependent, but applying a filter at the end does sound like a good solution to me.
-Matthias On 10/19/20 12:40 PM, Mathieu D wrote: > Hello there, > > Let's say I need to restart my streams app from a blank state (whether by > changing app.id or using application-reset-tool). > My app is designed on "at least once" paradigm, and outputs are upserts. > The input topics have a few days worth of data, and the app will restart > from there. > > If I restart it, states will be empty, meaning most outputs will be wrong. > So it should NOT upsert anything downstream for a certain period of time > (something like my biggest time window or something) > > I'm thinking of a filter at the end of the topology, removing any record > whose timestamp < topic retention + biggest window size. > > How do you guys manage this usually ? > > Thanks for your insights > > Mathieu >
