It's highly use-case dependent, but applying a filter at the end does
sound like a good solution to me.

-Matthias

On 10/19/20 12:40 PM, Mathieu D wrote:
> Hello there,
> 
> Let's say I need to restart my streams app from a blank state (whether by
> changing app.id or using application-reset-tool).
> My app is designed on "at least once" paradigm, and outputs are upserts.
> The input topics have a few days worth of data, and the app will restart
> from there.
> 
> If I restart it, states will be empty, meaning most outputs will be wrong.
> So it should NOT upsert anything downstream for a certain period of time
> (something like my biggest time window or something)
> 
> I'm thinking of a filter at the end of the topology, removing any record
> whose timestamp < topic retention + biggest window size.
> 
> How do you guys manage this usually ?
> 
> Thanks for your insights
> 
> Mathieu
> 

Reply via email to