Re: Message reprocessing logic

2019-07-09 Thread Alessandro Tagliapietra
Hi John, thanks a lot for the great explanation and the links. After I've sent the question I've researched a bit more about EOS and I'm currently testing that out. I'll read those links and see what I come up with! Thanks and have a great day! -- Alessandro Tagliapietra On Tue, Jul 9, 2019 at

Re: Message reprocessing logic

2019-07-09 Thread John Roesler
Hi Alessandro, Sorry if I'm missing some of the context, but could you just keep retrying the API call inside a loop? This would block any other processing by the same thread, but it would allow Streams to stay up in the face of transient failures. Otherwise, I'm afraid that throwing an exception

Message reprocessing logic

2019-07-05 Thread Alessandro Tagliapietra
Hello everyone, I'm looking into a way to reprocess messages in case of soft-errors (not exceptions) For example we have a topology that does this: input stream -> filtering/flatmap -> window and aggregate in our aggregate step (maybe should be moved into an additional step) we make an API ca