Re: [Question] Testing interaction of streaming main input and slowly updating side input

2021-09-13 Thread Johannes Frey
Hi Siyu, Thank you for your response. I‘m aware of that. The thing is… even if it runs forever the trigger should still fire every period if I’m not mistaken. So I still do not understand why the main-stream blocks the processing after the filter step and the enrich step gets never executed. I w

Re: [Question] Testing interaction of streaming main input and slowly updating side input

2021-09-13 Thread Siyu Lin
Hi Johannes, L20 in your code will make it run forever so it is better to put a finite number there, like `GenerateSequence.from(0).to(1).withRate(1, Duration.standardSeconds(2))` -siyu > On Sep 10, 2021, at 6:06 AM, Johannes Frey wrote: > > Hi everybody, > > I'm currently having a hard tim

[Question] Testing interaction of streaming main input and slowly updating side input

2021-09-10 Thread Johannes Frey
Hi everybody, I'm currently having a hard time wrapping my head around streaming data processing. Szenario: I have a main stream of data that is going to be processed (orders entering the system) and at some point in the pipeline in one processing step I need a side input to enrich the processed