Re: Samza Runner for Beam Processing Time support

2021-02-16 Thread Xinyu Liu
I see. This is an interesting use case, and seems quite common. I think the issue is that we are trying to use trigger in a global window of an unbounded stream. This somehow leads Beam to use AfterSynchronizedProcessingTime trigger. I am wondering is it OK for you to change the global window to a

Re: Samza Runner for Beam Processing Time support

2021-02-12 Thread Jan Bensien
Hey, Thank you for your help. With your help I was able to identify the problem within my code. The problem is that my trigger followed by an Latest.perKey transformation results in a AfterSynchronizedProcessingTime trigger. The minimal example of this problem is the following: https://gist.

Re: Samza Runner for Beam Processing Time support

2021-02-09 Thread Xinyu Liu
Jan, I tried the latest Beam 2.27 version and ran into the same issue as you saw. I dug a bit deeper and it was caused by the recent changes in beam to enable SplitableParDo in all runners. While we are going to work with Beam to get this resolved, you can avoid the issue by adding this argument "

Re: Samza Runner for Beam Processing Time support

2021-02-08 Thread Xinyu Liu
Hi, Jan, Thanks for reporting this issue to us. Processing time triggers are supported in Samza Runner with version Beam 2.22.0 [1]. The exception message wasn't updated after we added the support of processing time. Apologize for the confusion here. Looks most of the exception messages have been