Re: Count based triggers and latency

2020-10-12 Thread Rui Wang
On Mon, Oct 12, 2020 at 9:23 PM KV 59 wrote: > Thanks for your responses. > > I have a follow-up question, when you say > >> elementCountAtLeast means that the runner can buffer as many as it wants >> and can decide to offer a low latency pipeline by triggering often or >> better throughput throu

Re: Count based triggers and latency

2020-10-12 Thread KV 59
Thanks for your responses. I have a follow-up question, when you say > elementCountAtLeast means that the runner can buffer as many as it wants > and can decide to offer a low latency pipeline by triggering often or > better throughput through the use of buffering. Does it mean, I as a pipeline

Re: Count based triggers and latency

2020-10-12 Thread Kenneth Knowles
Another thing to keep in mind - apologies if it was already clear: triggering governs aggregation (GBK / Combine). It does not have any effect on stateful DoFn. On Mon, Oct 12, 2020 at 9:24 AM Luke Cwik wrote: > The default trigger will only fire when the global window closes which > does happen

Re: Count based triggers and latency

2020-10-12 Thread Luke Cwik
The default trigger will only fire when the global window closes which does happen with sources whose watermark goes > GlobalWindow.MAX_TIMESTAMP or during pipeline drain with partial results in streaming. Bounded sources commonly have their watermark advance to the end of time when they complete a