Re: Processing single events for minimum latency

2020-10-14 Thread Piotr Nowojski
No problem :) Piotrek czw., 15 paź 2020 o 08:18 Pankaj Chand napisał(a): > Thank you for the quick and informative reply, Piotrek! > > On Thu, Oct 15, 2020 at 2:09 AM Piotr Nowojski > wrote: > >> Hi Pankay, >> >> Yes, you can trigger a window per each element, take a look at the Window >> Trig

Re: Processing single events for minimum latency

2020-10-14 Thread Pankaj Chand
Thank you for the quick and informative reply, Piotrek! On Thu, Oct 15, 2020 at 2:09 AM Piotr Nowojski wrote: > Hi Pankay, > > Yes, you can trigger a window per each element, take a look at the Window > Triggers [1]. > > Flink is always processing all records immediately. The only things that >

Re: Processing single events for minimum latency

2020-10-14 Thread Piotr Nowojski
Hi Pankay, Yes, you can trigger a window per each element, take a look at the Window Triggers [1]. Flink is always processing all records immediately. The only things that can delay processing elements are: - buffering elements on the operator's state (vide WindowOperator) - buffer-timeout (but t

Re: Processing single events for minimum latency

2020-10-14 Thread Pankaj Chand
Hi Piotrek, Thank you for replying! I want to process each record as soon as it is ingested (or reaches an operator) without waiting for a window for records to arrive. However, by not using windows, I am not sure if each record gets emitted immediately upon processing. > You still can use window

Re: Processing single events for minimum latency

2020-10-14 Thread Piotr Nowojski
Hi Pankaj, I'm not entirely sure if I understand your question. If you want to minimize latency, you should avoid using windows or any other operators, that are buffering data for long periods of time. You still can use windowing, but you might want to emit updated value of the window per every p

Processing single events for minimum latency

2020-10-14 Thread Pankaj Chand
Hi all, What is the recommended way to make a Flink job that processes each event individually as soon as it comes and without waiting for a window, in order to minimize latency in the entire DAG of operators? For example, here is some sample WordCount code (without windws), followed by some know