Re: Process finite stream and notify upon completion

2021-07-14 Thread Tamir Sagi
d4fc7595] From: Piotr Nowojski Sent: Wednesday, July 14, 2021 4:32 PM To: Timo Walther Cc: user Subject: Re: Process finite stream and notify upon completion EXTERNAL EMAIL Hi Tamir, Ok, let's take a step back. First of all let's assume we have a bounded source already. If so

Re: Process finite stream and notify upon completion

2021-07-14 Thread Piotr Nowojski
che/flink/streaming/api/watermark/Watermark.html#MAX_WATERMARK > > < > https://ci.apache.org/projects/flink/flink-docs-release-1.13/api/java/org/apache/flink/streaming/api/watermark/Watermark.html#MAX_WATERMARK > > > > > > > > -

Re: Process finite stream and notify upon completion

2021-07-14 Thread Timo Walther
--- *From:* Piotr Nowojski *Sent:* Wednesday, July 14, 2021 1:36 PM *To:* Tamir Sagi *Cc:* user@flink.apache.org *Subject:* Re: Process finite stream and notify upon completion *EXTERNAL EMAIL* Hi Tamir, Sorry I missed that you want to use Kafka. In that case I would suggest trying

Re: Process finite stream and notify upon completion

2021-07-14 Thread Tamir Sagi
gi Cc: user@flink.apache.org Subject: Re: Process finite stream and notify upon completion EXTERNAL EMAIL Hi Tamir, Sorry I missed that you want to use Kafka. In that case I would suggest trying out the new KafkaSource [1] interface and it's built-in boundness support [2][3]. Maybe it wi

Re: Process finite stream and notify upon completion

2021-07-14 Thread Piotr Nowojski
--- > *From:* Piotr Nowojski > *Sent:* Tuesday, July 13, 2021 4:54 PM > *To:* Tamir Sagi > *Cc:* user@flink.apache.org > *Subject:* Re: Process finite stream and notify upon completion > > > *EXTERNAL EMAIL* > > > Hi, > > Sources when finishing a

Re: Process finite stream and notify upon completion

2021-07-14 Thread Tamir Sagi
2021 4:54 PM To: Tamir Sagi Cc: user@flink.apache.org Subject: Re: Process finite stream and notify upon completion EXTERNAL EMAIL Hi, Sources when finishing are emitting {{org.apache.flink.streaming.api.watermark.Watermark#MAX_WATERMARK}}, so I think the best approach is to register

Re: Process finite stream and notify upon completion

2021-07-13 Thread Piotr Nowojski
Hi, Sources when finishing are emitting {{org.apache.flink.streaming.api.watermark.Watermark#MAX_WATERMARK}}, so I think the best approach is to register an even time timer for {{Watermark#MAX_WATERMARK}} or maybe {{Watermark#MAX_WATERMARK - 1}}. If your function registers such a timer, it would b