Hi Navneeth,
There are 3 ways we can work with data now flowing and windows and
not being fired because watermark flow stopped.
1. Write a custom trigger which fires the window if no elements arrive.
2. Your watermark assigner function can also house the logic that if no
more watermarks are flowing, explicitly generate a higher watermark and
force windows to fire.
3. The new way since Flink 1.3 I suppose is to use SourceContext
functions markAsTemporarilyIdle function. Doc -
https://ci.apache.org/projects/flink/flink-docs-release-1.3/api/java/org/apache/flink/streaming/api/functions/source/SourceFunction.SourceContext.html
As per examples, I am not really sure if there is one documented for these
scenarios.
On Tue, Jul 30, 2019, 9:24 AM Navneeth Krishnan <reachnavnee...@gmail.com>
wrote:
> Thanks Taher. Are there any examples for this? In my scenario I would have
> data coming in and it might stop for sometime but I need the window to end
> after the duration.
>
> Also, I believe in version 1.3 the event time will progress only if all
> partitions in a kafka topic pass the event time. Is that still the case? If
> there is data in only few partitions will the event time progress?
>
> Thanks
>
> On Mon, Jul 29, 2019 at 10:51 AM taher koitawala <taher...@gmail.com>
> wrote:
>
>> I believe the approach to this is wrong... For fixing windows we can
>> write our custom triggers to fire them... However what I'm not convinced
>> with is switching between event and processing time.
>> Write a custom triggers and fire the event time window if you
>> don't see any activity. That's the only way.
>>
>> On Mon, Jul 29, 2019, 11:07 PM Navneeth Krishnan <
>> reachnavnee...@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> Any suggestions?
>>>
>>> Thanks
>>>
>>> On Thu, Jul 25, 2019 at 11:45 PM Navneeth Krishnan <
>>> reachnavnee...@gmail.com> wrote:
>>>
>>>> Hi All,
>>>>
>>>> I'm working on a very short tumbling window for 1 second per key. What
>>>> I want to achieve is if the event time per key doesn't progress after a
>>>> second I want to evict the window, basically a combination of event time
>>>> and processing time. I'm currently achieving it by registering a processing
>>>> time timer but is there a way to emit some global punctuator which can be
>>>> used to evict all keys window data.
>>>>
>>>> The issue with registering processing time timer for every key is
>>>> causing too much JVM pressure. Any suggestions on how this could be
>>>> implemented?
>>>>
>>>> Thanks
>>>>
>>>
