Proposal for Implementing Keyed Watermarks in Apache Flink

2023-09-04 Thread Tawfek Yasser Tawfek
Dear Apache Flink Development Team, I hope this email finds you well. I am writing to propose an exciting new feature for Apache Flink that has the potential to significantly enhance its capabilities in handling unbounded streams of events, particularly in the context of event-time windowing.

Proposal for Implementing Keyed Watermarks in Apache Flink

2023-10-27 Thread Tawfek Yasser Tawfek
Dear Apache Flink Development Team, I hope this email finds you well. I propose an exciting new feature for Apache Flink that has the potential to significantly enhance its capabilities in handling unbounded streams of events, particularly in the context of event-time windowing. As you may be

Re: Proposal for Implementing Keyed Watermarks in Apache Flink

2023-10-28 Thread Tawfek Yasser Tawfek
ch the events can be delayed beyond what is expected in the overall system. Do I get it correctly that the keyed watermarking only targets scenarios of calculating keyed windows (which are also keyed by the same producer ids)? Best, Alexander Fedulov On Fri, 27 Oct 2023 at 19:07, Tawfek Yasser Ta

Re: Proposal for Implementing Keyed Watermarks in Apache Flink

2023-10-30 Thread Tawfek Yasser Tawfek
ect to say that keyed watermarking is applicable only in scenario 2)? Best, Alexander Fedulov On Sat, 28 Oct 2023 at 14:33, Tawfek Yasser Tawfek wrote: > Thanks, Alexander for your reply. > > Our solution initiated from this inquiry on Stack Overflow: > > https://stackover

Re: Proposal for Implementing Keyed Watermarks in Apache Flink

2023-11-18 Thread Tawfek Yasser Tawfek
Hello Alexander, Will we continue the discussion? Thanks & BR, Tawfik From: Tawfek Yasser Tawfek Sent: 30 October 2023 15:32 To: dev@flink.apache.org Subject: Re: Proposal for Implementing Keyed Watermarks in Apache Flink Hi Alexander, Thank you for

Understanding the Internals of Watermark Generation | Actual Implementation

2022-11-15 Thread Tawfek Yasser Tawfek
Dear Flink Developers, Hope all is well, I'm trying to understand how flink in runtime generates and emits watermarks, I think that the TimestampsAndWatermarksOperator class is responsible for assigning timestamps and generating watermarks actually, based on the specified watermark strategy in t