Hi,
Wanted to follow up as I did have similar case.
So this means it is ok for Beam to use Sliding window of 1 day with 1
sec period (with using different trigger than after watermark to avoid
outputting data from every window) and there is no additional
performance penalty (duplicating input messages for storage or cpu for
resolving windows)? Interesting from both Flink and Dataflow perspective
(both Python and Java).
I ended up implementing the logic with Beam state and timers (which is
quite performant and readable) but also interested in other possibilities.
Best
Wiśniowski Piort
On 12.06.2024 21:50, Ruben Vargas wrote:
I imagined it but wasn't sure!
Thanks for the clarification!
On Wed, Jun 12, 2024 at 1:42 PM Robert Bradshaw via user
<[email protected]> wrote:
Beam implements Windowing itself (via state and timers) rather than
deferring to Flink's implementation.
On Wed, Jun 12, 2024 at 11:55 AM Ruben Vargas <[email protected]> wrote:
Hello guys
May be a silly question,
But in the Flink runner, the window implementation uses the Flink
windowing? Does that mean the runner will have performance issues like
Flink itself? see this:
https://issues.apache.org/jira/browse/FLINK-7001
I'm asking because I see the issue, it mentions different concepts
that Beam already handles at the API level. So my suspicion is that
the Beam model handles windowing a little differently from the pure
Flink app. But I'm not sure..
Regards.
