Hi Sachin
What exactly does the MyReducer do? Can you provide us with some code?
Just a wild guess from my side, did you check the watermarking? If the
Watermarks aren't progressing there's no way for Flink to know when to emit a
window and therefore you won't see any outgoing events.
Kind Regards
Dominik
From: Sachin Mittal <sjmit...@gmail.com>
Date: Monday, 8 April 2024 at 08:17
To: user@flink.apache.org <user@flink.apache.org>
Subject: How to debug window step in flink
Be aware: This is an external email.
Hi,
I have a following windowing step in my pipeline:
inputData
.keyBy(new MyKeySelector())
.window(
TumblingEventTimeWindows.of(Time.seconds(60)))
.reduce(new MyReducer())
.name("MyReducer");
Same step when I see in Flink UI shows as:
Window(TumblingEventTimeWindows(60000), EventTimeTrigger, MyReducer,
PassThroughWindowFunction) :- Sink: MyData sink +- Filter
So far I don't see any errors in the pipeline.
However when I check the following metrics:
0.MyReducer.numRecordsInPerSecond = 600 / s
0.MyReducer.numRecordsOutPerSecond = 0 / s
It shows that the step is receiving data but it is not outputting anything.
Looks like some problem in the step.
Most obvious thing that I can deduce is that the window step may not be
outputting any record.
Also MyReducer is a pretty simple function and it does not catch any exception,
so if there was a problem in this function it would have been thrown to the
pipeline
and I would have known.
Please let me know how I can debug this better.
Thanks
Sachin
