Hi Julio, If I understand correctly, you want to adjust your watermarks automatically? It is true that there are no direct ways to get metric from the AssignerWithPeriodicWatermarks. Adding ProcessFunction before assignTimestampsAndWatermarks seems a solution. In the ProcessFunction, you can count the late number and send the number to the downstream assignTimestampsAndWatermarks to adjust watermarks.
Best, Hequn On Tue, Jul 31, 2018 at 1:32 AM, Julio Biason <julio.bia...@azion.com> wrote: > Hello, > > Our current watermark model is "some time behind the most recent seen > element" (very close to what the docs have in "Periodic Watermark" > https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/event_ > timestamps_watermarks.html#with-periodic-watermarks). It fits our current > processing model. > > The thing is, we want to extract information about elements appearing > behind the watermark, to give some insight when we need to update the > amount of time behind the most seen element we need. The problem is, I > can't create any metrics inside the AssignerWithPeriodicWatermarks 'cause > it has no `getRuntime()` to attach the metric. > > Is there any way we can count those (a ProcessFunction before the . > assignTimestampsAndWatermarks(), maybe)? > > -- > *Julio Biason*, Sofware Engineer > *AZION* | Deliver. Accelerate. Protect. > Office: +55 51 3083 8101 <callto:+555130838101> | Mobile: +55 51 > <callto:+5551996209291>*99907 0554* >
