I think what you describe, Chesnay, is exactly what we should do... On Fri, Sep 30, 2016 at 1:15 PM, Chesnay Schepler <ches...@apache.org> wrote:
> Hello Dan, > > the technical reason is that this metric is only collected in the > *InputProcessor classes, which aren't used for source tasks. > > I do recall that there were discussions about source watermarks, but > frankly i don't remember why we didn't add them. > > In order to add them one would only have to modify the SourceContext > classes to a) store the last emitted watermark and b) expose it through a > metric. > > Regards, > Chesnay > > > On 29.09.2016 23:03, dan bress wrote: > >> I am trying to measure the currentLowWatermark throughout my dataflow, but >> I am not seeing it for tasks with sources. For those tasks I see these >> metrics: >> >> lastCheckpointSize >> numBytesInLocal >> numBytesInRemote >> numBytesOut >> >> why am I not seeing currentLowWatermark on these tasks? >> >> Thanks! >> >> Dan >> >> >
