Yes I am trying to track how long it takes for a single element to be ingested into the pipeline until it is output somewhere.
My pipeline is unbounded. I am using KafkaIO. I did not think about CPU time. if there is a way to track it too, it would be useful to improve my metrics. On Thu, May 28, 2020 at 12:52 PM Luke Cwik <lc...@google.com> wrote: > What do you mean by processing time? > > Are you trying to track how long it takes for a single element to be > ingested into the pipeline until it is output somewhere? > Do you have a bounded pipeline and want to know how long all the > processing takes? > Do you care about how much CPU time is being consumed in aggregate for all > the processing that your pipeline is doing? > > > On Thu, May 28, 2020 at 11:01 AM Talat Uyarer < > tuya...@paloaltonetworks.com> wrote: > >> I am using Dataflow Runner. The pipeline read from kafkaIO and send Http. >> I could not find any metadata field on the element to set first read time. >> >> On Thu, May 28, 2020 at 10:44 AM Kyle Weaver <kcwea...@google.com> wrote: >> >>> Which runner are you using? >>> >>> On Thu, May 28, 2020 at 1:43 PM Talat Uyarer < >>> tuya...@paloaltonetworks.com> wrote: >>> >>>> Hi, >>>> >>>> I have a pipeline which has 5 steps. What is the best way to measure >>>> processing time for my pipeline? >>>> >>>> Thnaks >>>> >>>
