Hi Talat, Could you elaborate what do you mean by “opening and closing bundle”?
Sometimes, starting a KafkaReader can take time since it will seek for a start offset for each assigned partition but it happens only once at pipeline start-up and mostly depends on network conditions. > On 9 Jun 2020, at 23:05, Talat Uyarer <[email protected]> wrote: > > Hi, > I added some metrics on a step right after KafkaIO. When I compare the read > time difference between producer and KafkaIO it is 800ms for P99. However > somehow that step's opening and closing bundle difference is 18 seconds for > p99. The step itself does not do any specific thing. Do you have any idea why > bundle latency is very high ? Where should I check or tune on KafkaIO ? > > Additional information I read from one topic. That topic has 15 partitions. > Producer write in a round robin fashion. > > Thanks
