Re: RPC logging in ipc Server

2021-04-12 Thread Steve Loughran
HADOOP-17450 added an API for IOStatistics -anything can be an IOStatistics source, which can be collected, aggregated, reported (There's a serializable form). This goes beyond simple counts -we include mins, maxes and means too S3A and ABFS support this in their streams, s3a does it for its remot

Re: RPC logging in ipc Server

2021-04-12 Thread Stephen O'Donnell
I have not tried to do this, but as we (Cloudera) deal with more and more performance related problems, I feel something like this is needed. It is a tricky problem due to the number of requests the NN handles and how performance sensitive it is. At the IPC Server level, we should be able to know

RPC logging in ipc Server

2021-03-17 Thread Fengnan Li
Hi community, Has someone ever tried to implement sampling logging for ipc Server? We would like to gain more observability for all of the traffics to the Namenode. Thanks, Fengnan