+ hdfs-dev. Thanks Haobo for your proposal. As you mentioned above, this may be related to RBF and ARR features, right? IMO, it is necessary to improve responder performance, but I am a little confused about nameservice will slow down the whole system, The first glance is client or package size should be more affected. Anyway, improving responder performance will be one good point, especially for the Router of HDFS. Thanks again.
Best Regards, - He Xiaoqiao On Wed, Apr 23, 2025 at 6:52 PM Steve Loughran <ste...@cloudera.com.invalid> wrote: > > think you meant to send this to hdfs-dev, not the -subscribe list. > > On Wed, 16 Apr 2025 at 14:09, Zhanghaobo <hfutzhan...@163.com> wrote: > > > Hello, everyone, so sorry to bother you and I would like to discuss how > > to optimize the invoke retry logic of async repsonder when some > > nameservices are slow. > > Currently, all nameservices share one async responder thread pool which > > is in class AsyncRpcProtocolPBUtil. > > Think below situation, we have two nameservices: ns1 and ns2, and the > > performance of ns2 is very pool. If we hit the invoke retry logic in > > handlerInvokeException, we will use the thread in async responder thread > > pool to perform invoke retry. As mentioned before, and the performance of > > ns2 is very pool. This will cause all threads in the async responder thread > > pool being occupied by the slow ns2 cluster in extreme situation. > > This is a critical problem and my idea is to separate async responder > > thread pool by nameservice as well like async handler thread pool. > > I would appreciate to hear your thoughts. > > > > > > Best Regards, > > - Zhang Haobo > > --------------------------------------------------------------------- To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
