Re: [E] Cost Based FairCallQueue latency issue

2020-11-05 Thread Fengnan Li
10:58 AM To: Fengnan Li Cc: Hdfs-dev Subject: Re: [E] Cost Based FairCallQueue latency issue We're internally running the patch I submitted on HDFS-14403 which was subsequently modified by other ppl in the community, so it's possible the community flavor may behave differently.

Re: [E] Cost Based FairCallQueue latency issue

2020-11-05 Thread Fengnan Li
notice that call taking long? Fengnan From: Daryn Sharp Date: Thursday, November 5, 2020 at 10:58 AM To: Fengnan Li Cc: Hdfs-dev Subject: Re: [E] Cost Based FairCallQueue latency issue We're internally running the patch I submitted on HDFS-14403 which was subsequently modifi

Re: [E] Cost Based FairCallQueue latency issue

2020-11-05 Thread Daryn Sharp
> > ipc.8020.decay-scheduler.period-ms > > 2 > > > > > > ipc.8020.decay-scheduler.thresholds > > 15 > > > > > > ipc.8020.faircallqueue.multiplexer.weights > > 99,1 > > > > > >

Re: [E] Cost Based FairCallQueue latency issue

2020-11-05 Thread Jim Brennan
> > > > ipc.8020.decay-scheduler.thresholds > > 15 > > > > > > ipc.8020.faircallqueue.multiplexer.weights > > 99,1 > > > > > > ipc.8020.scheduler.priority.levels > > 2 > > >

Re: [E] Cost Based FairCallQueue latency issue

2020-11-05 Thread Fengnan Li
, 2020 at 9:19 AM To: Fengnan Li Cc: Hdfs-dev Subject: Re: [E] Cost Based FairCallQueue latency issue I submitted the original 2.8 cost-based FCQ patch (thanks to community members for porting to other branches). We've been running with it since early 2019 on all clusters. Mul

Re: [E] Cost Based FairCallQueue latency issue

2020-11-05 Thread Daryn Sharp
I submitted the original 2.8 cost-based FCQ patch (thanks to community members for porting to other branches). We've been running with it since early 2019 on all clusters. Multiple clusters run at a baseline of ~30k+ ops/sec with some bursting over 100k ops/sec. If you are looking at the overall