Akira, We would be very interested in any feedback on the new monitor implementation, as we have not yet been able to test it on a large production cluster.
Particularly: 1. Does it work correctly at scale? 2. Does it appear to place less load on the NN (less locking, and it should do less checking so less CPU used overall)? 3. Does it appear to be faster? For speed, I would not expect massive gains, as the actual replication is still done the same way. However due to how it randomises the blocks scheduled for replication, it should do a better job of spreading the load across the DNs. Therefore if many nodes are being decommissioned at once, it might perform a bit better. I am happy to help with any questions or help with fixing any issues in this area. Thanks, Stephen. On Mon, Sep 28, 2020 at 11:26 AM Akira Ajisaka <aajis...@apache.org> wrote: > Hi Wei-Chiu and Stephen, > > We have upgraded to HDFS 3.3.0 and experimenting the new decommission > monitor. > I'd like to feedback to the community in the near future. > > Thanks, > Akira > > On Thu, Feb 6, 2020 at 2:45 PM Akira Ajisaka <aajis...@apache.org> wrote: > > > > Yes, I'm interested in the new decommission monitor. > > We're planning to adopt this after upgrading to HDFS 3.3.0. > > > > Thanks, > > Akira > > > > On Wed, Feb 5, 2020 at 3:57 AM Wei-Chiu Chuang <weic...@cloudera.com> > wrote: > >> > >> @Akira Ajisaka you said you'd be interested, right? Are you planning to > adopt this feature? > >> > >> On Tue, Feb 4, 2020 at 10:41 AM Stephen O'Donnell < > sodonn...@cloudera.com.invalid> wrote: > >>> > >>> Hi All, > >>> > >>> In https://issues.apache.org/jira/browse/HDFS-14854 we committed a new > >>> decommission monitor to trunk, which is disabled by default. The new > >>> implementation hopes to be an improvement over the original monitor, > but > >>> without running it on a real cluster it is hard to know for sure. > >>> > >>> I would like to ask if anyone has tried to use this new monitor in a > >>> production cluster, and if so did you find any problems or did it work > as > >>> expected etc? I would be very interested in any feedback from anyone > who > >>> has tried to use it. > >>> > >>> Thanks, > >>> > >>> Stephen. >
