I would recommend a proof of concept on your cluster, doing the simplest thing first, which is holding the close commands in SCM for a configurable delay, and see if you can alleviate the problem that way.
Adding a lease will be quite complex, and may not be needed if the simple delay works. You will need a heartbeat thread from client to OM, then relay that from OM to SCM, placing more front end load onto SCM. Then on SCM you will need to track all open blocks, lease expiry due to crashed clients etc. You would also need to decide what to do with a block that is kept open for many hours or days by a persistent client slowly writing to one or more files - we cannot keep the container open indefinitely. The front end load on SCM here is probably minimal, but there is work under way to cache the container locations in OM partly to speed up KeyInfo calls, but also to prevent increasing client load increasing load on SCM. On Tue, Sep 13, 2022 at 9:19 AM Kaijie Chen <c...@apache.org> wrote: > Thanks Anu and Steven for the suggestion. Granting a lease to client > sounds like a more controllable way. > > However, if I understand correctly, clients don't talk to SCM directly. > Does it mean OM has to relay the renew lease request to SCM? > Is there a better way to implement it? > > Regards > Kaijie > > ---- On Sat, 10 Sep 2022 01:10:28 +0800 anu engineer wrote --- > > Extending the same thought from Steven. If you are going to do a small > > delay, it is better to do it via a Lease. > > > > So SCM could offer a lease for 60 seconds, with a provision to reacquire > > the lease one more time. > > This does mean that a single container inside the data node technically > > could become larger than 5GB (but that is possible even today). > > > > I do think a lease or a timeout based approach (as suggested by Steven) > > might be easier than pre-allocating blocks. > > > > Thanks > > Anu > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@ozone.apache.org > For additional commands, e-mail: dev-h...@ozone.apache.org > >
