JV, sorry for so late reply.
Il lun 20 mag 2019, 07:54 Venkateswara Rao Jujjuri <jujj...@gmail.com> ha scritto: > > we should take care of designing a better API for the placement policy > +1 Our placement policy is super complex and confusing. > > > We could also take into account the ability of adding labels/tags to > bookies. > > Can you add more context and color to your statement? > In the k8s world we do have a way to add tags. Can you please elaborate > what you are thinking? > Most of my products have strong requirements about multi tenancy and smart resource allocation. I have always very small clusters (3-5 bookies) so rack/region awareness is not my primary focus (I have very few cases of multi region clusters). Currently if you want to drive the allocation of resources you have to write your own placement policy, put some tenant binding info on 'custom metadata' and use some out-of-band (in respect to BK) information about bookies and which bookie can be used for the tenant who is requesting the ledger placement. Having some sort of tags directly set on bookies zk directory will drop the need for such out of bound information distribution. Some year ago we discussed about this topic but the discussion did not go anywhere. IIRC the reason was that generic labels may be too generic and make the placement rules too complex. I see value on this fault domain id. I think we should have some information on zookeeper about which fault domains are defined. We can't discover new fault domains just by listing bookies metadata. I will also send soon a request for enhancements and a PR to have the list of bookies (currently we only have APIs to discovery bookies that are up and running). One of my colleagues is working on management tools and we found the lack of this information. The script approach for dns to switch mapping also makes it impossible to understand the topology of the cluster without testing every bookie address. Enrico > > On Sun, May 19, 2019 at 10:23 PM Enrico Olivelli <eolive...@gmail.com> > wrote: > > > Il lun 20 mag 2019, 05:03 Sijie Guo <guosi...@gmail.com> ha scritto: > > > > > +1 from me. `Cookie` was designed for keeping the informations that is > > > associated with a bookie (e.g. disk layouts, bookie id and etc). > > > > > > I think it is making sense to have `FaultZoneId` stored as part of the > > > cookie. > > > > > > > I agree. > > > > But we should take care of designing a better API for the placement > policy. > > We are changing signatures quite often, adding parameters, changing > return > > type.... > > > > We could also take into account the ability of adding labels/tags to > > bookies. > > > > Enrico > > > > > > > > > > > - Sijie > > > > > > On Sun, May 19, 2019 at 9:48 AM Venkateswara Rao Jujjuri < > > > jujj...@gmail.com> > > > wrote: > > > > > > > In the current code, bookie to faultDomain mapping is supplied > through > > > > different methods. Salesforce uses a script to read yaml file which > > > > contains racks/machines mapping. But I am wondering why can't we put > > this > > > > info in the Cookie? Assuming that these machines can never move > across > > > > fault zones. > > > > > > > > Currently cookies contain version, Host, JourlanDirs, ledgerDirs, > and > > > > instanceId. > > > > If we add faultzoneId to it, it will be always available for everyone > > to > > > > look into. > > > > Is there any reason why it would be a bad idea? > > > > > > > > Thanks, > > > > -- > > > > Jvrao > > > > --- > > > > First they ignore you, then they laugh at you, then they fight you, > > then > > > > you win. - Mahatma Gandhi > > > > > > > > > > > > -- > Jvrao > --- > First they ignore you, then they laugh at you, then they fight you, then > you win. - Mahatma Gandhi >
