Il giorno lun 30 lug 2018 alle ore 16:55 Venkateswara Rao Jujjuri < jujj...@gmail.com> ha scritto:
> We have introduced customMetadata in the past and that was not mandatory. > How did we make that work with text format? > Also, my preference is to wait until we have a need to introduce a new > field for this conversion if needed. > JV, So you are saying that you prefer to defer this change to the first case, when it will be really needed ? As far as I understand we are working on this change in order to be ready with a full "rollback" story, so we have to be one version ahead. Enrico > > JV > > On Mon, Jul 30, 2018 at 12:42 AM, Enrico Olivelli <eolive...@gmail.com> > wrote: > > > Il giorno lun 30 lug 2018 alle ore 09:17 Ivan Kelly <iv...@apache.org> > ha > > scritto: > > > > > >> Thank you for putting this together. It is also good to put this as > a > > > BP, > > > >> since it is about the metadata layout. > > > > > > I'll put a BP up this week once I have initial feedback. > > > > > > >> > - When writing a metadata, check what is in /ledgers/LAYOUT. If it > > is > > > >> > as above, write using the current text protobuf. If is bumped, use > > the > > > >> > new binary format. > > > > > > > > isn't it too costly? Adding a zk read for each write. We could add a > > > watch > > > > but is has an important cost > > > > > > This cost can be amortized. In simplest case, we only read when client > > > starts (as already happens). Client are able both read and write the > > > old format, so clients that read the pre-upgrade version will still > > > write in text format, and this is fine. > > > > > > > For me it is fine to read it when client starts. A rolling restart of the > > client application will be enough to load the new configuration. > > > > > > > > > > However, we would want them to eventually see the new format without a > > > restart. At one extreme, we could put a watch on the znode, but this > > > could be quite heavy (I think the only per-client watch we have now is > > > on /ledgers/available). Alternatively, we could just read > > > /ledgers/LAYOUT once per day. This way clients who saw the old version > > > will eventually get updated. > > > > > > > I prefer NOT to use the watch, this cost will be payed forever even when > > there will be no configuration changes. > > > > I would go with the 'load at start' option, it is clear and works fine > > > > > > > > > > > > > What about having a client side config writeMetadataVersion ? > > > > We start a new metadata version, the new one will be encoded as > > binary. > > > > By default 4.8 clients will use previous version, as we already do > for > > > > journal and fileinfo on bookies. > > > > > > > > Each ledger is independent from the others, there is no need of a > > global > > > > flag written on zk. > > > > > > There is a global property here though,which only the administrator > > > can control. It is "Can all clients in the cluster/application read > > > the new ledger metadata version?". This is why I'd like to keep it as > > > a centralized flag rather than allowing each client to decide for > > > themselves. > > > > > > > ok, let's keep it > > > > Enrico > > > > > > > > > > > > > Don't we already have some tool? (Maybe I only have seen such tool in > > my > > > > company applications) > > > > > > It may already exist, I haven't checked. > > > > > > -Ivan > > > > > > > > > -- > Jvrao > --- > First they ignore you, then they laugh at you, then they fight you, then > you win. - Mahatma Gandhi >
