> As far as I understand we are working on this change in order to be ready > with a full "rollback" story, so we have to be one version ahead.
If that is the case, I think we need to look at the entire roll-back story. At this point, I am not sure if we have identified all the pieces and if they fit together. Enrico, are you driving the roll-back story? or whoever, can you please come up with a comprehensive proposal? Thanks, JV On Mon, Jul 30, 2018 at 8:05 AM, Enrico Olivelli <eolive...@gmail.com> wrote: > Il giorno lun 30 lug 2018 alle ore 16:55 Venkateswara Rao Jujjuri < > jujj...@gmail.com> ha scritto: > > > We have introduced customMetadata in the past and that was not mandatory. > > How did we make that work with text format? > > Also, my preference is to wait until we have a need to introduce a new > > field for this conversion if needed. > > > > JV, > So you are saying that you prefer to defer this change to the first case, > when it will be really needed ? > > As far as I understand we are working on this change in order to be ready > with a full "rollback" story, so we have to be one version ahead. > > Enrico > > > > > > > > JV > > > > On Mon, Jul 30, 2018 at 12:42 AM, Enrico Olivelli <eolive...@gmail.com> > > wrote: > > > > > Il giorno lun 30 lug 2018 alle ore 09:17 Ivan Kelly <iv...@apache.org> > > ha > > > scritto: > > > > > > > >> Thank you for putting this together. It is also good to put this > as > > a > > > > BP, > > > > >> since it is about the metadata layout. > > > > > > > > I'll put a BP up this week once I have initial feedback. > > > > > > > > >> > - When writing a metadata, check what is in /ledgers/LAYOUT. If > it > > > is > > > > >> > as above, write using the current text protobuf. If is bumped, > use > > > the > > > > >> > new binary format. > > > > > > > > > > isn't it too costly? Adding a zk read for each write. We could add > a > > > > watch > > > > > but is has an important cost > > > > > > > > This cost can be amortized. In simplest case, we only read when > client > > > > starts (as already happens). Client are able both read and write the > > > > old format, so clients that read the pre-upgrade version will still > > > > write in text format, and this is fine. > > > > > > > > > > For me it is fine to read it when client starts. A rolling restart of > the > > > client application will be enough to load the new configuration. > > > > > > > > > > > > > > However, we would want them to eventually see the new format without > a > > > > restart. At one extreme, we could put a watch on the znode, but this > > > > could be quite heavy (I think the only per-client watch we have now > is > > > > on /ledgers/available). Alternatively, we could just read > > > > /ledgers/LAYOUT once per day. This way clients who saw the old > version > > > > will eventually get updated. > > > > > > > > > > I prefer NOT to use the watch, this cost will be payed forever even > when > > > there will be no configuration changes. > > > > > > I would go with the 'load at start' option, it is clear and works fine > > > > > > > > > > > > > > > > > > What about having a client side config writeMetadataVersion ? > > > > > We start a new metadata version, the new one will be encoded as > > > binary. > > > > > By default 4.8 clients will use previous version, as we already do > > for > > > > > journal and fileinfo on bookies. > > > > > > > > > > Each ledger is independent from the others, there is no need of a > > > global > > > > > flag written on zk. > > > > > > > > There is a global property here though,which only the administrator > > > > can control. It is "Can all clients in the cluster/application read > > > > the new ledger metadata version?". This is why I'd like to keep it as > > > > a centralized flag rather than allowing each client to decide for > > > > themselves. > > > > > > > > > > ok, let's keep it > > > > > > Enrico > > > > > > > > > > > > > > > > > > Don't we already have some tool? (Maybe I only have seen such tool > in > > > my > > > > > company applications) > > > > > > > > It may already exist, I haven't checked. > > > > > > > > -Ivan > > > > > > > > > > > > > > > -- > > Jvrao > > --- > > First they ignore you, then they laugh at you, then they fight you, then > > you win. - Mahatma Gandhi > > > -- Jvrao --- First they ignore you, then they laugh at you, then they fight you, then you win. - Mahatma Gandhi
