Il lun 30 lug 2018, 17:32 Venkateswara Rao Jujjuri <jujj...@gmail.com> ha scritto:
> > As far as I understand we are working on this change in order to be ready > > with a full "rollback" story, so we have to be one version ahead. > > If that is the case, I think we need to look at the entire roll-back story. > At this point, I am not sure if we have identified all the pieces and if > they fit together. > Enrico, are you driving the roll-back story? No, I am not. I am only coordinating 4.8 release and I would like to move things forward. I am not in a hurry for this change. To me we can defer it to 4.9 Enrico or whoever, can you please > come up with > a comprehensive proposal? > > Thanks, > JV > > > > > On Mon, Jul 30, 2018 at 8:05 AM, Enrico Olivelli <eolive...@gmail.com> > wrote: > > > Il giorno lun 30 lug 2018 alle ore 16:55 Venkateswara Rao Jujjuri < > > jujj...@gmail.com> ha scritto: > > > > > We have introduced customMetadata in the past and that was not > mandatory. > > > How did we make that work with text format? > > > Also, my preference is to wait until we have a need to introduce a new > > > field for this conversion if needed. > > > > > > > JV, > > So you are saying that you prefer to defer this change to the first case, > > when it will be really needed ? > > > > As far as I understand we are working on this change in order to be ready > > with a full "rollback" story, so we have to be one version ahead. > > > > Enrico > > > > > > > > > > > > > > JV > > > > > > On Mon, Jul 30, 2018 at 12:42 AM, Enrico Olivelli <eolive...@gmail.com > > > > > wrote: > > > > > > > Il giorno lun 30 lug 2018 alle ore 09:17 Ivan Kelly < > iv...@apache.org> > > > ha > > > > scritto: > > > > > > > > > >> Thank you for putting this together. It is also good to put this > > as > > > a > > > > > BP, > > > > > >> since it is about the metadata layout. > > > > > > > > > > I'll put a BP up this week once I have initial feedback. > > > > > > > > > > >> > - When writing a metadata, check what is in /ledgers/LAYOUT. > If > > it > > > > is > > > > > >> > as above, write using the current text protobuf. If is bumped, > > use > > > > the > > > > > >> > new binary format. > > > > > > > > > > > > isn't it too costly? Adding a zk read for each write. We could > add > > a > > > > > watch > > > > > > but is has an important cost > > > > > > > > > > This cost can be amortized. In simplest case, we only read when > > client > > > > > starts (as already happens). Client are able both read and write > the > > > > > old format, so clients that read the pre-upgrade version will still > > > > > write in text format, and this is fine. > > > > > > > > > > > > > For me it is fine to read it when client starts. A rolling restart of > > the > > > > client application will be enough to load the new configuration. > > > > > > > > > > > > > > > > > > However, we would want them to eventually see the new format > without > > a > > > > > restart. At one extreme, we could put a watch on the znode, but > this > > > > > could be quite heavy (I think the only per-client watch we have now > > is > > > > > on /ledgers/available). Alternatively, we could just read > > > > > /ledgers/LAYOUT once per day. This way clients who saw the old > > version > > > > > will eventually get updated. > > > > > > > > > > > > > I prefer NOT to use the watch, this cost will be payed forever even > > when > > > > there will be no configuration changes. > > > > > > > > I would go with the 'load at start' option, it is clear and works > fine > > > > > > > > > > > > > > > > > > > > > > > What about having a client side config writeMetadataVersion ? > > > > > > We start a new metadata version, the new one will be encoded as > > > > binary. > > > > > > By default 4.8 clients will use previous version, as we already > do > > > for > > > > > > journal and fileinfo on bookies. > > > > > > > > > > > > Each ledger is independent from the others, there is no need of a > > > > global > > > > > > flag written on zk. > > > > > > > > > > There is a global property here though,which only the administrator > > > > > can control. It is "Can all clients in the cluster/application read > > > > > the new ledger metadata version?". This is why I'd like to keep it > as > > > > > a centralized flag rather than allowing each client to decide for > > > > > themselves. > > > > > > > > > > > > > ok, let's keep it > > > > > > > > Enrico > > > > > > > > > > > > > > > > > > > > > > > Don't we already have some tool? (Maybe I only have seen such > tool > > in > > > > my > > > > > > company applications) > > > > > > > > > > It may already exist, I haven't checked. > > > > > > > > > > -Ivan > > > > > > > > > > > > > > > > > > > > > -- > > > Jvrao > > > --- > > > First they ignore you, then they laugh at you, then they fight you, > then > > > you win. - Mahatma Gandhi > > > > > > > > > -- > Jvrao > --- > First they ignore you, then they laugh at you, then they fight you, then > you win. - Mahatma Gandhi > -- -- Enrico Olivelli
