We have introduced customMetadata in the past and that was not mandatory. How did we make that work with text format? Also, my preference is to wait until we have a need to introduce a new field for this conversion if needed.
JV On Mon, Jul 30, 2018 at 12:42 AM, Enrico Olivelli <eolive...@gmail.com> wrote: > Il giorno lun 30 lug 2018 alle ore 09:17 Ivan Kelly <iv...@apache.org> ha > scritto: > > > >> Thank you for putting this together. It is also good to put this as a > > BP, > > >> since it is about the metadata layout. > > > > I'll put a BP up this week once I have initial feedback. > > > > >> > - When writing a metadata, check what is in /ledgers/LAYOUT. If it > is > > >> > as above, write using the current text protobuf. If is bumped, use > the > > >> > new binary format. > > > > > > isn't it too costly? Adding a zk read for each write. We could add a > > watch > > > but is has an important cost > > > > This cost can be amortized. In simplest case, we only read when client > > starts (as already happens). Client are able both read and write the > > old format, so clients that read the pre-upgrade version will still > > write in text format, and this is fine. > > > > For me it is fine to read it when client starts. A rolling restart of the > client application will be enough to load the new configuration. > > > > > > However, we would want them to eventually see the new format without a > > restart. At one extreme, we could put a watch on the znode, but this > > could be quite heavy (I think the only per-client watch we have now is > > on /ledgers/available). Alternatively, we could just read > > /ledgers/LAYOUT once per day. This way clients who saw the old version > > will eventually get updated. > > > > I prefer NOT to use the watch, this cost will be payed forever even when > there will be no configuration changes. > > I would go with the 'load at start' option, it is clear and works fine > > > > > > > > What about having a client side config writeMetadataVersion ? > > > We start a new metadata version, the new one will be encoded as > binary. > > > By default 4.8 clients will use previous version, as we already do for > > > journal and fileinfo on bookies. > > > > > > Each ledger is independent from the others, there is no need of a > global > > > flag written on zk. > > > > There is a global property here though,which only the administrator > > can control. It is "Can all clients in the cluster/application read > > the new ledger metadata version?". This is why I'd like to keep it as > > a centralized flag rather than allowing each client to decide for > > themselves. > > > > ok, let's keep it > > Enrico > > > > > > > > Don't we already have some tool? (Maybe I only have seen such tool in > my > > > company applications) > > > > It may already exist, I haven't checked. > > > > -Ivan > > > -- Jvrao --- First they ignore you, then they laugh at you, then they fight you, then you win. - Mahatma Gandhi
