>> Thank you for putting this together. It is also good to put this as a BP, >> since it is about the metadata layout.
I'll put a BP up this week once I have initial feedback. >> > - When writing a metadata, check what is in /ledgers/LAYOUT. If it is >> > as above, write using the current text protobuf. If is bumped, use the >> > new binary format. > > isn't it too costly? Adding a zk read for each write. We could add a watch > but is has an important cost This cost can be amortized. In simplest case, we only read when client starts (as already happens). Client are able both read and write the old format, so clients that read the pre-upgrade version will still write in text format, and this is fine. However, we would want them to eventually see the new format without a restart. At one extreme, we could put a watch on the znode, but this could be quite heavy (I think the only per-client watch we have now is on /ledgers/available). Alternatively, we could just read /ledgers/LAYOUT once per day. This way clients who saw the old version will eventually get updated. > What about having a client side config writeMetadataVersion ? > We start a new metadata version, the new one will be encoded as binary. > By default 4.8 clients will use previous version, as we already do for > journal and fileinfo on bookies. > > Each ledger is independent from the others, there is no need of a global > flag written on zk. There is a global property here though,which only the administrator can control. It is "Can all clients in the cluster/application read the new ledger metadata version?". This is why I'd like to keep it as a centralized flag rather than allowing each client to decide for themselves. > Don't we already have some tool? (Maybe I only have seen such tool in my > company applications) It may already exist, I haven't checked. -Ivan
