Re: Changing ledger metadata to binary format

2019-06-03 Thread Ivan Kelly
On Thu, May 30, 2019 at 12:15 AM Venkateswara Rao Jujjuri wrote: > > > "Let's decide this when we need it" > > Right. This is the time as we are trying to add fault domain info to the > cookie. When you say cookie, you mean under /ledgers/cookies? That falls outside the scope of binary metadata.

Re: Changing ledger metadata to binary format

2019-05-29 Thread Venkateswara Rao Jujjuri
> "Let's decide this when we need it" Right. This is the time as we are trying to add fault domain info to the cookie. > to make a flag in the metadata store that we flip to enable it. Right I do recall the same. But given your proposal in this thread, all readers try to read in binary format, if

Re: Changing ledger metadata to binary format

2019-05-29 Thread Ivan Kelly
> What is our plan to move forward with binary format? I've not plans regarding it. Moving forward with it will happen when someone comes with a metadata change which will break text metadata users (i.e. almost any metadata change). > Anyone using binary format in production? even for new cluster

Re: Changing ledger metadata to binary format

2019-05-28 Thread Venkateswara Rao Jujjuri
Hi all, bringing back this conversation back to life.. https://github.com/apache/bookkeeper/issues/723 What is our plan to move forward with binary format? Anyone using binary format in production? even for new clusters? What is blocking us from using binary format for the new clusters? On Mon,

Re: Changing ledger metadata to binary format

2018-07-30 Thread Sijie Guo
JV, 1) for the real world use case, this is the problem I have hit which I have to manually add a flag in the ledger metadata to avoid serializing ctime. https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java#L105 2) > We

Re: Changing ledger metadata to binary format

2018-07-30 Thread Enrico Olivelli
Il lun 30 lug 2018, 17:32 Venkateswara Rao Jujjuri ha scritto: > > As far as I understand we are working on this change in order to be ready > > with a full "rollback" story, so we have to be one version ahead. > > If that is the case, I think we need to look at the entire roll-back story. > At t

Re: Changing ledger metadata to binary format

2018-07-30 Thread Venkateswara Rao Jujjuri
> As far as I understand we are working on this change in order to be ready > with a full "rollback" story, so we have to be one version ahead. If that is the case, I think we need to look at the entire roll-back story. At this point, I am not sure if we have identified all the pieces and if they

Re: Changing ledger metadata to binary format

2018-07-30 Thread Enrico Olivelli
Il giorno lun 30 lug 2018 alle ore 16:55 Venkateswara Rao Jujjuri < jujj...@gmail.com> ha scritto: > We have introduced customMetadata in the past and that was not mandatory. > How did we make that work with text format? > Also, my preference is to wait until we have a need to introduce a new > fi

Re: Changing ledger metadata to binary format

2018-07-30 Thread Venkateswara Rao Jujjuri
We have introduced customMetadata in the past and that was not mandatory. How did we make that work with text format? Also, my preference is to wait until we have a need to introduce a new field for this conversion if needed. JV On Mon, Jul 30, 2018 at 12:42 AM, Enrico Olivelli wrote: > Il gior

Re: Changing ledger metadata to binary format

2018-07-30 Thread Enrico Olivelli
Il giorno lun 30 lug 2018 alle ore 09:17 Ivan Kelly ha scritto: > >> Thank you for putting this together. It is also good to put this as a > BP, > >> since it is about the metadata layout. > > I'll put a BP up this week once I have initial feedback. > > >> > - When writing a metadata, check what

Re: Changing ledger metadata to binary format

2018-07-30 Thread Ivan Kelly
>> Thank you for putting this together. It is also good to put this as a BP, >> since it is about the metadata layout. I'll put a BP up this week once I have initial feedback. >> > - When writing a metadata, check what is in /ledgers/LAYOUT. If it is >> > as above, write using the current text pr

Re: Changing ledger metadata to binary format

2018-07-28 Thread Enrico Olivelli
Il ven 27 lug 2018, 20:30 Sijie Guo ha scritto: > Ivan, > > Thank you for putting this together. It is also good to put this as a BP, > since it is about the metadata layout. > > On Fri, Jul 27, 2018 at 10:54 AM Ivan Kelly wrote: > > > Hi folks, > > > > I think this was discussed yesterday in th

Re: Changing ledger metadata to binary format

2018-07-27 Thread Sijie Guo
Ivan, Thank you for putting this together. It is also good to put this as a BP, since it is about the metadata layout. On Fri, Jul 27, 2018 at 10:54 AM Ivan Kelly wrote: > Hi folks, > > I think this was discussed yesterday in the meeting, and a bit on > slack, but I haven't seen anything much w

Changing ledger metadata to binary format

2018-07-27 Thread Ivan Kelly
Hi folks, I think this was discussed yesterday in the meeting, and a bit on slack, but I haven't seen anything much written down, so I'm starting a thread here. The crux of the problem is that the protobuf text format currently used for metadata cannot have new fields added without breaking clien