I think we should avoid the long representation and interpretation in the new code, basically -1 should be looked as MSB of the lsb-ledgerIDset to 1. Also, we should be able to provide BC without special cases. Branching should be based on if msb-ledgerId(scopeID) is 0 or not.
Maybe we should treat the entire 128 bit (two longs) as a bitmap and add corresponding utility library? > We are treating the global space of ledgerids as a 2 level hierarchy My thought is, we are treating it this way because of the lack of native support. Not for the backward compatibility reasons. > the full 128bit space is handled, but there is a class of combinations which won't work (scopeid = 0 and ledgerid < 0). Not sure, I understand this. We need to thrive towards handling full 128 bits. What is stopping us from that? On Sat, Aug 18, 2018 at 10:37 AM, Enrico Olivelli <eolive...@gmail.com> wrote: > Il sab 18 ago 2018, 18:43 Sijie Guo <guosi...@gmail.com> ha scritto: > > > On Sat, Aug 18, 2018 at 9:28 AM Enrico Olivelli <eolive...@gmail.com> > > wrote: > > > > > Il sab 18 ago 2018, 17:26 Sijie Guo <guosi...@gmail.com> ha scritto: > > > > > > > On Sat, Aug 18, 2018 at 2:12 AM Enrico Olivelli <eolive...@gmail.com > > > > > > wrote: > > > > > > > > > Nice work Sijie, > > > > > Some comments after the first read, just thinking out loud... > > > > > Overall looks a good idea and implementable. > > > > > > > > > > We should cite somewhere the case of scope=0 and ledgerId = -1 > which > > > is a > > > > > very special case for usthe. > > > > > > > > > > > > > For BC reason, negative ledger id are not supported when scope == 0. > We > > > > effectively only support 63 bit. > > > > > > > > > > Fine to me > > > > > > We should state clearly in the BP and then in javadocs and docs that > not > > > the full 128bit space is handled, > > > > but there is a class of combinations > > > which won't work (scopeid = 0 and ledgerid < 0). > > > > > > > Sure. But I think that would be clearly until we implement it. So I would > > defer the clarification until then. Because there can be other more > corner > > cases. > > > > > > > > > > > > > > > > > > > > > We can use versions on clients to distinguish whether they are old > > > clients > > > > or new clients, then handle this case differently. > > > > > > > > So this special case should not be special for any new clients. > > > > > > > > We should also state that scopeid and ledgerid < 0 is not supported > or > > it > > > > > is not clear to me how we will handle that case. > > > > > The case of scope < 0 is not clear to me as well, maybe I have to > > read > > > > more > > > > > carefully the doc. > > > > > > > > > > > > I think I answer this with my comment above. > > > > > > > > > > > > > > > > > > We are treating the global space of ledgerids as a 2 level > hierarchy > > > and > > > > > this is good for compatibility. > > > > > But thinking about a new application, which starts with uuids I > think > > > it > > > > > can be tricky, maybe we should add utilities/APIs to work with UUID > > > > > directly. > > > > > > > > > > > > The ledgerQualifiedName is the hex presentation of UUID, proposed to > > add > > > to > > > > CLI tool. > > > > > > > > We can add it to API as well. However I would defer adding it until > we > > > need > > > > it or after we are on 128 bits. BC is the top concern. > > > > > > > > > > > > > For this new kind of application the space of ids would be flat and > > not > > > > > hierarchical (UUID are not hierarchical as far as I know, but I may > > be > > > > > wrong). > > > > > > > > > > > > In a lot of places, 128 bit is not well supported. So people would > > divide > > > > into two parts any way, msb and lsb. > > > > > > > > > > > > > > > > > > > > > > > We should also support listing the whole flat space > > > > > > > > > > > > Sure it can be added. I don’t see a reason we can’t. However IMO > these > > > are > > > > new features after we are on 128 bit. For the concerns in this BP, we > > are > > > > more focused on how we can transition from 64 bit to 128 bit. > > > > > > > > > > Okay > > > > > > Last comment: > > > We are going to have a lot of special cases in layout of data/zk ledger > > > manager, to handle scopeid = 0. > > > > > > > I am thinking to a brand new installation, without upgrade/BC > concerns, I > > > wonder if we have some flag not to have that tricks, I am talking about > > the > > > different dbs on rocks db, the layout of indexes...and so on. > > > > > > I don’t think we should provide this flag since it would make things > very > > complicated. In that case, you can just provide a brand new metadata > > driver, a new ledger storage. The you don’t need to worry about this > > upgrade story. However that will be brand new implementation, which would > > be out of the scope of this BP. > > > > Okay > > To me it is clear. > > +1 > > Enrico > > > > Sijie > > > > > > > > > > > > This is not a real problem but I wonder if we ca handle this case of > new > > > applications cleanly > > > > > > Enrico > > > > > > > > > > > > > or as a last > > > > > alternative we should add a way to list the space of scopeids, > > because > > > > > otherwise the application must store the list of scopeids, and for > it > > > it > > > > > will be a set of half UUIDs. > > > > > > > > > > Thanks for driving this > > > > > > > > > > Enrico > > > > > > > > > > > > > > > > > > > > Il ven 17 ago 2018, 23:52 Venkateswara Rao Jujjuri < > > jujj...@gmail.com> > > > > ha > > > > > scritto: > > > > > > > > > > > Thanks a lot, Sijie for the awesome doc. > > > > > > Let's pool some more thoughts into this and size the work. > > > > > > > > > > > > Thanks, > > > > > > JV > > > > > > > > > > > > On Fri, Aug 17, 2018 at 2:36 PM, Sijie Guo <guosi...@gmail.com> > > > wrote: > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > As promised in last community meeting, I put up all the > thoughts > > > and > > > > > > > discussion with JV I had into a BP for supporting 128-bit > ledger > > > id. > > > > I > > > > > > > tried my best to cover all the aspects that I can think of. > There > > > can > > > > > be > > > > > > > places that I miss. so please take a look and let me know what > > you > > > > > think. > > > > > > > > > > > > > > BP PR: https://github.com/apache/bookkeeper/pull/1611 > > > > > > > Google doc: > > > > > > > > > https://docs.google.com/document/d/1cu54dNSV2ZrdWCi40LcyX8NxXGRCW > > > > > > > 0609T_ewmK9BWM > > > > > > > > > > > > > > - Sijie > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Jvrao > > > > > > --- > > > > > > First they ignore you, then they laugh at you, then they fight > you, > > > > then > > > > > > you win. - Mahatma Gandhi > > > > > > > > > > > -- > > > > > > > > > > > > > > > -- Enrico Olivelli > > > > > > > > > > > > -- > > > > > > > > > -- Enrico Olivelli > > > > > > -- > > > -- Enrico Olivelli > -- Jvrao --- First they ignore you, then they laugh at you, then they fight you, then you win. - Mahatma Gandhi
