Thank you all ! I will copy the content of the Final draft to the Wiki and mark the document as "Accepted"
I will send a PR soon but it will depend on BP-15 New CreateLeader API I hope we could make it for 4.6 Enrico 2017-09-11 18:58 GMT+02:00 Sijie Guo <guosi...@gmail.com>: > Enrico, > > Feel free to close the thread and mark this BP as accepted, if there is no > -1. > > - Sijie > > On Mon, Sep 11, 2017 at 2:26 AM, Enrico Olivelli <eolive...@gmail.com> > wrote: > > > Ping > > > > 2017-09-07 9:32 GMT+02:00 Enrico Olivelli <eolive...@gmail.com>: > > > > > Hi all, > > > > > > > > > You can find the revised proposal here > > > https://cwiki.apache.org/confluence/display/BOOKKEEPER/ > > > BP-14+Relax+durability > > > > > > The link to the document open for comments is this: > > > https://docs.google.com/document/d/1yNi9t2_ > > deOOMXDaGzrnmaHTQeB3B3Fnym82DU > > > ERH7LM/edit?usp=sharing > > > > > > Please check it out > > > We are going to review this Proposal at the meeting > > > > > > -- Enrico > > > > > > > > > 2017-08-30 8:56 GMT+02:00 Enrico Olivelli <eolive...@gmail.com>: > > > > > >> Thank you Sijie for summarizing and thanks to the community for > helping > > >> in this important enhancement to BookKeeper > > >> > > >> I am convinced that as JV pointed out we need to declare at ledger > > >> creation time that the ledger is going to perform no-sync writes. > > >> > > >> I think we need an explicit declaration currently to make things > "clear" > > >> to the developer which is using the LedgerHandle API even and ledger > > >> creation tyime. > > >> > > >> The case is that we are going to forbid "striping" ledgers (ensemble > > size > > >> > quorum size) for no-sync writes in the first implementation: > > >> - one option is to fail at the first no-sync addEntry, but this will > be > > >> really uncomfortable because usually the ack/write/ensemble sizes are > > >> configured by the admin, and there will be configurations in which > > errors > > >> will come out only after starting the system. > > >> - the second option is to make the developer explicitly enable no-sync > > >> writes at creation time and fail the creation of the ledger if the > > >> requested combination of options if not possible > > >> > > >> I am not sure that the changes to the bookie internals are a > Client-API > > >> matter, maybe we can leverage custom metadata (as JV said) in order to > > make > > >> the bookie handle ledgers in a different manner, this way will be > always > > >> open as custom metadata are already here. > > >> > > >> JV preferred the ledger-type approach, the dual solution is to > introduce > > >> a list of "capabilities" or "ledger options". > > >> I think that this ability to perform no-syc writes is so important > that > > >> "custom metadata" is not the good place to declare it, same for > "ledger > > >> type" > > >> > > >> So I am proposing to add a boolean 'allowNoSyncWrites" at ledger > > creation > > >> time, without writing in to ledger metadata on ZK, > > >> I think that if further improvements will need ledger metadata changes > > we > > >> will do. > > >> > > >> I have updated the BP-14 document, I have added an "Open issues" > footer > > >> with the open points, > > >> please add comments and I will correct the document as soon as > possible. > > >> > > >> > > >> Enrico > > >> > > >> > > >> > > >> > > >> 2017-08-30 1:24 GMT+02:00 Sijie Guo <guosi...@gmail.com>: > > >> > > >>> Thank you, Enrico, JV. > > >>> > > >>> These are great discussions. > > >>> > > >>> After reading these two proposals, I have a few very high-level > > comments, > > >>> dividing into three categories. > > >>> > > >>> > > >>> *API* > > >>> > > >>> - I think there are not fundamentally differences between these two > > >>> proposals. > > >>> They are trying to achieve similar goals by exposing durability > levels > > in > > >>> different way. > > >>> So this will be a discussion on what API/interface should look like > > from > > >>> user / admin perspective. > > >>> I would suggest focusing what would be the API itself, putting the > > >>> implementation design aside when talking about this. > > >>> > > >>> *Core* > > >>> > > >>> - Both proposals need to deal with a core function - what happen to > LAC > > >>> and > > >>> what semantic that bookkeeper provides. > > >>> JV did a good summary in his proposal. However I am not a fan of > > >>> maintaining two different semantics. So I am looking for > > >>> a solution that bookkeeper can only maintain one semantic. The > semantic > > >>> is > > >>> basically: > > >>> > > >>> 1) LAC only advanced when entries before LAC are committed to the > > >>> persistent storage > > >>> 2) All the entries until LAC are successfully committed to the > > >>> persistence > > >>> storage > > >>> 3) Entries until LAC: all the entries must be readable all the time. > > >>> > > >>> If we maintain such semantic, there is no need to change the auto > > >>> recovery > > >>> protocol in bookkeeper. All what we guarantee are the entries durably > > >>> persistent. > > >>> > > >>> In order to maintain such semantic, I think both me and JV proposed > > >>> similar > > >>> solution in either proposal. I am trying to finalize one here: > > >>> > > >>> * bookie maintains a LAS (Last Add Synced) point for each entry. > > >>> * LAS can be piggybacked on AddResponses > > >>> * Client uses the LAS to advance LAC. > > >>> > > >>> If we can agree on the core semantic we are going to provide, the > other > > >>> things are just logistics. > > >>> > > >>> *Others* > > >>> > > >>> - Regarding separating journal or bypassing journal, there is no > > >>> difference > > >>> when we talking from the core semantic. They are all non-durably > writes > > >>> (acknowledging before fsyncing). > > >>> We can start with same journal approach (but just acknowledge before > > >>> fsyncing), implement the core and add other options later on. > > >>> > > >>> > > >>> From my point of view, I'd be more interesting in providing a single > > >>> consistent durable semantic that application can rely on for both > > durable > > >>> writes and non-durable writes. The other stuffs seem to be more > > logistics > > >>> things. > > >>> > > >>> > > >>> - Sijie > > >>> > > >>> > > >>> On Mon, Aug 28, 2017 at 11:27 PM, Enrico Olivelli < > eolive...@gmail.com > > > > > >>> wrote: > > >>> > > >>> > 2017-08-29 8:01 GMT+02:00 Venkateswara Rao Jujjuri < > > jujj...@gmail.com > > >>> >: > > >>> > > > >>> > > I don't believe I fully followed your second case. But even in > this > > >>> case, > > >>> > > your major concern is about the additional 'sync' RPC? > > >>> > > > > >>> > > > >>> > yes apart from that I am fine with your proposal too, that is to > > have a > > >>> > LedgerType which drives durability > > >>> > and I think we need to add per-entry durability options > > >>> > > > >>> > I think that at least for the 'simple' no-sync addEntry we do not > > need > > >>> to > > >>> > change many things, I am drafting a prototype, I will share it as > > soon > > >>> as > > >>> > we all agree on the roadmap > > >>> > > > >>> > The first implementation can cover the first cases (no-sync > addEntry) > > >>> and > > >>> > change the way the writer advances the LAC in order to support > > 'relaxed > > >>> > durability writes'. > > >>> > This change will be compatible with future improvements and it will > > >>> open > > >>> > the door for big changes on the bookie side like bypassing the > > journal > > >>> or > > >>> > leveraging multiple journals..... > > >>> > > > >>> > -- Enrico > > >>> > > > >>> > or something else that the LedgerType proposal won't work? > > >>> > > > > >>> > > > >>> > > > > >>> > > > > >>> > > On Mon, Aug 28, 2017 at 7:35 AM, Enrico Olivelli < > > >>> eolive...@gmail.com> > > >>> > > wrote: > > >>> > > > > >>> > > > I think that having a set of options on the ledger metadata > will > > >>> be a > > >>> > > good > > >>> > > > enhancement and I am sure we will do it as soon as it will be > > >>> needed, > > >>> > > maybe > > >>> > > > we do not need it now. > > >>> > > > > > >>> > > > Actually I think we will need to declare this durability-level > at > > >>> entry > > >>> > > > level to support some uses cases in BP-14 document, let me > > explain > > >>> two > > >>> > of > > >>> > > > my usecases for which I need it: > > >>> > > > > > >>> > > > At higher level we have to choices: > > >>> > > > > > >>> > > > A) per-ledger durability options (JV proposal) > > >>> > > > all addEntry operations are durable or non-durable and there is > > an > > >>> > > explicit > > >>> > > > 'sync' API (+ forced sync at close) > > >>> > > > > > >>> > > > B) per-entry durability options (original BP-14 proposal) > > >>> > > > every addEntry has an own durable/non-durable option > > >>> (sync/no-sync), > > >>> > with > > >>> > > > the ability to call 'sync' without addEntry (+ forced sync at > > >>> close) > > >>> > > > > > >>> > > > I am speaking about the the database WAL case, I am using the > > >>> ledger as > > >>> > > > segment for the WAL of a database and I am writing all data > > >>> changes in > > >>> > > the > > >>> > > > scope of a 'transaction' with the relaxed-durability flag, > then I > > >>> am > > >>> > > > writing the 'transaction committed' entry with "strict > > durability" > > >>> > > > requirement, this will in fact require that all previous > entries > > >>> are > > >>> > > > persisted durably and so that the transaction will never be > lost. > > >>> > > > > > >>> > > > In this scenario we would need an addEntry + sync API in fact: > > >>> > > > > > >>> > > > using option A) the WAL will look like: > > >>> > > > - open ledger no-sync = true > > >>> > > > - addEntry (set foo=bar) (this will be no-sync) > > >>> > > > - addEntry (set foo=bar2) (this will be no-sync) > > >>> > > > - addEntry (commit) > > >>> > > > - sync > > >>> > > > > > >>> > > > using option B) the WAL will look like > > >>> > > > - open ledger > > >>> > > > - addEntry (set foo=bar), no-sync > > >>> > > > - addEntry (set foo=bar2), no-sync > > >>> > > > - addEntry (commit), sync > > >>> > > > > > >>> > > > in case B) we are "saving" one RPC call to every bookie (the > > 'sync' > > >>> > one) > > >>> > > > same for single data change entries, like updating a single > > record > > >>> on > > >>> > the > > >>> > > > database, this with BK 4.5 "costs" only a single RPC to every > > >>> bookie > > >>> > > > > > >>> > > > Second case: > > >>> > > > I am using BookKeeper to store binary objects, so I am packing > > more > > >>> > > > 'objects' (named sequences of bytes) into a single ledger, like > > >>> you do > > >>> > > when > > >>> > > > you write many records to a file in a streaming fashion and > keep > > >>> track > > >>> > of > > >>> > > > offsets of the beginning of every record (LedgerHandeAdv is > > >>> perfect for > > >>> > > > this case). > > >>> > > > I am not using a single ledger per 'file' because it kills > > >>> zookeeper to > > >>> > > > create many ledgers very fast, in my systems I have big busts > of > > >>> > writes, > > >>> > > > which need to be really "fast", so I am writing multiple > 'files' > > to > > >>> > every > > >>> > > > single ledger. So the close-to-open consistency at ledger level > > is > > >>> not > > >>> > > > suitable for this case. > > >>> > > > I have to write as fast as possible to this 'ledger-backed' > > >>> stream, and > > >>> > > as > > >>> > > > with a 'traditional' filesystem I am writing parts of each > file > > >>> and > > >>> > than > > >>> > > > requiring 'sync' at the end of each file. > > >>> > > > Using BookKeeper you need to split big 'files' into "little" > > >>> parts, you > > >>> > > > cannot transmit the contents as to "real" stream on network. > > >>> > > > > > >>> > > > I am not talking about bookie level implementation details I > > would > > >>> like > > >>> > > to > > >>> > > > define the high level API in order to support all the relevant > > >>> known > > >>> > use > > >>> > > > cases and keep space for the future, > > >>> > > > at this moment adding a per-entry 'durability option' seems to > be > > >>> very > > >>> > > > flexible and simple to implement, it does not prevent us from > > doing > > >>> > > further > > >>> > > > improvements, like namely skipping the journal. > > >>> > > > > > >>> > > > Enrico > > >>> > > > > > >>> > > > > > >>> > > > > > >>> > > > 2017-08-26 19:55 GMT+02:00 Enrico Olivelli < > eolive...@gmail.com > > >: > > >>> > > > > > >>> > > > > > > >>> > > > > > > >>> > > > > On sab 26 ago 2017, 19:19 Venkateswara Rao Jujjuri < > > >>> > jujj...@gmail.com> > > >>> > > > > wrote: > > >>> > > > > > > >>> > > > >> Hi all, > > >>> > > > >> > > >>> > > > >> As promised during Thursday call, here is my proposal. > > >>> > > > >> > > >>> > > > >> *NOTE*: Major difference in this proposal compared to > Enrico’s > > >>> > > > >> <https://docs.google.com/document/d/1JLYO3K3tZ5PJGmyS0YK_- > > >>> > > > >> NW8VOUUgUWVBmswCUOG158/edit#heading=h.q2rewiqndr5v> > > >>> > > > >> is > > >>> > > > >> making the durability a property of the ledger(type) as > > opposed > > >>> to > > >>> > > > >> addEntry(). Rest of the technical details have a lot of > > >>> > similarities. > > >>> > > > >> > > >>> > > > > > > >>> > > > > Thank you JV. I have just read quickly the doc and your view > is > > >>> > > centantly > > >>> > > > > broader. > > >>> > > > > I will dig into the doc as soon as possible on Monday. > > >>> > > > > For me it is ok to have a ledger wide configuration I think > > that > > >>> the > > >>> > > most > > >>> > > > > important decision is about the API we will provide as in the > > >>> future > > >>> > it > > >>> > > > > will be difficult to change it. > > >>> > > > > > > >>> > > > > > > >>> > > > > Cheers > > >>> > > > > Enrico > > >>> > > > > > > >>> > > > > > > >>> > > > > > > >>> > > > >> https://docs.google.com/document/d/ > 1g1eBcVVCZrTG8YZliZP0LVqv > > >>> Wpq43 > > >>> > > > >> 2ODEghrGVQ4d4Q/edit?usp=sharing > > >>> > > > >> > > >>> > > > >> On Thu, Aug 24, 2017 at 1:14 AM, Enrico Olivelli < > > >>> > eolive...@gmail.com > > >>> > > > > > >>> > > > >> wrote: > > >>> > > > >> > > >>> > > > >> > Thank you all for the comments and for taking a look to > the > > >>> > document > > >>> > > > so > > >>> > > > >> > soon. > > >>> > > > >> > I have updated the doc, we will discuss the document at > the > > >>> > meeting, > > >>> > > > >> > > > >>> > > > >> > > > >>> > > > >> > Enrico > > >>> > > > >> > > > >>> > > > >> > 2017-08-24 2:27 GMT+02:00 Sijie Guo <guosi...@gmail.com>: > > >>> > > > >> > > > >>> > > > >> > > Enrico, > > >>> > > > >> > > > > >>> > > > >> > > Thank you so much! It is a great effort for putting this > > up. > > >>> > > Overall > > >>> > > > >> > looks > > >>> > > > >> > > good. I made some comments, we can discuss at tomorrow's > > >>> > community > > >>> > > > >> > meeting. > > >>> > > > >> > > > > >>> > > > >> > > - Sijie > > >>> > > > >> > > > > >>> > > > >> > > On Wed, Aug 23, 2017 at 8:25 AM, Enrico Olivelli < > > >>> > > > eolive...@gmail.com > > >>> > > > >> > > > >>> > > > >> > > wrote: > > >>> > > > >> > > > > >>> > > > >> > > > Hi all, > > >>> > > > >> > > > I have drafted a first proposal for BP-14 - Relax > > >>> Durability > > >>> > > > >> > > > > > >>> > > > >> > > > We are talking about limiting the number of fsync to > the > > >>> > journal > > >>> > > > >> while > > >>> > > > >> > > > preserving the correctness of the LAC protocol. > > >>> > > > >> > > > > > >>> > > > >> > > > This is the link to the wiki page, but as the issue is > > >>> huge we > > >>> > > > >> prefer > > >>> > > > >> > to > > >>> > > > >> > > > use Google Documents for sharing comments > > >>> > > > >> > > > https://cwiki.apache.org/ > confluence/display/BOOKKEEPER/ > > >>> > > > >> > > > BP+-+14+Relax+durability > > >>> > > > >> > > > > > >>> > > > >> > > > This is the document > > >>> > > > >> > > > https://docs.google.com/document/d/ > > 1JLYO3K3tZ5PJGmyS0YK_- > > >>> > > > >> > > > NW8VOUUgUWVBmswCUOG158/edit?usp=sharing > > >>> > > > >> > > > > > >>> > > > >> > > > All comments are welcome > > >>> > > > >> > > > > > >>> > > > >> > > > I have added DL dev list in cc as the discussion is > > >>> > interesting > > >>> > > > for > > >>> > > > >> > both > > >>> > > > >> > > > groups > > >>> > > > >> > > > > > >>> > > > >> > > > Enrico Olivelli > > >>> > > > >> > > > > > >>> > > > >> > > > > >>> > > > >> > > > >>> > > > >> > > >>> > > > >> > > >>> > > > >> > > >>> > > > >> -- > > >>> > > > >> Jvrao > > >>> > > > >> --- > > >>> > > > >> First they ignore you, then they laugh at you, then they > fight > > >>> you, > > >>> > > then > > >>> > > > >> you win. - Mahatma Gandhi > > >>> > > > >> > > >>> > > > > -- > > >>> > > > > > > >>> > > > > > > >>> > > > > -- Enrico Olivelli > > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > -- > > >>> > > Jvrao > > >>> > > --- > > >>> > > First they ignore you, then they laugh at you, then they fight > you, > > >>> then > > >>> > > you win. - Mahatma Gandhi > > >>> > > > > >>> > > > >>> > > >> > > >> > > > > > >
