I have created BP-44 for this proposal. Regarding Flavio's questions, we can address those during the review process and Ivan can chip in regarding the performance aspects.
https://github.com/apache/bookkeeper/issues/2705 https://github.com/apache/bookkeeper/pull/2706 Jack On Wed, May 5, 2021 at 9:01 AM Lothruin Mirwen <lothruin.mir...@gmail.com> wrote: > [ External sender. Exercise caution. ] > > Great news! > I'm so excited to see it at work :) > > Diego > > Il giorno mar 4 mag 2021 alle ore 20:38 Venkateswara Rao Jujjuri < > jujj...@gmail.com> ha scritto: > > > > how you justify removing the ledger as opposed to removing the ledger > > storage and preserving the journal > > > > I will be waiting for the BP on this point too. :) But glad to see that > we > > are working to avoid double writes. :) > > With the EntryLogPerLedger (ELPL) feature, it is a more-or-less journal > per > > ledger. > > With the entrylogs, we need to maintain index files and journals, hence > > two writes if we want to persist > > data in-lieu of Journal. > > > > Another way to think about this is, having a ledger durability mode. Does > > it need fragment level durability, or durability at close. > > Based on that we can completely avoid journal writes with ELPL + flush on > > close. > > > > Thanks, > > JV > > > > On Mon, May 3, 2021 at 8:34 AM Anup Ghatage <ghat...@gmail.com> wrote: > > > > > HI! > > > > > > I know we are interested in this for sure. (cc @Venkateswara Rao > Jujjuri > > > <jujj...@gmail.com>) > > > Is this similar to Matteo Merli's PR which I found was simple and still > > > got the job done: https://github.com/apache/bookkeeper/pull/2401/files > > > > > > Regards, > > > Anup > > > > > > On Mon, May 3, 2021 at 8:04 AM Flavio Junqueira <f...@apache.org> > wrote: > > > > > >> +1, it makes sense to enable bookies to run without duplicating IOs > for > > >> entry data. I'm curious to see how you justify removing the ledger as > > >> opposed to removing the ledger storage and preserving the journal. I > > >> suspect that the random reads against the ledger storage matter more > to > > you > > >> than the sequential writes, and you're possibly able to make it > perform > > >> well enough with SSD and even NVMe drives. > > >> > > >> I should wait for your write up rather than speculate. Looking forward > > to > > >> seeing the BP. > > >> > > >> -Flavio > > >> > > >> > On 3 May 2021, at 16:52, Enrico Olivelli <eolive...@gmail.com> > wrote: > > >> > > > >> > Il giorno lun 3 mag 2021 alle ore 16:30 Jack Vanlightly > > >> > <jvanligh...@splunk.com.invalid <mailto: > > jvanligh...@splunk.com.invalid>> > > >> ha scritto: > > >> >> > > >> >> Hi all, > > >> >> > > >> >> At Splunk we have defined and implemented changes to BookKeeper to > > >> allow > > >> >> bookies to run without the journal. The motivation for this work is > > to > > >> >> allow BookKeeper to be run with lower operating costs while still > > >> offering > > >> >> decent data safety guarantees. > > >> >> > > >> >> Before submitting the work as a PR we'd like to formalise the > > proposed > > >> >> changes in a BP where we state our motivation, explain the protocol > > >> >> changes, the work on formally verifying the proposal and be open to > > >> comment. > > >> >> > > >> >> We'll create a BP this week if that sounds good to you all. > > >> > > > >> > Great to hear that ! > > >> > > > >> > Thanks > > >> > Enrico > > >> > > > >> >> > > >> >> Thanks > > >> >> Jack > > >> >> > > >> >> -- > > >> >> *Jack Vanlightly* > > >> >> Principal Software Engineer > > >> >> Splunk Inc. > > >> >> jvanligh...@splunk.com <mailto:jvanligh...@splunk.com> < > > >> kramas...@splunk.com <mailto:kramas...@splunk.com>> > > >> >> Barcelona > > >> > > >> > > > > > > -- > > > Anup Ghatage > > > www.ghatage.com > > > > > > > > > -- > > Jvrao > > --- > > First they ignore you, then they laugh at you, then they fight you, then > > you win. - Mahatma Gandhi > > >
