> how you justify removing the ledger as opposed to removing the ledger storage and preserving the journal
I will be waiting for the BP on this point too. :) But glad to see that we are working to avoid double writes. :) With the EntryLogPerLedger (ELPL) feature, it is a more-or-less journal per ledger. With the entrylogs, we need to maintain index files and journals, hence two writes if we want to persist data in-lieu of Journal. Another way to think about this is, having a ledger durability mode. Does it need fragment level durability, or durability at close. Based on that we can completely avoid journal writes with ELPL + flush on close. Thanks, JV On Mon, May 3, 2021 at 8:34 AM Anup Ghatage <ghat...@gmail.com> wrote: > HI! > > I know we are interested in this for sure. (cc @Venkateswara Rao Jujjuri > <jujj...@gmail.com>) > Is this similar to Matteo Merli's PR which I found was simple and still > got the job done: https://github.com/apache/bookkeeper/pull/2401/files > > Regards, > Anup > > On Mon, May 3, 2021 at 8:04 AM Flavio Junqueira <f...@apache.org> wrote: > >> +1, it makes sense to enable bookies to run without duplicating IOs for >> entry data. I'm curious to see how you justify removing the ledger as >> opposed to removing the ledger storage and preserving the journal. I >> suspect that the random reads against the ledger storage matter more to you >> than the sequential writes, and you're possibly able to make it perform >> well enough with SSD and even NVMe drives. >> >> I should wait for your write up rather than speculate. Looking forward to >> seeing the BP. >> >> -Flavio >> >> > On 3 May 2021, at 16:52, Enrico Olivelli <eolive...@gmail.com> wrote: >> > >> > Il giorno lun 3 mag 2021 alle ore 16:30 Jack Vanlightly >> > <jvanligh...@splunk.com.invalid <mailto:jvanligh...@splunk.com.invalid>> >> ha scritto: >> >> >> >> Hi all, >> >> >> >> At Splunk we have defined and implemented changes to BookKeeper to >> allow >> >> bookies to run without the journal. The motivation for this work is to >> >> allow BookKeeper to be run with lower operating costs while still >> offering >> >> decent data safety guarantees. >> >> >> >> Before submitting the work as a PR we'd like to formalise the proposed >> >> changes in a BP where we state our motivation, explain the protocol >> >> changes, the work on formally verifying the proposal and be open to >> comment. >> >> >> >> We'll create a BP this week if that sounds good to you all. >> > >> > Great to hear that ! >> > >> > Thanks >> > Enrico >> > >> >> >> >> Thanks >> >> Jack >> >> >> >> -- >> >> *Jack Vanlightly* >> >> Principal Software Engineer >> >> Splunk Inc. >> >> jvanligh...@splunk.com <mailto:jvanligh...@splunk.com> < >> kramas...@splunk.com <mailto:kramas...@splunk.com>> >> >> Barcelona >> >> > > -- > Anup Ghatage > www.ghatage.com > -- Jvrao --- First they ignore you, then they laugh at you, then they fight you, then you win. - Mahatma Gandhi
