👍 On Sun, Jul 9, 2017 at 2:07 AM, Sijie Guo <guosi...@gmail.com> wrote:
> Thank you for your feedback, Flavio. > > On Fri, Jul 7, 2017 at 3:10 AM, Flavio Junqueira <f...@apache.org> wrote: > > > I like the idea of updating the characterization of the project. It makes > > sense that the project shapes up over time according to use cases, and > the > > description of years ago is not necessarily a good fit. I'm not sure if > you > > are looking for suggestions for the text or just feedback on what BK has > > become. My only suggestion in addition to what everyone has said for the > > description itself is to make sure that it starts with a single crispy > > sentence of what BK is, like you suggested, something along the lines of > > append-only, fault tolerant, scalable, and low latency storage. I often > > find myself wanting some sentence that describes succinctly a project, in > > general, not specifically BK. We can then complement it with layers of > more > > detail. > > > I am more looking for what BK has become. The sentences can be reviewed > when we are sending out the pull requests for BP-11. "append-only, fault > tolerant, scalable, and low latency storage" sounds a good sentence to > characterize what BK is. > > > > > > > > With respect to WAL vs. not only WAL, the problem I have faced over time > > with this is that the various ways of using an append-only abstraction > are > > fairly nuanced. Logging is an overloaded term and qualifiers like > "change" > > or "write-ahead" or "transaction" are not necessarily perceived in the > same > > way, so I'm in favor of moving away from WAL as a primary way of > > characterizing BK to avoid any confusion around capabilities it provides. > > > > Yes. That's my feeling too. WAL isn't a good term to characterize the > capabilities that BookKeeper provides. > > > > > > I think it has been more than 6-7 years depending on how you count. The > > first commit is from 2011, but we had been doing it for at least a couple > > of years in Yahoo!. I'm personally very happy to see how it developed and > > also happy to see all the effort to consolidate. It is great to see the > > community growing and thriving. > > > > -Flavio > > > > > On 04 Jul 2017, at 17:17, Sijie Guo <guosi...@gmail.com> wrote: > > > > > > +1 agree. > > > > > > On Jul 3, 2017 7:21 PM, "Venkateswara Rao Jujjuri" <jujj...@gmail.com> > > > wrote: > > > > > >> Everything said on this thread is important and accurate. The > > description > > >> on the website must be a story rather than a blurb. > > >> We should talk about BK's strengths as Enrico pointed out, and because > > of > > >> its versatility it became fundamental building block > > >> for various other technologies and usecases. IMO, the entire story is > > very > > >> powerful and appealing for BK. > > >> > > >> On Mon, Jul 3, 2017 at 7:55 AM, Sijie Guo <guosi...@gmail.com> wrote: > > >> > > >>> On Mon, Jul 3, 2017 at 1:35 AM, Enrico Olivelli <eolive...@gmail.com > > > > >>> wrote: > > >>> > > >>>> 2017-07-03 7:00 GMT+02:00 Sijie Guo <guosi...@gmail.com>: > > >>>>> Hi all, > > >>>>> > > >>>>> It has been almost 6-7 years since Apache BookKeeper was born. > Apache > > >>>>> BookKeeper has already grown beyond a WAL system. Both Twitter and > > >>> Yahoo > > >>>>> have used it as their storage foundation for their messaging > systems, > > >>>>> Salesforce is using it for storage service. We also started talking > > >>>> Apache > > >>>>> BookKeeper as a storage service since 2016 ([1][2]). > > >>>>> > > >>>>> I am thinking of changing the description of Apache BookKeeper > from a > > >>> WAL > > >>>>> system to "a High Performance and Low Latency Storage Service (that > > >>>>> optimized for immutable/append-only data)" in the new website that > we > > >>> are > > >>>>> building for BP-11 > > >>>>> <https://cwiki.apache.org/confluence/pages/viewpage. > > >>>> action?pageId=71012301>. > > >>>>> This will help us to bring more use cases/adoptions to the project > > >> and > > >>>> help > > >>>>> grow the community. > > >>>>> > > >>>>> Any thoughts? > > >>>> > > >>>> My two cents > > >>>> > > >>>> Honestly when I found BookKeeper I was very happy because I found an > > >>>> "original" building block to build replicated state machines. > > >>>> I think that the main soul of BK is exactly to be a WAL and this is > > >>>> really "original". > > >>>> > > >>>> From my point of view the "key features" of BookKeeper are "Fencing" > > >>>> and "Last-Add-Confirmed protocol" > > >>>> > > >>>> BookKeeper is really good at storing data, but IMHO it is because it > > >>>> has been designed and implemented by very skilled engineers, > > >>>> BookKeeper needs to be "fast", because in order to provide a fast > WAL > > >>>> you have to give an ultra-fast storage, because the essence of a WAL > > >>>> is "durability" and usually "durable" comes together with 'sync' > and > > >>>> so with 'slow' . > > >>>> > > >>>> I am not a "marketing expert" but IMHO we should stress on the > > >>>> distinctive features of BK in respect to other softwares. > > >>>> > > >>>> I am not against the proposed change but as an user I wanted to > point > > >>>> that I happy with BK because it is the most powerful distributed WAL > > >>>> (and maybe it is the unique in the opensource/free world) > > >>>> > > >>>> I would like to write in the website that BookKeeper is the real > > >>>> answer to whom who are looking for a distributed WAL. > > >>> > > >>> > > >>> Agree, we should make a clear case for distributed WAL. > > >>> > > >>> It is worth just putting down all the use cases that BookKeeper has > > >>> supported. > > >>> > > >>> - WAL (e.g. HDFS NameNode) > > >>> - Message Store (e.g. Apache Pulsar, Twitter Pub/Sub via > > DistributedLog) > > >>> - Offset/Cursor Store (e.g. Apache Pulsar stores cursors in ledgers) > > >>> - Object/Blob Store (e.g. in replicated state machine, storing state > > >>> machine snapshots in ledgers. We used this pattern at distributedlog > > >> based > > >>> replicated state machines.) > > >>> - ... > > >>> > > >>> They are not all typical WAL use cases. But the common thing on all > > these > > >>> use cases - they are using bookkeeper as an append-only/immutable > > store. > > >>> > > >>> - Sijie > > >>> > > >>> > > >>> > > >>> > > >>>> > > >>>> > > >>>> -- Enrico > > >>>> > > >>>> > > >>>> > > >>>>> > > >>>>> [1] > > >>>>> https://www.slideshare.net/hustlmsp/apache-bookkeeper-a- > > >>>> high-performance-and-low-latency-storage-service > > >>>>> [2] https://www.slideshare.net/jujjuri/apache-con2016final > > >>>> > > >>> > > >> > > >> > > >> > > >> -- > > >> Jvrao > > >> --- > > >> First they ignore you, then they laugh at you, then they fight you, > then > > >> you win. - Mahatma Gandhi > > >> > > > > >
