I like the idea of updating the characterization of the project. It makes sense that the project shapes up over time according to use cases, and the description of years ago is not necessarily a good fit. I'm not sure if you are looking for suggestions for the text or just feedback on what BK has become. My only suggestion in addition to what everyone has said for the description itself is to make sure that it starts with a single crispy sentence of what BK is, like you suggested, something along the lines of append-only, fault tolerant, scalable, and low latency storage. I often find myself wanting some sentence that describes succinctly a project, in general, not specifically BK. We can then complement it with layers of more detail.
With respect to WAL vs. not only WAL, the problem I have faced over time with this is that the various ways of using an append-only abstraction are fairly nuanced. Logging is an overloaded term and qualifiers like "change" or "write-ahead" or "transaction" are not necessarily perceived in the same way, so I'm in favor of moving away from WAL as a primary way of characterizing BK to avoid any confusion around capabilities it provides. I think it has been more than 6-7 years depending on how you count. The first commit is from 2011, but we had been doing it for at least a couple of years in Yahoo!. I'm personally very happy to see how it developed and also happy to see all the effort to consolidate. It is great to see the community growing and thriving. -Flavio > On 04 Jul 2017, at 17:17, Sijie Guo <guosi...@gmail.com> wrote: > > +1 agree. > > On Jul 3, 2017 7:21 PM, "Venkateswara Rao Jujjuri" <jujj...@gmail.com> > wrote: > >> Everything said on this thread is important and accurate. The description >> on the website must be a story rather than a blurb. >> We should talk about BK's strengths as Enrico pointed out, and because of >> its versatility it became fundamental building block >> for various other technologies and usecases. IMO, the entire story is very >> powerful and appealing for BK. >> >> On Mon, Jul 3, 2017 at 7:55 AM, Sijie Guo <guosi...@gmail.com> wrote: >> >>> On Mon, Jul 3, 2017 at 1:35 AM, Enrico Olivelli <eolive...@gmail.com> >>> wrote: >>> >>>> 2017-07-03 7:00 GMT+02:00 Sijie Guo <guosi...@gmail.com>: >>>>> Hi all, >>>>> >>>>> It has been almost 6-7 years since Apache BookKeeper was born. Apache >>>>> BookKeeper has already grown beyond a WAL system. Both Twitter and >>> Yahoo >>>>> have used it as their storage foundation for their messaging systems, >>>>> Salesforce is using it for storage service. We also started talking >>>> Apache >>>>> BookKeeper as a storage service since 2016 ([1][2]). >>>>> >>>>> I am thinking of changing the description of Apache BookKeeper from a >>> WAL >>>>> system to "a High Performance and Low Latency Storage Service (that >>>>> optimized for immutable/append-only data)" in the new website that we >>> are >>>>> building for BP-11 >>>>> <https://cwiki.apache.org/confluence/pages/viewpage. >>>> action?pageId=71012301>. >>>>> This will help us to bring more use cases/adoptions to the project >> and >>>> help >>>>> grow the community. >>>>> >>>>> Any thoughts? >>>> >>>> My two cents >>>> >>>> Honestly when I found BookKeeper I was very happy because I found an >>>> "original" building block to build replicated state machines. >>>> I think that the main soul of BK is exactly to be a WAL and this is >>>> really "original". >>>> >>>> From my point of view the "key features" of BookKeeper are "Fencing" >>>> and "Last-Add-Confirmed protocol" >>>> >>>> BookKeeper is really good at storing data, but IMHO it is because it >>>> has been designed and implemented by very skilled engineers, >>>> BookKeeper needs to be "fast", because in order to provide a fast WAL >>>> you have to give an ultra-fast storage, because the essence of a WAL >>>> is "durability" and usually "durable" comes together with 'sync' and >>>> so with 'slow' . >>>> >>>> I am not a "marketing expert" but IMHO we should stress on the >>>> distinctive features of BK in respect to other softwares. >>>> >>>> I am not against the proposed change but as an user I wanted to point >>>> that I happy with BK because it is the most powerful distributed WAL >>>> (and maybe it is the unique in the opensource/free world) >>>> >>>> I would like to write in the website that BookKeeper is the real >>>> answer to whom who are looking for a distributed WAL. >>> >>> >>> Agree, we should make a clear case for distributed WAL. >>> >>> It is worth just putting down all the use cases that BookKeeper has >>> supported. >>> >>> - WAL (e.g. HDFS NameNode) >>> - Message Store (e.g. Apache Pulsar, Twitter Pub/Sub via DistributedLog) >>> - Offset/Cursor Store (e.g. Apache Pulsar stores cursors in ledgers) >>> - Object/Blob Store (e.g. in replicated state machine, storing state >>> machine snapshots in ledgers. We used this pattern at distributedlog >> based >>> replicated state machines.) >>> - ... >>> >>> They are not all typical WAL use cases. But the common thing on all these >>> use cases - they are using bookkeeper as an append-only/immutable store. >>> >>> - Sijie >>> >>> >>> >>> >>>> >>>> >>>> -- Enrico >>>> >>>> >>>> >>>>> >>>>> [1] >>>>> https://www.slideshare.net/hustlmsp/apache-bookkeeper-a- >>>> high-performance-and-low-latency-storage-service >>>>> [2] https://www.slideshare.net/jujjuri/apache-con2016final >>>> >>> >> >> >> >> -- >> Jvrao >> --- >> First they ignore you, then they laugh at you, then they fight you, then >> you win. - Mahatma Gandhi >>
