Thank you for your feedback, Flavio. On Fri, Jul 7, 2017 at 3:10 AM, Flavio Junqueira <f...@apache.org> wrote:
> I like the idea of updating the characterization of the project. It makes > sense that the project shapes up over time according to use cases, and the > description of years ago is not necessarily a good fit. I'm not sure if you > are looking for suggestions for the text or just feedback on what BK has > become. My only suggestion in addition to what everyone has said for the > description itself is to make sure that it starts with a single crispy > sentence of what BK is, like you suggested, something along the lines of > append-only, fault tolerant, scalable, and low latency storage. I often > find myself wanting some sentence that describes succinctly a project, in > general, not specifically BK. We can then complement it with layers of more > detail. I am more looking for what BK has become. The sentences can be reviewed when we are sending out the pull requests for BP-11. "append-only, fault tolerant, scalable, and low latency storage" sounds a good sentence to characterize what BK is. > > With respect to WAL vs. not only WAL, the problem I have faced over time > with this is that the various ways of using an append-only abstraction are > fairly nuanced. Logging is an overloaded term and qualifiers like "change" > or "write-ahead" or "transaction" are not necessarily perceived in the same > way, so I'm in favor of moving away from WAL as a primary way of > characterizing BK to avoid any confusion around capabilities it provides. > Yes. That's my feeling too. WAL isn't a good term to characterize the capabilities that BookKeeper provides. > > I think it has been more than 6-7 years depending on how you count. The > first commit is from 2011, but we had been doing it for at least a couple > of years in Yahoo!. I'm personally very happy to see how it developed and > also happy to see all the effort to consolidate. It is great to see the > community growing and thriving. > -Flavio > > > On 04 Jul 2017, at 17:17, Sijie Guo <guosi...@gmail.com> wrote: > > > > +1 agree. > > > > On Jul 3, 2017 7:21 PM, "Venkateswara Rao Jujjuri" <jujj...@gmail.com> > > wrote: > > > >> Everything said on this thread is important and accurate. The > description > >> on the website must be a story rather than a blurb. > >> We should talk about BK's strengths as Enrico pointed out, and because > of > >> its versatility it became fundamental building block > >> for various other technologies and usecases. IMO, the entire story is > very > >> powerful and appealing for BK. > >> > >> On Mon, Jul 3, 2017 at 7:55 AM, Sijie Guo <guosi...@gmail.com> wrote: > >> > >>> On Mon, Jul 3, 2017 at 1:35 AM, Enrico Olivelli <eolive...@gmail.com> > >>> wrote: > >>> > >>>> 2017-07-03 7:00 GMT+02:00 Sijie Guo <guosi...@gmail.com>: > >>>>> Hi all, > >>>>> > >>>>> It has been almost 6-7 years since Apache BookKeeper was born. Apache > >>>>> BookKeeper has already grown beyond a WAL system. Both Twitter and > >>> Yahoo > >>>>> have used it as their storage foundation for their messaging systems, > >>>>> Salesforce is using it for storage service. We also started talking > >>>> Apache > >>>>> BookKeeper as a storage service since 2016 ([1][2]). > >>>>> > >>>>> I am thinking of changing the description of Apache BookKeeper from a > >>> WAL > >>>>> system to "a High Performance and Low Latency Storage Service (that > >>>>> optimized for immutable/append-only data)" in the new website that we > >>> are > >>>>> building for BP-11 > >>>>> <https://cwiki.apache.org/confluence/pages/viewpage. > >>>> action?pageId=71012301>. > >>>>> This will help us to bring more use cases/adoptions to the project > >> and > >>>> help > >>>>> grow the community. > >>>>> > >>>>> Any thoughts? > >>>> > >>>> My two cents > >>>> > >>>> Honestly when I found BookKeeper I was very happy because I found an > >>>> "original" building block to build replicated state machines. > >>>> I think that the main soul of BK is exactly to be a WAL and this is > >>>> really "original". > >>>> > >>>> From my point of view the "key features" of BookKeeper are "Fencing" > >>>> and "Last-Add-Confirmed protocol" > >>>> > >>>> BookKeeper is really good at storing data, but IMHO it is because it > >>>> has been designed and implemented by very skilled engineers, > >>>> BookKeeper needs to be "fast", because in order to provide a fast WAL > >>>> you have to give an ultra-fast storage, because the essence of a WAL > >>>> is "durability" and usually "durable" comes together with 'sync' and > >>>> so with 'slow' . > >>>> > >>>> I am not a "marketing expert" but IMHO we should stress on the > >>>> distinctive features of BK in respect to other softwares. > >>>> > >>>> I am not against the proposed change but as an user I wanted to point > >>>> that I happy with BK because it is the most powerful distributed WAL > >>>> (and maybe it is the unique in the opensource/free world) > >>>> > >>>> I would like to write in the website that BookKeeper is the real > >>>> answer to whom who are looking for a distributed WAL. > >>> > >>> > >>> Agree, we should make a clear case for distributed WAL. > >>> > >>> It is worth just putting down all the use cases that BookKeeper has > >>> supported. > >>> > >>> - WAL (e.g. HDFS NameNode) > >>> - Message Store (e.g. Apache Pulsar, Twitter Pub/Sub via > DistributedLog) > >>> - Offset/Cursor Store (e.g. Apache Pulsar stores cursors in ledgers) > >>> - Object/Blob Store (e.g. in replicated state machine, storing state > >>> machine snapshots in ledgers. We used this pattern at distributedlog > >> based > >>> replicated state machines.) > >>> - ... > >>> > >>> They are not all typical WAL use cases. But the common thing on all > these > >>> use cases - they are using bookkeeper as an append-only/immutable > store. > >>> > >>> - Sijie > >>> > >>> > >>> > >>> > >>>> > >>>> > >>>> -- Enrico > >>>> > >>>> > >>>> > >>>>> > >>>>> [1] > >>>>> https://www.slideshare.net/hustlmsp/apache-bookkeeper-a- > >>>> high-performance-and-low-latency-storage-service > >>>>> [2] https://www.slideshare.net/jujjuri/apache-con2016final > >>>> > >>> > >> > >> > >> > >> -- > >> Jvrao > >> --- > >> First they ignore you, then they laugh at you, then they fight you, then > >> you win. - Mahatma Gandhi > >> > >
