Second. This would be useful.
Please put constraints of protocol and WAIB, with regard to persistence, rather than "requirements" as that can be assumptive. Also differentiate actual software needs from other environmental needs that might also be anticipated but might not necessarily turn out to be the case. include the reasoning behind these. This would give us the broadest spectrum of solutions to pick from. --- On Tue, 25/1/11, Yuri Z <[email protected]> wrote: From: Yuri Z <[email protected]> Subject: Re: Waves Persistence To: [email protected] Date: Tuesday, 25 January, 2011, 18:50 Interesting. I think in order to compare we need a "shopping list" with requirements for DB. Then we can discuss the pros and cons. Some requirements were already mentioned but would be nice if someone would compile a complete list. 2011/1/25 Paul Thomas <[email protected]> > > > > > > > > > > Here is my 5p. > > I support any > number of persistence implementations, the more the merrier, so long > as it doesn't disrupt WAIB development. > > I do > however think that WAIB should support one default implementation > that is hassle free but still scalable, an most of all fit the out of > a box idea of WAIB. I think it is important not just to support the > one that suits what one would ideally want for a personal project but > fits WAIB mains objective, which is to get implementations out there > so the technology is used and is useful. > > My primary > motivation for saying this is to not only make the user life easy, > but also the developers, and free up time, etc. > > I have by > doubts about doing things from scratch even as just an example > implementation. Why reinvent the wheel if that is not your thing? I > have my doubts about MongoDB, even if new changes come in. I > don't think the performance is much of plus really. I have my doubts > about their commitment to durability. However I still support the > option to use it or anything really. Same thing for Cassandra, I'm > sure it is excellent but is it really justified or convenient? Not > even Twitter can justify a change for tweets themselves. Sure that is > not a green project, however, perhaps it is still a bit overkill for > WAIB. > > I was looking around for different data stores, > generally nosql, but not exclusively. I came across a > relatively unknown one called Terrastore. I still not absolutely sure > it is right for the job, but I thought I'd mention it becuase it > impressed me just how hassle free it was and also some features that > you may make things easier. Primarily it is consistent, elastic and > easily scalable , written in Java, uses pure schemaless JSON. > Accessed through HTTP or Java Client API, supports single cluster, > really simple to set-up. It also has many other feature like active > eventing, Atomic Update using using built in methods and custom > implementations. It won't be as fast as Mongo but is will > still be pretty fast. it is also has easy search implementation with > ElasticSearch (great search engine built atop of Lucene). > > > http://code.google.com/p/terrastore/ > http://code.google.com/p/terrastore/wiki/Documentation > > I > did want to ask specifically about that question about disk write > eventing: > > > http://groups.google.com/group/terrastore-discussions/browse_thread/thread/44122d9d0a426eb9 > > So > there is some interest in helping us out, provided it makes sense. He > would recommend something else (I'm sure) should Terrasore not > be the one. I'm sure you will find things that you like, and > presisely due to this being at the stage it is (new but not totally > unstable), I think they are willing to entertain feature requests > should all the 1+s add up. > > Certainly I would be for something > that simple to set-up and use. It is a good benchmark in that > respect. > > Paul--- On Sat, 22/1/11, Tad Glines <[email protected]> wrote: > > From: Tad Glines <[email protected]> > Subject: Re: Waves Persistence > To: [email protected] > Date: Saturday, 22 January, 2011, 17:44 > > According to this (http://wiki.apache.org/cassandra/Durability) > Cassandra can be configured such that writes do not return until the > write has been synched to the commit log. So it looks like Cassandra > CAN provide the persistence guarantees that WiaB needs. > > -Tad > > On Sat, Jan 22, 2011 at 5:01 AM, Joseph Gentle <[email protected]> wrote: > > Does it tell you when your data is safely committed to disk? WIAB > > requires this, and as far as I know cassandra doesn't provide such an > > event. > > > > -J > > > > On Sat, Jan 22, 2011 at 3:08 AM, Tad Glines <[email protected]> > wrote: > >> Has anyone taken a look at Casandra? > >> > >> I've looked at it a little bit because there is a project called > >> Lucandra that implements Lucene on top of Casandra. Casandra is a > >> column order DB that provides configurable (at operation time) levels > >> of persistence guarantees. The main options are (ONE, QUORUM, and ALL) > >> where the operation doesn't return until the data has been persisted > >> at one, a quorum or all Cassandra instances. > >> > >> -Tad > >> > >> On Thu, Jan 20, 2011 at 10:44 PM, Joseph Gentle <[email protected]> > wrote: > >>> Yeah - great to know. May as well just stick with mongodb then. > >>> > >>> Cheers! > >>> > >>> -J > >>> > >>> > >>> On Fri, Jan 21, 2011 at 5:21 PM, Alex North <[email protected]> wrote: > >>>> Wow, great work. Thanks Anthony. > >>>> > >>>> On 21 January 2011 16:53, Anthony Watkins <[email protected]> > wrote: > >>>> > >>>>> I've recently looked into Redis for a different project and I don't > >>>>> think it will meet the needs of the wave server, mostly because of > the > >>>>> amount of memory it requires and its deficiencies with persistence. > >>>>> > >>>>> Here are some of the articles I came across during my investigation: > >>>>> > >>>>> > http://blog.mjrusso.com/2010/10/17/redis-from-the-ground-up.html#heading_toc_j_9 > >>>>> http://blog.kennejima.com/post/1226487020/thoughts-on-redis > >>>>> http://antirez.com/post/a-few-key-problems-in-redis-persistence.html > >>>>> > >>>>> All of the authors are fans of Redis, but come to the same overall > >>>>> conclusion: "In short: the strength is the data model, and the > >>>>> deficiency is the persistence." > >>>>> > >>>>> A few highlights from these articles: > >>>>> > >>>>> - "Redis requires that the whole dataset be loaded into main memory > at > >>>>> all times. (Redis Virtual Memory, which we’ll discuss later, relaxes > >>>>> this requirement, but still needs all keys to always be in memory.). > >>>>> Guaranteed in-memory access to most of the dataset is Redis' main > >>>>> performance driver — and is also responsible for creating its main > >>>>> limitations." > >>>>> > >>>>> - "The amount of RAM that Redis needs is proportional to the size of > >>>>> the dataset. Large datasets in Redis are going to be fast, but > >>>>> expensive." > >>>>> > >>>>> - "Redis persistence is highly configurable but the implementation > >>>>> makes extremely heavy use of I/O resources. Furthermore, most save > >>>>> operations require additional memory to complete successfully, and, > in > >>>>> some cases, asynchronous saves can block the server for lengthy > >>>>> periods of time." > >>>>> > >>>>> - "my recommended setup is: Use Redis for small datasets that don’t > >>>>> grow fast (stay far less than 1GB). Have at least 2x memory than the > >>>>> dataset. Use default snapshotting and disable AOF." > >>>>> > >>>>> On a related note mongodb will have single server durability in > >>>>> version 1.8, the release candidate of which is scheduled for January > >>>>> 28th. I imagine by the time we implement any other non file based > >>>>> persistence solution for wave, mongodb will have satisfied its > biggest > >>>>> negative for use in wave. > >>>>> > >>>>> I am fully in favor of multiple persistence solutions being available > >>>>> to the community, so I am not trying to be discouraging. Just > sharing > >>>>> what I've recently encountered. > >>>>> > >>>>> R, > >>>>> > >>>>> Anthony > >>>>> > >>>>> On Thu, Jan 20, 2011 at 7:37 PM, Joseph Gentle <[email protected]> > wrote: > >>>>> > > >>>>> > If we want to stick to nosql databases, redis might be worth a > look. > >>>>> > Redis supports transactions and it has a synchronous save > operation. > >>>>> > > >>>>> > -J > >>>>> > > >>>>> > > >>>>> > On Tue, Jan 11, 2011 at 1:59 PM, Michael MacFadden > >>>>> > <[email protected]> wrote: > >>>>> > > This has come up several times. Tad Glines and I have talked > about > >>>>> this a few times on the forum. I the idea would be to implement a > SQL DB > >>>>> persistence layer via JPA and/or apache JDO. Tad has a bit of a > schema > >>>>> design worked up, which I am sure he could re-post if necessary. I > think we > >>>>> were hoping the the file based implementation would stabilize a bit > before > >>>>> starting on the SQL DB implementation, since as Alex mentions they > are > >>>>> working out some core issues. Also we would likely step on each > others > >>>>> toes. > >>>>> > > > >>>>> > > ~Michael > >>>>> > > > >>>>> > > On Jan 10, 2011, at 5:17 PM, Alex North wrote: > >>>>> > > > >>>>> > >> Soren and I are currently working on file-based persistence. The > final > >>>>> > >> hookup is currently held up behind bugs it exposed in other > parts of > >>>>> the > >>>>> > >> system, which I'm fixing now. > >>>>> > >> > >>>>> > >> The filesystem based persistence demonstrates what would be > required > >>>>> of > >>>>> > >> another datastore implementation. There's been some talk about > hooking > >>>>> up an > >>>>> > >> SQL layer but I don't believe anyone's designing it yet. > >>>>> > >> > >>>>> > >> That talk may have been on the [email protected], > >>>>> which is > >>>>> > >> where all future discussion should occur. > >>>>> > >> > >>>>> > >> Alex > >>>>> > >> > >>>>> > >> On 11 January 2011 02:20, faisalbhagat <[email protected]> > >>>>> wrote: > >>>>> > >> > >>>>> > >>> Is there anyone actively working on waves persistence using > some db? > >>>>> > >>> > >>>>> > >>> -- > >>>>> > >>> You received this message because you are subscribed to the > Google > >>>>> Groups > >>>>> > >>> "Wave Protocol" group. > >>>>> > >>> To post to this group, send email to > [email protected]. > >>>>> > >>> To unsubscribe from this group, send email to > >>>>> > >>> [email protected]<wave-protocol%[email protected]> > <wave-protocol%[email protected]<wave-protocol%[email protected]> > > > >>>>> <wave-protocol%[email protected]<wave-protocol%[email protected]> > <wave-protocol%[email protected]<wave-protocol%[email protected]> > > > >>>>> > > >>>>> > >>> . > >>>>> > >>> For more options, visit this group at > >>>>> > >>> http://groups.google.com/group/wave-protocol?hl=en. > >>>>> > >>> > >>>>> > >>> > >>>>> > > > >>>>> > > -- > >>>>> > > You received this message because you are subscribed to the > Google > >>>>> Groups "Wave Protocol" group. > >>>>> > > To post to this group, send email to > [email protected]. > >>>>> > > To unsubscribe from this group, send email to > >>>>> [email protected]<wave-protocol%[email protected]> > <wave-protocol%[email protected]<wave-protocol%[email protected]> > > > >>>>> . > >>>>> > > For more options, visit this group at > >>>>> http://groups.google.com/group/wave-protocol?hl=en. > >>>>> > > > >>>>> > > > >>>>> > >>>> > >>> > >> > > > > > > >
