Interesting. I think in order to compare we need a "shopping list" with requirements for DB. Then we can discuss the pros and cons. Some requirements were already mentioned but would be nice if someone would compile a complete list.
2011/1/25 Paul Thomas <[email protected]> > > > > > > > > > > Here is my 5p. > > I support any > number of persistence implementations, the more the merrier, so long > as it doesn't disrupt WAIB development. > > I do > however think that WAIB should support one default implementation > that is hassle free but still scalable, an most of all fit the out of > a box idea of WAIB. I think it is important not just to support the > one that suits what one would ideally want for a personal project but > fits WAIB mains objective, which is to get implementations out there > so the technology is used and is useful. > > My primary > motivation for saying this is to not only make the user life easy, > but also the developers, and free up time, etc. > > I have by > doubts about doing things from scratch even as just an example > implementation. Why reinvent the wheel if that is not your thing? I > have my doubts about MongoDB, even if new changes come in. I > don't think the performance is much of plus really. I have my doubts > about their commitment to durability. However I still support the > option to use it or anything really. Same thing for Cassandra, I'm > sure it is excellent but is it really justified or convenient? Not > even Twitter can justify a change for tweets themselves. Sure that is > not a green project, however, perhaps it is still a bit overkill for > WAIB. > > I was looking around for different data stores, > generally nosql, but not exclusively. I came across a > relatively unknown one called Terrastore. I still not absolutely sure > it is right for the job, but I thought I'd mention it becuase it > impressed me just how hassle free it was and also some features that > you may make things easier. Primarily it is consistent, elastic and > easily scalable , written in Java, uses pure schemaless JSON. > Accessed through HTTP or Java Client API, supports single cluster, > really simple to set-up. It also has many other feature like active > eventing, Atomic Update using using built in methods and custom > implementations. It won't be as fast as Mongo but is will > still be pretty fast. it is also has easy search implementation with > ElasticSearch (great search engine built atop of Lucene). > > > http://code.google.com/p/terrastore/ > http://code.google.com/p/terrastore/wiki/Documentation > > I > did want to ask specifically about that question about disk write > eventing: > > > http://groups.google.com/group/terrastore-discussions/browse_thread/thread/44122d9d0a426eb9 > > So > there is some interest in helping us out, provided it makes sense. He > would recommend something else (I'm sure) should Terrasore not > be the one. I'm sure you will find things that you like, and > presisely due to this being at the stage it is (new but not totally > unstable), I think they are willing to entertain feature requests > should all the 1+s add up. > > Certainly I would be for something > that simple to set-up and use. It is a good benchmark in that > respect. > > Paul--- On Sat, 22/1/11, Tad Glines <[email protected]> wrote: > > From: Tad Glines <[email protected]> > Subject: Re: Waves Persistence > To: [email protected] > Date: Saturday, 22 January, 2011, 17:44 > > According to this (http://wiki.apache.org/cassandra/Durability) > Cassandra can be configured such that writes do not return until the > write has been synched to the commit log. So it looks like Cassandra > CAN provide the persistence guarantees that WiaB needs. > > -Tad > > On Sat, Jan 22, 2011 at 5:01 AM, Joseph Gentle <[email protected]> wrote: > > Does it tell you when your data is safely committed to disk? WIAB > > requires this, and as far as I know cassandra doesn't provide such an > > event. > > > > -J > > > > On Sat, Jan 22, 2011 at 3:08 AM, Tad Glines <[email protected]> > wrote: > >> Has anyone taken a look at Casandra? > >> > >> I've looked at it a little bit because there is a project called > >> Lucandra that implements Lucene on top of Casandra. Casandra is a > >> column order DB that provides configurable (at operation time) levels > >> of persistence guarantees. The main options are (ONE, QUORUM, and ALL) > >> where the operation doesn't return until the data has been persisted > >> at one, a quorum or all Cassandra instances. > >> > >> -Tad > >> > >> On Thu, Jan 20, 2011 at 10:44 PM, Joseph Gentle <[email protected]> > wrote: > >>> Yeah - great to know. May as well just stick with mongodb then. > >>> > >>> Cheers! > >>> > >>> -J > >>> > >>> > >>> On Fri, Jan 21, 2011 at 5:21 PM, Alex North <[email protected]> wrote: > >>>> Wow, great work. Thanks Anthony. > >>>> > >>>> On 21 January 2011 16:53, Anthony Watkins <[email protected]> > wrote: > >>>> > >>>>> I've recently looked into Redis for a different project and I don't > >>>>> think it will meet the needs of the wave server, mostly because of > the > >>>>> amount of memory it requires and its deficiencies with persistence. > >>>>> > >>>>> Here are some of the articles I came across during my investigation: > >>>>> > >>>>> > http://blog.mjrusso.com/2010/10/17/redis-from-the-ground-up.html#heading_toc_j_9 > >>>>> http://blog.kennejima.com/post/1226487020/thoughts-on-redis > >>>>> http://antirez.com/post/a-few-key-problems-in-redis-persistence.html > >>>>> > >>>>> All of the authors are fans of Redis, but come to the same overall > >>>>> conclusion: "In short: the strength is the data model, and the > >>>>> deficiency is the persistence." > >>>>> > >>>>> A few highlights from these articles: > >>>>> > >>>>> - "Redis requires that the whole dataset be loaded into main memory > at > >>>>> all times. (Redis Virtual Memory, which we’ll discuss later, relaxes > >>>>> this requirement, but still needs all keys to always be in memory.). > >>>>> Guaranteed in-memory access to most of the dataset is Redis' main > >>>>> performance driver — and is also responsible for creating its main > >>>>> limitations." > >>>>> > >>>>> - "The amount of RAM that Redis needs is proportional to the size of > >>>>> the dataset. Large datasets in Redis are going to be fast, but > >>>>> expensive." > >>>>> > >>>>> - "Redis persistence is highly configurable but the implementation > >>>>> makes extremely heavy use of I/O resources. Furthermore, most save > >>>>> operations require additional memory to complete successfully, and, > in > >>>>> some cases, asynchronous saves can block the server for lengthy > >>>>> periods of time." > >>>>> > >>>>> - "my recommended setup is: Use Redis for small datasets that don’t > >>>>> grow fast (stay far less than 1GB). Have at least 2x memory than the > >>>>> dataset. Use default snapshotting and disable AOF." > >>>>> > >>>>> On a related note mongodb will have single server durability in > >>>>> version 1.8, the release candidate of which is scheduled for January > >>>>> 28th. I imagine by the time we implement any other non file based > >>>>> persistence solution for wave, mongodb will have satisfied its > biggest > >>>>> negative for use in wave. > >>>>> > >>>>> I am fully in favor of multiple persistence solutions being available > >>>>> to the community, so I am not trying to be discouraging. Just > sharing > >>>>> what I've recently encountered. > >>>>> > >>>>> R, > >>>>> > >>>>> Anthony > >>>>> > >>>>> On Thu, Jan 20, 2011 at 7:37 PM, Joseph Gentle <[email protected]> > wrote: > >>>>> > > >>>>> > If we want to stick to nosql databases, redis might be worth a > look. > >>>>> > Redis supports transactions and it has a synchronous save > operation. > >>>>> > > >>>>> > -J > >>>>> > > >>>>> > > >>>>> > On Tue, Jan 11, 2011 at 1:59 PM, Michael MacFadden > >>>>> > <[email protected]> wrote: > >>>>> > > This has come up several times. Tad Glines and I have talked > about > >>>>> this a few times on the forum. I the idea would be to implement a > SQL DB > >>>>> persistence layer via JPA and/or apache JDO. Tad has a bit of a > schema > >>>>> design worked up, which I am sure he could re-post if necessary. I > think we > >>>>> were hoping the the file based implementation would stabilize a bit > before > >>>>> starting on the SQL DB implementation, since as Alex mentions they > are > >>>>> working out some core issues. Also we would likely step on each > others > >>>>> toes. > >>>>> > > > >>>>> > > ~Michael > >>>>> > > > >>>>> > > On Jan 10, 2011, at 5:17 PM, Alex North wrote: > >>>>> > > > >>>>> > >> Soren and I are currently working on file-based persistence. The > final > >>>>> > >> hookup is currently held up behind bugs it exposed in other > parts of > >>>>> the > >>>>> > >> system, which I'm fixing now. > >>>>> > >> > >>>>> > >> The filesystem based persistence demonstrates what would be > required > >>>>> of > >>>>> > >> another datastore implementation. There's been some talk about > hooking > >>>>> up an > >>>>> > >> SQL layer but I don't believe anyone's designing it yet. > >>>>> > >> > >>>>> > >> That talk may have been on the [email protected], > >>>>> which is > >>>>> > >> where all future discussion should occur. > >>>>> > >> > >>>>> > >> Alex > >>>>> > >> > >>>>> > >> On 11 January 2011 02:20, faisalbhagat <[email protected]> > >>>>> wrote: > >>>>> > >> > >>>>> > >>> Is there anyone actively working on waves persistence using > some db? > >>>>> > >>> > >>>>> > >>> -- > >>>>> > >>> You received this message because you are subscribed to the > Google > >>>>> Groups > >>>>> > >>> "Wave Protocol" group. > >>>>> > >>> To post to this group, send email to > [email protected]. > >>>>> > >>> To unsubscribe from this group, send email to > >>>>> > >>> [email protected]<wave-protocol%[email protected]> > <wave-protocol%[email protected]<wave-protocol%[email protected]> > > > >>>>> <wave-protocol%[email protected]<wave-protocol%[email protected]> > <wave-protocol%[email protected]<wave-protocol%[email protected]> > > > >>>>> > > >>>>> > >>> . > >>>>> > >>> For more options, visit this group at > >>>>> > >>> http://groups.google.com/group/wave-protocol?hl=en. > >>>>> > >>> > >>>>> > >>> > >>>>> > > > >>>>> > > -- > >>>>> > > You received this message because you are subscribed to the > Google > >>>>> Groups "Wave Protocol" group. > >>>>> > > To post to this group, send email to > [email protected]. > >>>>> > > To unsubscribe from this group, send email to > >>>>> [email protected]<wave-protocol%[email protected]> > <wave-protocol%[email protected]<wave-protocol%[email protected]> > > > >>>>> . > >>>>> > > For more options, visit this group at > >>>>> http://groups.google.com/group/wave-protocol?hl=en. > >>>>> > > > >>>>> > > > >>>>> > >>>> > >>> > >> > > > > > > >
