Just a quick question from a much less experienced developer... is Couchdb a bad fit? It looks interesting given its mobile and offline support, and automatic replication, but I don't pretend to understand the db world to the extent you all do. I just am intrigued with both Couch and Wave and wonder if they are a good fit for each other. I think a lot about use cases for Wave that are mobile in nature, which of course often comes with offline situations and async comm. Couch looks good for that, but from what I hear gives up a bit of speed vis a vis Mongo.
Pardon the intrusion, Shaun On Wed, Jan 26, 2011 at 5:47 AM, Paul Thomas <[email protected]> wrote: > Second. This would be useful. > > > > Please put constraints of protocol and WAIB, with regard to persistence, > rather than "requirements" as that can be assumptive. Also > differentiate actual software needs from other environmental needs that > might also be anticipated but might not necessarily turn out to be the > case. include the reasoning behind these. > > This would give us the broadest spectrum of solutions to pick from. > > --- On Tue, 25/1/11, Yuri Z <[email protected]> wrote: > > From: Yuri Z <[email protected]> > Subject: Re: Waves Persistence > To: [email protected] > Date: Tuesday, 25 January, 2011, 18:50 > > Interesting. I think in order to compare we need a "shopping list" with > requirements for DB. Then we can discuss the pros and cons. > Some requirements were already mentioned but would be nice if someone would > compile a complete list. > > 2011/1/25 Paul Thomas <[email protected]> > > > > > > > > > > > > > > > > > > > > > Here is my 5p. > > > > I support any > > number of persistence implementations, the more the merrier, so long > > as it doesn't disrupt WAIB development. > > > > I do > > however think that WAIB should support one default implementation > > that is hassle free but still scalable, an most of all fit the out of > > a box idea of WAIB. I think it is important not just to support the > > one that suits what one would ideally want for a personal project but > > fits WAIB mains objective, which is to get implementations out there > > so the technology is used and is useful. > > > > My primary > > motivation for saying this is to not only make the user life easy, > > but also the developers, and free up time, etc. > > > > I have by > > doubts about doing things from scratch even as just an example > > implementation. Why reinvent the wheel if that is not your thing? I > > have my doubts about MongoDB, even if new changes come in. I > > don't think the performance is much of plus really. I have my doubts > > about their commitment to durability. However I still support the > > option to use it or anything really. Same thing for Cassandra, I'm > > sure it is excellent but is it really justified or convenient? Not > > even Twitter can justify a change for tweets themselves. Sure that is > > not a green project, however, perhaps it is still a bit overkill for > > WAIB. > > > > I was looking around for different data stores, > > generally nosql, but not exclusively. I came across a > > relatively unknown one called Terrastore. I still not absolutely sure > > it is right for the job, but I thought I'd mention it becuase it > > impressed me just how hassle free it was and also some features that > > you may make things easier. Primarily it is consistent, elastic and > > easily scalable , written in Java, uses pure schemaless JSON. > > Accessed through HTTP or Java Client API, supports single cluster, > > really simple to set-up. It also has many other feature like active > > eventing, Atomic Update using using built in methods and custom > > implementations. It won't be as fast as Mongo but is will > > still be pretty fast. it is also has easy search implementation with > > ElasticSearch (great search engine built atop of Lucene). > > > > > > http://code.google.com/p/terrastore/ > > http://code.google.com/p/terrastore/wiki/Documentation > > > > I > > did want to ask specifically about that question about disk write > > eventing: > > > > > > > http://groups.google.com/group/terrastore-discussions/browse_thread/thread/44122d9d0a426eb9 > > > > So > > there is some interest in helping us out, provided it makes sense. He > > would recommend something else (I'm sure) should Terrasore not > > be the one. I'm sure you will find things that you like, and > > presisely due to this being at the stage it is (new but not totally > > unstable), I think they are willing to entertain feature requests > > should all the 1+s add up. > > > > Certainly I would be for something > > that simple to set-up and use. It is a good benchmark in that > > respect. > > > > Paul--- On Sat, 22/1/11, Tad Glines <[email protected]> wrote: > > > > From: Tad Glines <[email protected]> > > Subject: Re: Waves Persistence > > To: [email protected] > > Date: Saturday, 22 January, 2011, 17:44 > > > > According to this (http://wiki.apache.org/cassandra/Durability) > > Cassandra can be configured such that writes do not return until the > > write has been synched to the commit log. So it looks like Cassandra > > CAN provide the persistence guarantees that WiaB needs. > > > > -Tad > > > > On Sat, Jan 22, 2011 at 5:01 AM, Joseph Gentle <[email protected]> > wrote: > > > Does it tell you when your data is safely committed to disk? WIAB > > > requires this, and as far as I know cassandra doesn't provide such an > > > event. > > > > > > -J > > > > > > On Sat, Jan 22, 2011 at 3:08 AM, Tad Glines <[email protected]> > > wrote: > > >> Has anyone taken a look at Casandra? > > >> > > >> I've looked at it a little bit because there is a project called > > >> Lucandra that implements Lucene on top of Casandra. Casandra is a > > >> column order DB that provides configurable (at operation time) levels > > >> of persistence guarantees. The main options are (ONE, QUORUM, and ALL) > > >> where the operation doesn't return until the data has been persisted > > >> at one, a quorum or all Cassandra instances. > > >> > > >> -Tad > > >> > > >> On Thu, Jan 20, 2011 at 10:44 PM, Joseph Gentle <[email protected]> > > wrote: > > >>> Yeah - great to know. May as well just stick with mongodb then. > > >>> > > >>> Cheers! > > >>> > > >>> -J > > >>> > > >>> > > >>> On Fri, Jan 21, 2011 at 5:21 PM, Alex North <[email protected]> > wrote: > > >>>> Wow, great work. Thanks Anthony. > > >>>> > > >>>> On 21 January 2011 16:53, Anthony Watkins <[email protected]> > > wrote: > > >>>> > > >>>>> I've recently looked into Redis for a different project and I don't > > >>>>> think it will meet the needs of the wave server, mostly because of > > the > > >>>>> amount of memory it requires and its deficiencies with persistence. > > >>>>> > > >>>>> Here are some of the articles I came across during my > investigation: > > >>>>> > > >>>>> > > > http://blog.mjrusso.com/2010/10/17/redis-from-the-ground-up.html#heading_toc_j_9 > > >>>>> http://blog.kennejima.com/post/1226487020/thoughts-on-redis > > >>>>> > http://antirez.com/post/a-few-key-problems-in-redis-persistence.html > > >>>>> > > >>>>> All of the authors are fans of Redis, but come to the same overall > > >>>>> conclusion: "In short: the strength is the data model, and the > > >>>>> deficiency is the persistence." > > >>>>> > > >>>>> A few highlights from these articles: > > >>>>> > > >>>>> - "Redis requires that the whole dataset be loaded into main memory > > at > > >>>>> all times. (Redis Virtual Memory, which we’ll discuss later, > relaxes > > >>>>> this requirement, but still needs all keys to always be in > memory.). > > >>>>> Guaranteed in-memory access to most of the dataset is Redis' main > > >>>>> performance driver — and is also responsible for creating its main > > >>>>> limitations." > > >>>>> > > >>>>> - "The amount of RAM that Redis needs is proportional to the size > of > > >>>>> the dataset. Large datasets in Redis are going to be fast, but > > >>>>> expensive." > > >>>>> > > >>>>> - "Redis persistence is highly configurable but the implementation > > >>>>> makes extremely heavy use of I/O resources. Furthermore, most save > > >>>>> operations require additional memory to complete successfully, and, > > in > > >>>>> some cases, asynchronous saves can block the server for lengthy > > >>>>> periods of time." > > >>>>> > > >>>>> - "my recommended setup is: Use Redis for small datasets that don’t > > >>>>> grow fast (stay far less than 1GB). Have at least 2x memory than > the > > >>>>> dataset. Use default snapshotting and disable AOF." > > >>>>> > > >>>>> On a related note mongodb will have single server durability in > > >>>>> version 1.8, the release candidate of which is scheduled for > January > > >>>>> 28th. I imagine by the time we implement any other non file based > > >>>>> persistence solution for wave, mongodb will have satisfied its > > biggest > > >>>>> negative for use in wave. > > >>>>> > > >>>>> I am fully in favor of multiple persistence solutions being > available > > >>>>> to the community, so I am not trying to be discouraging. Just > > sharing > > >>>>> what I've recently encountered. > > >>>>> > > >>>>> R, > > >>>>> > > >>>>> Anthony > > >>>>> > > >>>>> On Thu, Jan 20, 2011 at 7:37 PM, Joseph Gentle <[email protected]> > > wrote: > > >>>>> > > > >>>>> > If we want to stick to nosql databases, redis might be worth a > > look. > > >>>>> > Redis supports transactions and it has a synchronous save > > operation. > > >>>>> > > > >>>>> > -J > > >>>>> > > > >>>>> > > > >>>>> > On Tue, Jan 11, 2011 at 1:59 PM, Michael MacFadden > > >>>>> > <[email protected]> wrote: > > >>>>> > > This has come up several times. Tad Glines and I have talked > > about > > >>>>> this a few times on the forum. I the idea would be to implement a > > SQL DB > > >>>>> persistence layer via JPA and/or apache JDO. Tad has a bit of a > > schema > > >>>>> design worked up, which I am sure he could re-post if necessary. I > > think we > > >>>>> were hoping the the file based implementation would stabilize a bit > > before > > >>>>> starting on the SQL DB implementation, since as Alex mentions they > > are > > >>>>> working out some core issues. Also we would likely step on each > > others > > >>>>> toes. > > >>>>> > > > > >>>>> > > ~Michael > > >>>>> > > > > >>>>> > > On Jan 10, 2011, at 5:17 PM, Alex North wrote: > > >>>>> > > > > >>>>> > >> Soren and I are currently working on file-based persistence. > The > > final > > >>>>> > >> hookup is currently held up behind bugs it exposed in other > > parts of > > >>>>> the > > >>>>> > >> system, which I'm fixing now. > > >>>>> > >> > > >>>>> > >> The filesystem based persistence demonstrates what would be > > required > > >>>>> of > > >>>>> > >> another datastore implementation. There's been some talk about > > hooking > > >>>>> up an > > >>>>> > >> SQL layer but I don't believe anyone's designing it yet. > > >>>>> > >> > > >>>>> > >> That talk may have been on the > [email protected], > > >>>>> which is > > >>>>> > >> where all future discussion should occur. > > >>>>> > >> > > >>>>> > >> Alex > > >>>>> > >> > > >>>>> > >> On 11 January 2011 02:20, faisalbhagat < > [email protected]> > > >>>>> wrote: > > >>>>> > >> > > >>>>> > >>> Is there anyone actively working on waves persistence using > > some db? > > >>>>> > >>> > > >>>>> > >>> -- > > >>>>> > >>> You received this message because you are subscribed to the > > Google > > >>>>> Groups > > >>>>> > >>> "Wave Protocol" group. > > >>>>> > >>> To post to this group, send email to > > [email protected]. > > >>>>> > >>> To unsubscribe from this group, send email to > > >>>>> > >>> [email protected]<wave-protocol%[email protected]> > <wave-protocol%[email protected]<wave-protocol%[email protected]> > > > > <wave-protocol%[email protected]<wave-protocol%[email protected]> > <wave-protocol%[email protected]<wave-protocol%[email protected]> > > > > > > > >>>>> <wave-protocol%[email protected]<wave-protocol%[email protected]> > <wave-protocol%[email protected]<wave-protocol%[email protected]> > > > > <wave-protocol%[email protected]<wave-protocol%[email protected]> > <wave-protocol%[email protected]<wave-protocol%[email protected]> > > > > > > > >>>>> > > > >>>>> > >>> . > > >>>>> > >>> For more options, visit this group at > > >>>>> > >>> http://groups.google.com/group/wave-protocol?hl=en. > > >>>>> > >>> > > >>>>> > >>> > > >>>>> > > > > >>>>> > > -- > > >>>>> > > You received this message because you are subscribed to the > > Google > > >>>>> Groups "Wave Protocol" group. > > >>>>> > > To post to this group, send email to > > [email protected]. > > >>>>> > > To unsubscribe from this group, send email to > > >>>>> [email protected]<wave-protocol%[email protected]> > <wave-protocol%[email protected]<wave-protocol%[email protected]> > > > > <wave-protocol%[email protected]<wave-protocol%[email protected]> > <wave-protocol%[email protected]<wave-protocol%[email protected]> > > > > > > > >>>>> . > > >>>>> > > For more options, visit this group at > > >>>>> http://groups.google.com/group/wave-protocol?hl=en. > > >>>>> > > > > >>>>> > > > > >>>>> > > >>>> > > >>> > > >> > > > > > > > > > > > > > > > > > >
