I have always paired Riak with Redis for exactly this issue. Redis provides the data structures that Riak lacks. Pagination schemes via Redis are fairly trivial. To obviate the SPOF issue you should replicate Redis but what I do in addition is persist Redis keys to Riak on some frequency. Of course, YMMV based on your own specific needs.
-Alexander Sicular @siculars On Jul 19, 2012, at 10:33 AM, Alex Thompson wrote: > I agree, but sometimes you make sacrifices. ;) > > Alternatively, you could use something like redis or memcache to hold > keylists, either populated dynamically with key lists, or loaded with data on > startup. You might even be able to get away with just some reasonably complex > send-current-and-refresh style page caching and a script to warm your caches > from a cold start. > > Riak is a wonderful, scalable database, but unless you're holding lots data > in ram _somewhere_, your service is never really going to run at interactive > speeds. This is true of all data systems, not just riak, consider highly > random access to a traditional RDBMS with data > ram and no indexes. > > - Alex > > ----- Original Message ----- > From: "Martin Stabenfeldt" <mar...@stabenfeldt.net> > To: riak-users@lists.basho.com > Sent: Thursday, July 19, 2012 3:15:25 AM > Subject: Re: Pagination with Ripple > > > > Hi, > > > We´re also in the hundreds of thousands scale. So loading everything is > unfortunately not possible. > Would be nice if one could settle for one database, instead of mixing > different types. Nice to only care about scaling and providing redundancy for > Riak, instead of Riak and PostgreSQL :-) > > > If anyone has implemented pagination using Ripple or other solutions with > Ruby, I´d love to see it! > > > > > Cheers, > > > Martin Stabenfeldt > > > > On Wednesday, 18 July 2012 at 16:24, Alex Thompson wrote: > > > > > > My 2c: > > > 1000s of subscribers per list isn't a problem. Just have a data object which > represents the subscriber list, store a list of subscriber ids in the > subscriber list object. Load the whole list every time, and sort/paginate in > your app code. > > > Unless you're planning on shuffling the lists a lot, that will be the most > direct way to implement it. > > > Caveats: > Unless you're planning on running into the object size limit with subscriber > ids or planning on being able to handle thousands of simultaneous > subscriptions to the same list, but don't want to handle conflict resolution. > > > My solution: > I ran into this same problem with my app, but with hundreds of thousands to > millions of uuids per list. I tried a couple of Riak based solutions with > different data formats, but I wasn't really satisfied with any of them. I > ended up just having a uuid to uuid lookup table in a postgres database to > handle internal item to item relations using the uuid datatype for efficient > indexing. > > > I'm not sure if there are any good support libraries for handling polyglot > consistency in ruby, or if you could blend database ActiveRecord and ripple > models in the same app (I mean, you probably can, right? You'd just have > objects inherit from different base classes? There are probably some > unhandled relation-chaining edge cases though... although ActiveRecord can be > magic sometimes.) > > > - Alex > > > ----- Original Message ----- > From: "Martin Stabenfeldt" < mar...@stabenfeldt.net > > To: riak-users@lists.basho.com > Sent: Wednesday, July 18, 2012 2:54:35 AM > Subject: Pagination with Ripple > > > > > Dear List, > > > > > I´m planning to use Riak as my primary DB for my Rails app. > > > > > The first issue I´ve encountered is pagination. > > > > > I got a SubscriberList which may contain 1000´s of Subscribers. Any > suggestion on how I can implement pagination? My plan was to use > will_paginate . > > > > > > > > > > > > > > > > > class Subscriber > include Ripple :: Document > > > > > # Belongs to a SubscriberList > one :subscriber_list > property :subscriber_list_key , String > property :email , String > property :referer , String > timestamps! > > > > > validates_presence_of :email > end > > > > > > > > > > > > > class SubscriberList > include Ripple :: Document > one :user > property :user_key , String > > > > > # Has_many Subscribers > many :subscribers , :using => :reference > # Used by User to find which SubscriberList he has > property :subscriber_key , String > property :title , String > property :deleted , Boolean > timestamps! > > > > > validates_presence_of :user > end > > > > > -- > Martin Stabenfeldt > Tlf: +47 93441707 > > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
