HTTP-based client for Riak database, written in Dart
Hi, I'm working on a Dart-based project that uses Riak database as a backend, and I'm releasing the related riak-dart client codes: http://code.google.com/p/riak-dart/ The project covers the basic use-cases of Riak: * fetch object * store object * with easy conditional updates * secondary index support + helpers * delete object (with easy conditional delete) * query secondary index Contributions are welcome, especially in the following areas: * testing, logging, documentation, examples * query secondary index with large result set (streaming processing) * conditional fetch object (etag, not-modified-since support) * Riak Protobuf API support * Riak Map-Reduce API support Regards, Istvan ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
pub/sub messaging e.g. worker queue on riak?
Hi, I'm playing with the idea how Riak could be used to build a pub/sub messaging on it. I know that there are specialized products for that, I just find it an interesting problem to solve it with Riak only (+ not too much client code). The example scenario is the following: - N producers create e.g. "task items", store it in Riak - M workers/consumers try to complete them and after completion delete the items from Riak - workers choose from "one of the oldest" task items, e.g. there is a close-to-fair scheduling (but no absolute ordering) - workers try to not work on the same task In the first iteration, I'd use: - secondary index with the actual timestamp to provide ordering - the workers could use (hash modulo M) decision to work on a given task - the workers periodically re-sync their status (e.g. number of active workers) through a different bucket I'm not entirely convinced that this is a scalable design, but it seems good on the design board. Does anyone have a working experience with similar designs? Any hint that x/y/z could go wrong? Thanks, Istvan ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: pub/sub messaging e.g. worker queue on riak?
Hi Mark, The proposed design has only one major benefit: it reduces the number of different technologies to work with / support. I haven't implemented it myself (yet), but a friend of mine might do soon, I'll try to get the results published. Regards, Istvan On Wed, Nov 28, 2012 at 12:33 PM, Mark Phillips wrote: > Hi Istvan, > > I've not yet seen Riak hacked into a queue, and I suspect it's for > good reason. Your design sounds like a lot of work just to emulate the > functionality that something that like Kafka or RabbitMQ can do out of > the box (though I have to admit the "Riak as a Queue" headline on > HackerNews would be quite nice). > > Out of curiosity, any chance you made a pass at it over the last few weeks? > > Mark > twitter.com/pharkmillups > > > On Wed, Nov 7, 2012 at 11:58 PM, Istvan Soos wrote: >> Hi, >> >> I'm playing with the idea how Riak could be used to build a pub/sub >> messaging on it. I know that there are specialized products for that, >> I just find it an interesting problem to solve it with Riak only (+ >> not too much client code). >> >> The example scenario is the following: >> - N producers create e.g. "task items", store it in Riak >> - M workers/consumers try to complete them and after completion delete >> the items from Riak >> - workers choose from "one of the oldest" task items, e.g. there is a >> close-to-fair scheduling (but no absolute ordering) >> - workers try to not work on the same task >> >> In the first iteration, I'd use: >> - secondary index with the actual timestamp to provide ordering >> - the workers could use (hash modulo M) decision to work on a given task >> - the workers periodically re-sync their status (e.g. number of active >> workers) through a different bucket >> >> I'm not entirely convinced that this is a scalable design, but it >> seems good on the design board. Does anyone have a working experience >> with similar designs? Any hint that x/y/z could go wrong? >> >> Thanks, >> Istvan >> >> ___ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
riak as drop-in replacement for DynamoDB?
I'm wondering if someone tried or is working on a Riak wrapper that is a drop-in replacement for AWS DynamoDB? It seems that it could work with the combination of the next-gen search. Any thoughts? Istvan ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Change Capture out of Riak
> Is anyone doing Change Capture (like the Databus project from LinkedIn) > directly out of Riak? Right now I have something hacked up using > Ripple+ActiveModel::Dirty, but I'd like to divorce it from our app > completely if possible. Was thinking a post-commit hook might work? End > goal is to feed it to a logging backend (in the Kafka sense of the word) > for other systems to feed off of. > > I'd love any pointers you might have to past posts I've missed (tried > searching the archive but came up dry), code, ideas, "tried it and it was a > pain", "that's a stupid idea", etc. I was thinking about a similar idea, but I haven't followed it through yet. It is certainly harder than it looks, especially on the question of how to handle backpressure or what kind of guarantees are you able to satisfy. A queue semantics might work for my use case, where the event broker app receives events, stores them in Riak and passes them on to other clients or brokers. This could work if none of them is interested in event ordering or past events (e.g. if events are immutable change events, there is a client-side conflict resolution, and there is a separate place to store the latest known/accepted state for each object). It would be interesting to hear what others have achieved. Istvan ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
