Riak 0.14 brings key filters - it's still going to take time to filter the keys in memory, but it's an in memory operation. Using 'smart keys' along the lines of UNIXTIMESTAMP:placement:campaign:customer you can rapidly filter your keys using meaningful criteria and perform MapReduce jobs on the results.
Nothing says you can't also store the same data in multiple buckets in multiple formats to make querying easier. In response to number 2 - there's a way to set Riak to auto expire data from a bucket. It'll only be removed when compactions occur, but if you're storing clickstream data that should be happen often enough. -- Jeremiah Peschka Microsoft SQL Server MVP MCITP: Database Developer, DBA On Thursday, February 10, 2011 at 9:35 AM, Mat Ellis wrote: > We are converting a mysql based schema to Riak using Ripple. We're tracking a > lot of clicks, and each click belongs to a cascade of other objects: > > click -> placement -> campaign -> customer > > i.e. we do a lot of operations on these clicks grouped by placement or sets > of placements. > > Reading this > http://lists.basho.com/pipermail/riak-users_lists.basho.com/2010-July/001591.html > gave me pause for thought. I was hoping the time needed to crunch each day's > data would be proportional to the volume of clicks on that day but it seems > that it would be proportional to the total number of clicks ever. > > What's the best approach here? I can see a number of 'solutions' each of them > complicated: > > (1) Maintain an index of clicks by day so that we can focus our operations on > a time bound set of clicks > > (2) Delete or archive clicks once they have been processed or after a certain > number of days > > (3) Add many links to each placement, one per click (millions potentially) > > On a related noob-note, what would be the best way of creating a set of the > clicks for a given placement? Map Reduce or Riak Search or some other method? > > Thanks in advance. > > M. > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
