Leeds Meet-up: March 26th
Howdy, The first Leeds Riak meet-up of 2013 will be on the 26th of March at 18.15. This time we're holding it at the Round Foundry Media Centre, just behind the city centre train station. The main talk is from Martin Sumner, of the NHS's Connecting for Health team, and his talk is called, "Riak and the big NHS Computer System". There'll also be a Riak 1.3 overview, food and drinks. Sign up here: http://www.meetup.com/Leeds-Riak/events/106662212/ -- Matthew Revell Community Manager EMEA, Basho Proud makers of the Riak database T: +44 (0)20 3 298 6603 M: +44 (0)7970 377 071 ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
2i streaming
Hi everyone, Is there any way to request a large number of keys through 2i streaming? Say, there is index with 10M entries, I want to extract 1M of them. Obviously the block request (i.e. all data packed into the single response) is not a best idea since it requires a good amount of memory either on client and the server. One can suggest to feed 2i output into the Map/Reduce job with streaming output, but this way is not so hot either: it is really slow (our 3-node cluster stumbles on 100k keys for a minutes); and sometimes it just isn't working (streaming may stop occasionally before all data being kicked out). Not to mention that on 1M of keys Map/Reduce job just never starts. Is it possible to perform 2i queries for large number of keys, or shall I use another storage for indexing instead? (like Redis maybe) Thanks in advance. Pavel. ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: 2i streaming
We have done so far up to 3.5m keys listing for the same purpose using 2i search over protocol buffers and it seems to be fast enough. Maybe it is fast because it streams and compress the keys list directly into protocol buffer I/O stream without leaving a big foot print in memory? I don't know the answer to that question though. Using 2i listing has never failed for us where Map reduce identity on a 2i (for counting few millions keys based on 2i) have had a 50% chance to fail depending on the key's size/count, at least for us. We use Riak Java client, so that's also another concern, if you are using other programming language then you would wonder if the client uses 2i over PB. Hope that helps, Guido. On 04/03/13 13:18, Pavel Kirienko wrote: Hi everyone, Is there any way to request a large number of keys through 2i streaming? Say, there is index with 10M entries, I want to extract 1M of them. Obviously the block request (i.e. all data packed into the single response) is not a best idea since it requires a good amount of memory either on client and the server. One can suggest to feed 2i output into the Map/Reduce job with streaming output, but this way is not so hot either: it is really slow (our 3-node cluster stumbles on 100k keys for a minutes); and sometimes it just isn't working (streaming may stop occasionally before all data being kicked out). Not to mention that on 1M of keys Map/Reduce job just never starts. Is it possible to perform 2i queries for large number of keys, or shall I use another storage for indexing instead? (like Redis maybe) Thanks in advance. Pavel. ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Set the default search schema analyzer_factory to 'standard_analyzer_factory' for all future buckets
Is there anyway to set the default search schema analyzer_factory to 'standard_analyzer_factory' for all future buckets (i.e. without having to manually set schema each time a new bucket is created) ? Thank you, -- View this message in context: http://riak-users.197444.n3.nabble.com/Set-the-default-search-schema-analyzer-factory-to-standard-analyzer-factory-for-all-future-buckets-tp4027079.html Sent from the Riak Users mailing list archive at Nabble.com. ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: 2i range queries
Pavel, On first investigation, this does appear to be a bug. We'll be looking into it today. Reid On Mar 4, 2013, at 8:34 AM, Pavel Kirienko wrote: > Hello, > > There is the key that indexed with value 1362400142028, index is integer. > > I perform the query like that: > curl -v > http://:/buckets//index//0/1362400142028 > > Or like that: > curl -v > http://:/buckets//index//11000/1362400142028 > > Everything is fine. > > However, when I do that: > curl -v > http://:/buckets//index//1/1362400142028 > > I see no key in the output. Has 2i some limitations on integer range? > > Pavel. > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
RE: Riak Data-UI
I am, just haven't had time to teach myself erlang! Should be getting some training at a conference soon, then will probably give it a go. The "Introductory" tag helps though, thanks! -Chad -Original Message- From: Christopher Meiklejohn [mailto:cmeiklej...@basho.com] Sent: Friday, March 01, 2013 5:40 PM To: Chad Engler Cc: Christopher Meiklejohn; riak-users@lists.basho.com Subject: Re: Riak Data-UI Hey Chad, I just want to let you know that I'm opening a bunch of issues on Riak Control with the tag "Introductory", which are smaller items which need to be addressed if you're interested in contributing. - Chris -- Christopher Meiklejohn Software Engineer Basho Technologies, Inc. On Friday, February 15, 2013 at 9:50 AM, Chad Engler wrote: > Very cool! This is really helpful for getting a cold-start on > riak_control. I guess I will need to learn me some erlang, JavaScript > is more of my strong suit :Ps > > -Chad > > > -Original Message- > From: Christopher Meiklejohn [mailto:cmeiklej...@basho.com] > Sent: Thursday, February 14, 2013 1:31 PM > To: Chad Engler > Cc: Ian Ha; riak-users@lists.basho.com > (mailto:riak-users@lists.basho.com) > Subject: Re: Riak Data-UI > > Hi Chad, > > I've started sketching out contribution instructions and guidelines here: > > https://github.com/basho/riak_control/wiki/Contributing-to-Riak-Contro > l > > I'd appreciate any feedback on how this could be clearer, or more helpful. > > - Chris > > -- > Christopher Meiklejohn > Software Engineer > Basho Technologies, Inc. > > > > On Monday, February 11, 2013 at 2:17 PM, Chad Engler wrote: > > > Perfect! Thanks Chris, look forward to getting my hands dirty. > > > > -Chad > > > > > > -Original Message- > > From: Christopher Meiklejohn [mailto:cmeiklej...@basho.com] > > Sent: Monday, February 11, 2013 12:53 PM > > To: Chad Engler > > Cc: Ian Ha; riak-users@lists.basho.com > > (mailto:riak-users@lists.basho.com) > > Subject: Re: Riak Data-UI > > > > Hi Chad, > > > > First off, let me apologize for my delayed reply. The blizzard > > caused me to be away from my computer a bit longer than I would have > > liked to. ;) > > > > I'm in complete agreement with you regarding the roadmap and a document > > assisting people in putting together a development environment and > > contributing. > > > > I'm going to be working today to put together a contributors document on > > Riak Control and will post back to the list once it's been completed. In > > regards to the roadmap, once we decide which features will be targeted for > > the next release, I'll have issues created for each of these on the GitHub > > project, which will be publicly visible. > > > > Thanks for being part of the community, and do not hesitate to contact me > > (on list preferably, for the benefit of the community) if you have any > > further questions or feedback. > > > > - Chris > > > > -- > > Christopher Meiklejohn > > Software Engineer > > Basho Technologies, Inc. > > > > > > > > On Monday, February 11, 2013 at 9:27 AM, Chad Engler wrote: > > > > > Just bumping this, I think the most important part is if there is a > > > roadmap, or a way to get involved. Object browsing and disk/merge stats > > > are important features that would be really helpful and if there are docs > > > on how we can get started contributing to Riak Control, that would be > > > great. > > > > > > -Chad > > > > > > From: Ian Ha [mailto:i...@blutrumpet.com] > > > Sent: Thursday, February 07, 2013 10:09 AM > > > To: Chad Engler > > > Cc: Christopher Meiklejohn; riak-users@lists.basho.com > > > (mailto:riak-users@lists.basho.com) > > > (mailto:riak-users@lists.basho.com) > > > Subject: Re: Riak Data-UI > > > > > > Also: Is there an ETA on Riak Control? > > > > > > On Thu, Feb 7, 2013 at 10:08 AM, Ian Ha > > (mailto:i...@blutrumpet.com)> wrote: > > > +1. > > > > > > > > > Developing a comprehensive admin console would be great. > > > > > > > > > I would add merge statistics to Chad's feature requests. Would be nice to > > > see who how much disk space is saved during merges, which merge/threshold > > > triggers were hit, and possible disk usage at that time. > > > > > > > > > On Wed, Feb 6, 2013 at 2:21 PM, Chad Engler > > (mailto:chad.eng...@patlive.com)> wrote: > > > > > > > > > > Some of my favorite features of Futon were the ability to edit the > > > > configuration of the server from the web interface, check job status > > > > (compaction, view indexing, etc), and used almost all the time was the > > > > overview to see data size and number of documents: > > > > > > > > > > > > This little table was the best because I could immediately see who is > > > > using what, how many documents are there (many times to make sure the > > > > count is going up at a rate that it should be); and of course I could > > > > run map-reduces in there as well, but I un
Re: 2i range queries
I've created this [1] issue on riak_kv to track progress. The short story is the bug has been tracked down to `sext`, a library we use for binary encoding of erlang terms that (should) preserves sort order. [1] https://github.com/basho/riak_kv/issues/499 Reid On Mar 4, 2013, at 11:42 AM, Reid Draper wrote: > Pavel, > > On first investigation, this does appear to be a bug. We'll be looking into > it today. > > Reid > > On Mar 4, 2013, at 8:34 AM, Pavel Kirienko > wrote: > >> Hello, >> >> There is the key that indexed with value 1362400142028, index is integer. >> >> I perform the query like that: >> curl -v >> http://:/buckets//index//0/1362400142028 >> >> Or like that: >> curl -v >> http://:/buckets//index//11000/1362400142028 >> >> Everything is fine. >> >> However, when I do that: >> curl -v >> http://:/buckets//index//1/1362400142028 >> >> I see no key in the output. Has 2i some limitations on integer range? >> >> Pavel. >> ___ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Multiple writes to same key/value
Hi Sean, Another question, for the User object in the described scenario, would CRDT set still be required? Say, if an user clicked like and unlike multiple times in a short period of time, is it correct in saying that two riak nodes could potentially be updating the same User object in an non deterministic order ? Thanks Regards, Jimmy. On Sun, Mar 3, 2013 at 3:54 PM, Sean Cribbs wrote: > Hi Jimmy, > > Sorry for taking this off-list; I meant to reply-all on that message! > Anyway, we do have a work-in-progress CRDT integration and we hope to > have it production-ready later this year. > > On Sun, Mar 3, 2013 at 9:02 AM, Jimmy Ho wrote: > > Hi Sean, > > > > Thanks for the quick response. > > All makes good sense. > > > > Am using python, found this on git hub which seems to do what I need > (will > > need to test it further) > > > > https://github.com/ericmoritz/riak_crdt > > > > Any future plans to built CRDT features into Riak, or is it something > > believed to sit more at the client side to implement merging/conflict > > resolution? > > > > Thanks for the help. > > Regards, > > Jimmy > > > > On Sat, Mar 2, 2013 at 8:57 PM, Sean Cribbs wrote: > >> > >> Jimmy, > >> > >> Links are going to quickly become difficult to manage. > >> > >> This is the perfect problem for a CRDT, probably an OR-set. The nice > >> thing about CRDTs is that even when multiple writers are involved, > >> there's a single way to merge conflicts which always moves toward the > >> real value (monotonic). When someone clicks the "like button", you > >> would add their user ID to the set. (Assuming they can un-like, you > >> would remove them from the set when they do that.) You would probably > >> do something similar for the other direction, the user's list of post > >> -- but is easy to manage from the concurrent-writes standpoint, since > >> a user is probably the only one modifying that. If you never need to > >> "un-like", then the set only grows and is thus simpler to put > >> together. > >> > >> Unfortunately, the CRDT support built into Riak is somewhat limited > >> right now. You would need to implement the convergent set yourself, or > >> use an existing client-side library. Here are a few: > >> > >> * Knockbox (Clojure) https://github.com/reiddraper/knockbox > >> * Statebox (Erlang) https://github.com/mochi/statebox > >> > >> A quick search on Github will find you a lot of other options, too (in > >> various states of completion). > >> > >> On Sat, Mar 2, 2013 at 2:10 PM, Jimmy Ho wrote: > >> > Hi guys, > >> > > >> > Am currently designing the data schema for an application; I have a > >> > question > >> > hoping some of you could help. Thanks in advance. > >> > > >> > One of the functionalities is a 'like button' to posts > >> > > >> > Each post is stored as an object in Riak, eg /post/ > >> > > >> > To like a post, I am planning on creating a link from user object to > the > >> > post object, and the reverse direction, such that it would be easier > to > >> > link-walk/mapreduce. > >> > > >> > My question is, if a particular post is very very popular, and > everyone > >> > is > >> > pressing 'like'. Would the write requests (to the post's object > >> > header) > >> > conflict each other? > >> > > >> > Is the above a bad schema design from that perspective? Any > suggestion > >> > of a > >> > better design? > >> > > >> > Thanks. > >> > Regards, Jimmy > >> > > >> > ___ > >> > riak-users mailing list > >> > riak-users@lists.basho.com > >> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > >> > > >> > >> > >> > >> -- > >> Sean Cribbs > >> Software Engineer > >> Basho Technologies, Inc. > >> http://basho.com/ > > > > > > > > -- > Sean Cribbs > Software Engineer > Basho Technologies, Inc. > http://basho.com/ > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
