Hi Roger, Sorry, meant to reply to the mailing list, but accidentally replied directly to you.
Before I can say whether or not secondary indexes are suitable, I need to know more about your data, access patterns and query patterns. Can you share this information with me here? What kind of data is being searched and how frequently is it updated? Best, Todd On Wed, Nov 20, 2013 at 5:05 PM, Roger Diller <ro...@flexrentalsolutions.com > wrote: > Do you have any other suggestions on how we can find data in real time > from a bucket (without moving to 2.0)? What about secondary indexes? > > > On Wed, Nov 20, 2013 at 11:59 AM, Todd Tyree <t...@basho.com> wrote: > >> Hi Roger, >> >> You are essentially correct in that map reduce was never designed as a >> realtime query tool. >> >> However, we do have a solution in a technology preview release stage that >> may solve this problem for you: Yokozuna [0]. It is a tight integration of >> Riak and Solr and brings the best of both technologies together. >> >> It is currently scheduled for release as part of Riak 2.0, but you can >> clone the repo and build it now if you would like. Just be aware it is >> still undergoing development and the API may be subject to change before >> the final release. >> >> [0] https://github.com/basho/yokozuna >> >> Best, >> Todd >> > On Wed, Nov 20, 2013 at 4:45 PM, Roger Diller <ro...@flexrentalsolutions.com > wrote: > I could dig up all our nitty gritty Riak details but I don't think that > will help really. > > The point I think is this: Using search map reduce is not a viable way to > do real time search queries. Especially ones that may have 2000+ plus > results each. Couple that with search requests coming in every few seconds > from 300+ customer app instances and you literally bring Riak to it's > knees. > > Not that Riak is the problem really, it's just we are using it in a way it > was not designed for. In essence, we are using Riak as a search engine for > our application data. Correct me if I'm wrong but Riak is more for storing > large amounts of KV data, but not really for finding that data in a search > sense. > > Am I missing something here? Is there a viable way for doing real time > search queries on a bucket with 1 million keys? > > > On Mon, Nov 18, 2013 at 5:29 PM, Alexander Sicular <sicul...@gmail.com>wrote: > >> More info please... >> >> Version >> Current config >> Hardware >> Data size >> Search Schema >> Etc. >> >> But I would probably say that your search is returning too many keys to >> your mr. More inline. >> >> @siculars >> http://siculars.posthaven.com >> >> Sent from my iRotaryPhone >> >> On Nov 18, 2013, at 13:59, Roger Diller <ro...@flexrentalsolutions.com> >> wrote: >> >> Using the Riak Java client, I am executing a search map reduce like this: >> >> MapReduceResult result = riakClient.mapReduce(SEARCH_BUCKET, >> search).execute(); >> >> >> ^is this part a typo. Cause otherwise it looks like you do a s>mr, set >> the search and then another s>mr. >> >> >> String search = "systemId:" + systemName + " AND indexId:" + indexId; >> >> MapReduceResult result = riakClient.mapReduce(SEARCH_BUCKET, >> search).execute(); >> >> This worked fine when the bucket contained a few thousand keys. Now that >> we have far more data stored in the bucket (at least 250K keys), it's >> throwing this generic error: >> >> com.basho.riak.client.RiakException: java.io.IOException: >> {"error":"map_reduce_error"} >> >> We've also noticed that storing new key/values in the bucket has slowed >> WAY down. >> >> Any idea what's going on? >> >> >> Your data set is incorrectly sized to your production config. >> >> Are there limitations to Search Map Reduce? >> >> >> Certainly >> >> Are there configuration options that need changed? >> >> >> Possibly >> >> Any help would be greatly appreciated. >> >> >> -- >> Roger Diller >> Flex Rental Solutions, LLC >> Email: ro...@flexrentalsolutions.com >> Skype: rogerdiller >> Time Zone: Eastern Time >> >> _______________________________________________ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> > > > -- > Roger Diller > Flex Rental Solutions, LLC > Email: ro...@flexrentalsolutions.com > Skype: rogerdiller > Time Zone: Eastern Time > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > -- *Todd Tyree* Client Services Engineer Basho <http://www.basho.com/> mobile: +44(0)7861 220 182 web: www.basho.com github: tatyree <http://github.com/tatyree>
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
