Re: riaksearch performance, row limit, sorting not necessary

2011-04-14 Thread Gordon Tillman
Hi Daniel, If you use search to provide (streaming) input to map/reduce then you can do additional processing in the M/R phases to condition and limit your results. For example you can do additional filtering in a map phase if necessary, as well as perhaps extracting some subset of the data th

Re: riaksearch performance, row limit, sorting not necessary

2011-04-14 Thread Daniel Rathbone
To be clear, I'm only talking about the solr interface. I'm wondering if my query time will remain fixed (since it's capped at rows=1000) as I add several million docs to the index. If I use my search as an input into Map/Reduce, won't my response time grow with my index? My search query would qu

Re: riaksearch performance, row limit, sorting not necessary

2011-04-14 Thread Gordon Tillman
Daniel the max_search_results only applies to searches done via the solr interface. From http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-January/002974.html: - System now aborts queries that would queue up too many documents in a result set. This is controlled by a 'max_searc

riaksearch performance, row limit, sorting not necessary

2011-04-14 Thread Daniel Rathbone
Hi list, I'm wondering how riaksearch performance will degrade as I add documents. For my purpose I limit rows at 1k and sorting is not necessary. I have a single node cluster for development. I know I can increase performance if I add nodes but I'd like to understand this before I do. My docu