Re: Solr search performance

2016-09-21 Thread sean mcevoy
Hi Fred, Thanks for the pointer! 'cursorMark' is a lot more performant alright, though apparently it doesn't suit our use case. I've written a loop function using OTP's httpc that reads each page, gets the cursorMark and repeats, and it returns all 147 pages with consistent times in the 40-60ms b

Re: Solr search performance

2016-09-19 Thread Fred Dushin
All great questions, Sean. A few things. First off, for result sets that are that large, you are probably going to want to use Solr cursor marks [1], which are supported in the current version of Solr we ship. Riak allows queries using cursor marks through the HTTP interface. At present, it

Solr search performance

2016-09-19 Thread sean mcevoy
Hi All, We have an index with ~548,000 entries, ~14,000 of which match one of our queries. We read these in a paginated search and the first page (of 100 hits) returns quickly in ~70ms. This response time seems to increase exponentially as we walk through the pages: the 4th page takes ~200ms, the