Re: riaksearch performace when numFound is high

2011-07-15 Thread Greg Pascale
Ryan, Thanks for the detailed reply. We're a node.js shop and we're hitting riak via the riak-js library, which AFAIK just uses the http client. We haven't actually scaled out to the point where we're seeing huge #s of results returned, but if we extrapolate from the numbers we are seeing, it's

Re: riaksearch performace when numFound is high

2011-07-15 Thread Ryan Zezeski
Greg, I'm curious to know how you are querying search (i.e. which client/API) and what your setup looks like (i.e. # of nodes, disk, network between them, etc). How many of these #2 query types are you seeing and what's the average # of results being returned? Is your search instance opened up t

Re: riaksearch performace when numFound is high

2011-07-14 Thread Greg Pascale
Hi Ryan, Yes we are using 14.2. I think I get what you are saying about inline fields. It looks like it will fix some of our problems, but not all of them. If you'll indulge me in a little contrived example, I think I can explain what I mean. Let's say I'm implementing a simple twitter clone. In

Re: riaksearch performace when numFound is high

2011-07-13 Thread Ryan Zezeski
Greg, I'm assuming you are using 14.2. 1) There is a bug in 14.2 that will cause a (potentially very fast growing) memory leak when using AND. This is unfortunate, sorry. The good news I have since patched it [1]. 2) This is your best course of action, and you were so close but you've actually