Re: A question on performance

2015-01-07 Thread Arvind Kalyan
Performance measurements must be made carefully. Have you performed any warmup? I recommend doing 10k calls just to let the dust settle including stuff like jit, before taking any kind if measurements. Also use mmapdirectory, if not already, to help with spikes in disk accesses. Also keep track

RE: A question on performance

2015-01-07 Thread Toke Eskildsen
rama44ster [rama44s...@gmail.com] wrote: [3 MUST clauses] > 50 = 16 ms > 75 = 52 ms > 90 = 121 ms > 95 = 262 ms > 99 = 76010 ms > 99.9 = 76037 ms > Is the latency expected to degrade when the number of docs is as high as > 480M? Try plotting response times as a function of hit count. My guess i

A question on performance

2015-01-07 Thread rama44ster
Hi, I have a lucene index which has close to 480M documents. And I ran around 1000 queries against the index. Each query is a boolean query with 3 different tokens. That is the query has 3 operands which MUST occur. Executing such 3 token queries gives the following latency percentiles. 50 = 16 ms