Re: CPU usage 100% during search

2017-01-03 Thread Rajnish kamboj
@Evert We have 16 GB total RAM, out of which Heap is using 4 GB. Initially we use RAMDirectory, but later switched to MMapDirectory since RAMDirectory is not meant for production use. @Adrien Might be you are right as we are running our system under load test so we are firing 200 requests with JMe

Re: CPU usage 100% during search

2017-01-03 Thread Adrien Grand
Are you running search requests in a loop from more than 4 threads? If yes then this situation is ideal like Trejkaz said. Otherwise please describe your workload and explain why you think 100% CPU usage is not ideal. Le mar. 3 janv. 2017 à 12:52, Rajnish kamboj a écrit : > High CPU may be ideal

Re: CPU usage 100% during search

2017-01-03 Thread Evert Wagenaar
4 Gb RAM for 4 CPU's may be too small. Try to increase the RAM to 16 Gb and load the Index in RAMDisk. That may help. Op di 3 jan. 2017 om 12:52 schreef Rajnish kamboj > High CPU may be ideal, but 100% CPU utilization is not ideal. (We might be > > missing very silly thing) > > > > More inform

Re: CPU usage 100% during search

2017-01-03 Thread Rajnish kamboj
High CPU may be ideal, but 100% CPU utilization is not ideal. (We might be missing very silly thing) More information on our environment: Machine details: 4 vCPUs and 4 GB RAM Search Load: ~200 simultaneous users searching requests. Data set : 3.5 Million records (~ 2 GB index size). We reduced o

Re: Lucene performance benchmark | search throughput

2017-01-03 Thread Michael McCandless
When you add MUST sub-clauses to a BooleanQuery (AND to the query parsers) it can make the search run faster because Lucene will take the most restrictive clause and use that to "drive" the iteration of matching documents to the other clauses, allowing those other clauses to iterate much faster th

Re: Lucene performance benchmark | search throughput

2017-01-03 Thread Rajnish kamboj
The answer is not clear. Suppose I have following query and I want 10 records. Condition1 AND Condition2 AND Condition3 As per my understanding Lucene will first evaluate all conditions separately and then merge the Documents as per AND/OR clauses. At last it will return me 10 records. So, if I

Re: Lucene performance benchmark | search throughput

2017-01-03 Thread Michael Wilkowski
My guess: more conditions = less documents to score and sort to return. On Mon, Jan 2, 2017 at 7:23 PM, Rajnish kamboj wrote: > Hi > > Is there any Lucene performance benchmark against certain set of data? > [i.e Is there any stats for search throughput which Lucene can provide for > a certain d