Re: Returning large resultset is slow and resource intensive

2022-03-08 Thread Michael Sokolov
Another approach for retrieving large result sets can work if you have a unique sort key. and don't mind retrieving your results sorted by this key. Then you can retrieve the results in batches using a cursor-style approach; request the top N sorted by the key. Then request the top N s.t. the key i

RE: Returning large resultset is slow and resource intensive

2022-03-08 Thread Uwe Schindler
Hi, > For our use case, we need to run queries which return the full > matched result set. In some cases, this result set can be large (50k+ > results out of 4 million total documents). > Perf test showed that just 4 threads running random queries returning 50k > results make Lucene utilize 100% C

Returning large resultset is slow and resource intensive

2022-03-08 Thread Alexander Lukyanchikov
Hello everyone, For our use case, we need to run queries which return the full matched result set. In some cases, this result set can be large (50k+ results out of 4 million total documents). Perf test showed that just 4 threads running random queries returning 50k results make Lucene utilize 100%