Hello All,
Thanks a lot for the replies.
So the conclusion i came is to use either /exporthandler or cursor for
fetching large number of documents so i am going to use that and will
try to run tests to see whether all works fine or not.
Thanks again for all the help.
Regards
Neha Gupta
On 29/04/2022 04:13, David Hastings wrote:
the 30+ million records I retrieved were always from a single standalone
solr node, and yes you can do that frequently and it doesnt have an impact
on the rest of the searches happening assuming you have enough memory to
deal with it. there is nothing wrong with requesting every one of your
documents as well as every single stored field. It simply does just work.
so like was stated before, just try it and make a run and see what
happens.
On Thu, Apr 28, 2022 at 7:39 PM Yonik Seeley <ysee...@gmail.com> wrote:
It depends ;-)
If you are directly querying a single Solr node, then the additional memory
usage is (max_results * 4) if not retrieving scores. It's just
a single int per document to keep track of the docids that matched the
query. Documents are "streamed" to the client... the
actual stored fields for each document are only loaded when needed to write
to the output stream. If one is retrieving scores as well,
then the memory usage is (max_results * 8) (4 bytes for the int id, 4 bytes
for the float score.)
However, if one is using distributed search, then the entire response *is*
aggregated in memory before sending back to the client.
So if you are using SolrCloud and wish to do big bulk operations like this,
target individual nodes with distrib=false.
-Yonik
On Wed, Apr 27, 2022 at 4:23 PM Neha Gupta <neha.gu...@uni-jena.de> wrote:
Dear Solr Community,
I would like to know what is the safe number of documents that can be
returned from a SOLR.
Just for information I will be firing queries from Java application to
SOLR using SOLRJ and would like to know how much maximum documents (i.e
maximum number of rows that i can request in the query) can be returned
safely from SOLR.
It would be great if you can please share your experience with regard to
the same.
Thanks and Regards
Neha Gupta
