Hi Iram, Is the machine doing lots of IO? If the hnsw graphs are not entirely in memory, performance will be poor. What JVM? You may get some benefit from simd support in java 21. Can you use the latest quantisation changes in Lucene to reduce memory footprint of the hnsw graphs? That's a large topk, but I guess you need it?
Best regards Kent Fitch On Thu, 28 Mar 2024, 5:12 am Iram Tariq, <iram.ta...@northbaysolutions.net.invalid> wrote: > Hi All, > > I am using Dense vectors in SOLR and facing slowness in it. Each search is > taking 10-25 seconds. I want to reduce the time to 5 seconds (or less > ideally). > > Following configurations are being used. > > > 1. *SOLR Version:* 9.3.0 > 2. *Lucene Version:* 9.7.0 > 3. *Vector Dimensions*: 384 > 4. *Total Shards:* 5 > 5. *Number of Vectors (Per shard*): 43209158 > 6. *JVM for each Instance:* 35GB > 7. *TopK: *1000 (Getting 1000 from each shard) > 8. *Rows: *1000 > 9. *Vector Field Schema: *<fieldType name="knn_vector_384" > class="solr.DenseVectorField" hnswMaxConnections="20" > knnAlgorithm="hnsw" > vectorDimension="384" similarityFunction="cosine" hnswBeamWidth="40"/> > 10. *Stored*: False > 11. *WebServer:* Apache Tomcat > 12. *System Specs*: Linux ( CPU:64, RAM:488 GB, OS:Ubuntu 20.04.6 ) > > Any sort of help/clue will be appreciated. > > > > Regards, > > > Iram Tariq | Software Architect > > NorthBay > > Direct: +1 (902) 329-7329 > > iram.ta...@northbaysolutions.net > > www.northbaysolutions.com >
