Hey Satya, The first hundred slow queries are almost certainly cold page cache plus JIT, not your HNSW parameters. Vectors and the graph live off heap in mmap'd files, so they get pulled from disk on first touch. 2M docs at 1024 dims float32 is 8.2 GB of raw vectors on its own, plus roughly docs x M x 2 x 4 bytes for the graph (around 250 MB at the default M=16). If the heap takes most of the RAM, every knn query does random disk reads until the OS cache fills. Two things fix it: size the box so vectors plus graph fit in free RAM outside the heap, and put a real knn query in firstSearcher and newSearcher so a searcher is warm before it serves traffic. Nothing else we ever changed moved that first minute as much.
The default config is not production ready at that size. The knobs that actually matter, in order: Segment count, more than M and beamWidth. knn runs per segment, so ten segments means ten graph traversals per query. Fewer and larger segments cut both the cold and the steady state latency. topK is the search beam, not a row limit. Measured on one of our small indexes: topK=5 missed the true nearest neighbour entirely, topK=100 returned it at rank 1 with a clear margin. People lower topK for speed and quietly destroy recall. Control the result count with rows instead. If you filter, use the knn preFilter local param (Solr 9.0+). Without it Solr takes the global top K first and applies fq after, and a selective filter can post filter that down to zero hits. This is also the strongest argument for keeping vectors in Solr: the filter runs inside the graph traversal, not after it. Sizing is mostly arithmetic. Bytes = docs x dims x 4 for the vectors, plus docs x M x 2 x 4 for the graph, and both want to be in page cache. 200M docs at 1024 dims is about 820 GB of raw vectors. That does not fit one node's page cache, so at that scale the conversation is shards, replicas and cheaper vectors (fewer dimensions, or byte encoding if your quality budget allows), not HNSW tuning. For calibration, our own production numbers are about 1.2M vectors across 37 indexes, largest single index 206K docs at 1024 dims, on a 128 GB 32 core box. We have not run 200M vectors in Solr, and anyone quoting you node counts for that number is guessing. On Qdrant for retrieval and Solr for the rest: it works, with two costs. The filters have to be duplicated on the Qdrant side, otherwise you hit the same post filtering problem plus an fq holding thousands of IDs. And the score fusion is harder than it looks. e5 class cosine is anisotropic: ours sits in a band of roughly 0.927 to 0.935, so blending raw cosine with BM25 leaves the vector leg effectively dead weight. Min max normalize both legs within the candidate set before you blend. That one change took our results from editorial articles at the top to the actual products, same model, same index, same day. For what it is worth, we do exactly this at Opensolr: BM25 and vector fused per document inside a custom Solr query parser, on managed Solr. These pages describe the mechanics and there is a live RAG demo you can run against a real index: https://opensolr.com/opensolr-platform-user-documentation/hybrid-search https://opensolr.com/opensolr-platform-user-documentation/ai-vector https://opensolr.com/opensolr-platform-user-documentation/search-tuning https://opensolr.com/rag-in-60-seconds Opensolr.com Your Path to AI Search <https://opensolr.com/> [email protected] https://opensolr.com <https://opensolr.com/> VAT: RO-35410526 > On 10 Sep 2026, at 09:35, Satya Nand <[email protected]> wrote: > > Hi Solr Community, > > We are evaluating Solr for vector search and would appreciate some guidance > on performance and sizing. > > We have ~2M vector documents, but are seeing *multiple-second latency for > the first ~100 queries*. After that, performance improves. we want to scale > it to 200 M documents in future. > > A few questions: > > - Is the default Solr vector-search configuration production-ready, or > does it typically require tuning? > - What are the key parameters/settings we should tune for HNSW/KNN > performance? > - How should we estimate the *CPU, memory, storage, and node > requirements* for a given vector count, dimensions, QPS, and target > latency? > - For larger scale, how does Solr vector search typically compare with a > dedicated vector DB such as *Qdrant* in terms of hardware requirements > and scalability? > - Would this architecture make sense: *Qdrant for vector retrieval → > return candidate IDs → Solr for existing business logic, filtering, and > ranking*, or is it better to keep the entire flow in Solr? > > Our preference is to keep vector search in Solr if it can meet the required > latency and scale without significantly higher infrastructure cost. > > Any recommendations or real-world experience would be very helpful. > > -- > Satya Nand Kanodia > Architect > Search > +91-8130189159
smime.p7s
Description: S/MIME cryptographic signature
