Hi Ciprian,
Thanks for the detailed inputs. I want to clarify a few things and get your
advice on a few points:
1. We have three vector fields of 2048 dimensions in a single index. Since
Solr doesn't natively support bfloat16, is there anything we can do to
reduce memory and computation overhead? How significant is the expected
quality drop?
2. Beside vectors, we currently store other textual fields with
stored=true so we can update vectors and text fields
independently(atomically). This is taking up considerable index size. Is
this standard practice, or is there a better approach for managing this?
3. The preFilter feature looks promising, but I'm struggling with the
exact syntax. We need to search across all three vector fields using a
boolean parser. Will implicit pre-filtering work in this setup?
payload = [
("q", "{!bool should=$imgb should=$textb should=$multib}"),
("defType", "lucene"),
("fl", "displayid,title,score"),
("rows", ROWS),
("wt", "json"),
("*route*", "gshard123"),
("imgq", "{!knn f=img_dot_emb topK=$topK v=$vector_str}"),
("imgb", "{!boost b=%s v=$imgq}" % img_w),
("textq", "{!knn f=text_dot_emb topK=$topK v=$vector_str}"),
("textb", "{!boost b=%s v=$textq}" % text_w),
("multiq", "{!knn f=multimodal_dot_emb topK=$topK v=$vector_str}"),
("multib", "{!boost b=%s v=$multiq}" % multi_w),
("vector_str", vector_str),
("fq", "has_img_dot_emb:true"),
("fq", "has_text_dot_emb:true"),
("fq", "has_multimodal_dot_emb:true"),
("topK", TOP_K),
]
4. Regarding your point on "Min max normalize both legs within the
candidate set before you blend": could you share more details ? We face a
similar issue where lexical boosts completely overrun vector scores when
merging for final ranking.
Satya Nand Kanodia
Architect
Search
+91-8130189159
On Thu, Sep 10, 2026 at 12:20 PM Ciprian Dimofte - Opensolr.com via users <
[email protected]> wrote:
> Hey Satya,
>
> The first hundred slow queries are almost certainly cold page cache plus
> JIT, not your HNSW parameters. Vectors and the graph live off heap in
> mmap'd files, so they get pulled from disk on first touch. 2M docs at 1024
> dims float32 is 8.2 GB of raw vectors on its own, plus roughly docs x M x 2
> x 4 bytes for the graph (around 250 MB at the default M=16). If the heap
> takes most of the RAM, every knn query does random disk reads until the OS
> cache fills. Two things fix it: size the box so vectors plus graph fit in
> free RAM outside the heap, and put a real knn query in firstSearcher and
> newSearcher so a searcher is warm before it serves traffic. Nothing else we
> ever changed moved that first minute as much.
>
> The default config is not production ready at that size. The knobs that
> actually matter, in order:
>
> Segment count, more than M and beamWidth. knn runs per segment, so ten
> segments means ten graph traversals per query. Fewer and larger segments
> cut both the cold and the steady state latency.
>
> topK is the search beam, not a row limit. Measured on one of our small
> indexes: topK=5 missed the true nearest neighbour entirely, topK=100
> returned it at rank 1 with a clear margin. People lower topK for speed and
> quietly destroy recall. Control the result count with rows instead.
>
> If you filter, use the knn preFilter local param (Solr 9.0+). Without it
> Solr takes the global top K first and applies fq after, and a selective
> filter can post filter that down to zero hits. This is also the strongest
> argument for keeping vectors in Solr: the filter runs inside the graph
> traversal, not after it.
>
> Sizing is mostly arithmetic. Bytes = docs x dims x 4 for the vectors, plus
> docs x M x 2 x 4 for the graph, and both want to be in page cache. 200M
> docs at 1024 dims is about 820 GB of raw vectors. That does not fit one
> node's page cache, so at that scale the conversation is shards, replicas
> and cheaper vectors (fewer dimensions, or byte encoding if your quality
> budget allows), not HNSW tuning. For calibration, our own production
> numbers are about 1.2M vectors across 37 indexes, largest single index 206K
> docs at 1024 dims, on a 128 GB 32 core box. We have not run 200M vectors in
> Solr, and anyone quoting you node counts for that number is guessing.
>
> On Qdrant for retrieval and Solr for the rest: it works, with two costs.
> The filters have to be duplicated on the Qdrant side, otherwise you hit the
> same post filtering problem plus an fq holding thousands of IDs. And the
> score fusion is harder than it looks. e5 class cosine is anisotropic: ours
> sits in a band of roughly 0.927 to 0.935, so blending raw cosine with BM25
> leaves the vector leg effectively dead weight. Min max normalize both legs
> within the candidate set before you blend. That one change took our results
> from editorial articles at the top to the actual products, same model, same
> index, same day.
>
> For what it is worth, we do exactly this at Opensolr: BM25 and vector
> fused per document inside a custom Solr query parser, on managed Solr.
> These pages describe the mechanics and there is a live RAG demo you can run
> against a real index:
>
> https://opensolr.com/opensolr-platform-user-documentation/hybrid-search
> https://opensolr.com/opensolr-platform-user-documentation/ai-vector
> https://opensolr.com/opensolr-platform-user-documentation/search-tuning
> https://opensolr.com/rag-in-60-seconds
>
> *Opensolr.com*
> *Your Path to AI Search <https://opensolr.com>*
> [email protected]
> https://opensolr.com
> VAT: RO-35410526
>
> On 10 Sep 2026, at 09:35, Satya Nand <[email protected]> wrote:
>
> Hi Solr Community,
>
> We are evaluating Solr for vector search and would appreciate some guidance
> on performance and sizing.
>
> We have ~2M vector documents, but are seeing *multiple-second latency for
> the first ~100 queries*. After that, performance improves. we want to scale
> it to 200 M documents in future.
>
> A few questions:
>
> - Is the default Solr vector-search configuration production-ready, or
> does it typically require tuning?
> - What are the key parameters/settings we should tune for HNSW/KNN
> performance?
> - How should we estimate the *CPU, memory, storage, and node
> requirements* for a given vector count, dimensions, QPS, and target
> latency?
> - For larger scale, how does Solr vector search typically compare with a
> dedicated vector DB such as *Qdrant* in terms of hardware requirements
> and scalability?
> - Would this architecture make sense: *Qdrant for vector retrieval →
> return candidate IDs → Solr for existing business logic, filtering, and
> ranking*, or is it better to keep the entire flow in Solr?
>
> Our preference is to keep vector search in Solr if it can meet the required
> latency and scale without significantly higher infrastructure cost.
>
> Any recommendations or real-world experience would be very helpful.
>
> --
> Satya Nand Kanodia
> Architect
> Search
> +91-8130189159 <+91%2081301%2089159>
>
>
>
--
Satya Nand Kanodia
Architect
Search
+91-8130189159
