Hi Sunny, What you're describing makes sense to me.
The preFilter in Solr will cause the HNSW search to traverse the graph more deeply to find the top 100 candidates that meet the filter conditions before terminating. That would explain the increased latency you're seeing with a filter selectivity of ~50%. As mentioned, postFilter will apply the filter after the candidates are retrieved. That means the latency will be similar to a regular unfiltered query but candidates will be dropped and there can be an impact on recall. I imagine this works for your queries currently because they're requesting K=100 with rows=10, which would allow for a large set of results to be filtered out. Could you confirm what version of Solr you're using? As mentioned in an earlier reply, reducing the size of the index could help. For Solr 9.x, the only supported option for that is BYTE (int8) encoding, or using fewer dimensions. For Solr 10, Scalar Quantization is worth looking into. With Solr 10, there are a number of other improvements that could help: 1. ACORN-1 filtering Solr 10 has a new "filteredSearchThreshold" parameter that uses a variant of the ACORN algorithm. Lucene benchmarks show that this improves latency for filtered searches quite significantly. https://solr.apache.org/guide/solr/latest/query-guide/dense-vector-search.html#knn-query-parser 2. earlyTermination Solr 10 also has an earlyTermination parameter that will terminate the ANN search earlier than normal, with some configurable parameters, at the cost of a small trade-off in recall. That's another dial that could be experimented with. Regards, Adam Quigley From: [email protected] At: 09/14/26 14:50:31 UTC-4:00To: [email protected] Cc: [email protected], [email protected] Subject: Re: Solr Vector Search Performance and Sizing Guidance Hi all, We benchmarked Solr KNN preFilter (filters evaluated inside the HNSW graph traversal) against our current postFilter approach (filters applied as a regular fq after the ANN search) to see if preFilter reduces query latency. preFilter came out about 2x slower than postFilter. Wanted to check if this is expected for our setup or if there's a better way to combine KNN search with metadata filters. 10,000 queries each: postFilter solr QTime median 84ms, avg 100.57ms, p95 143ms, max 6456ms. preFilter median 188ms, avg 209.96ms, p95 228ms, max 13722ms. preFilter is consistently around 2x slower across every percentile. Actual postFilter request body we send (POST params, vector_str is the 2048-dim query embedding): q={!bool should=$imgb should=$textb should=$multib} defType=lucene fl=displayid,title,score rows=10 wt=json imgq={!knn f=img_dot_emb topK=100 v=$vector_str} imgb={!boost b=0.2 v=$imgq} textq={!knn f=text_dot_emb topK=100 v=$vector_str} textb={!boost b=0.4 v=$textq} multiq={!knn f=multimodal_dot_emb topK=100 v=$vector_str} multib={!boost b=0.4 v=$multiq} vector_str=[<2048 floats>] fq=has_img_dot_emb:true fq=has_text_dot_emb:true fq=has_multimodal_dot_emb:true fq=CustTypeWt:[0 TO 699] OR CustTypeWt:(1114 1199 1099 1139 1219) fq=productVisibilityFlag:1 Actual preFilter request body we send (same fields/weights, filters moved into each knn clause instead of fq): q={!bool should=$imgb should=$textb should=$multib} defType=lucene fl=displayid,title,score rows=10 wt=json imgq={!knn f=img_dot_emb topK=100 preFilter=$pf1 preFilter=$pf2 preFilter=$pfimg v=$vector_str} imgb={!boost b=0.2 v=$imgq} textq={!knn f=text_dot_emb topK=100 preFilter=$pf1 preFilter=$pf2 preFilter=$pftext v=$vector_str} textb={!boost b=0.4 v=$textq} multiq={!knn f=multimodal_dot_emb topK=100 preFilter=$pf1 preFilter=$pf2 preFilter=$pfmulti v=$vector_str} multib={!boost b=0.4 v=$multiq} vector_str=[<2048 floats>] pf1=CustTypeWt:[0 TO 699] OR CustTypeWt:(1114 1199 1099 1139 1219) pf2=productVisibilityFlag:1 pfimg=has_img_dot_emb:true pftext=has_text_dot_emb:true pfmulti=has_multimodal_dot_emb:true Filter volume(2,644,090 total docs): has_img_dot_emb:true matches 1,830,535 docs (69.2%). has_text_dot_emb:true matches 1,771,756 docs (67.0%). has_multimodal_dot_emb:true matches 1,771,755 docs (67.0%). The CustTypeWt condition matches 2,613,219 docs (98.8%) - barely selective. productVisibilityFlag:1 matches 2,311,558 docs (87.4%). All 5 conditions combined (AND) match 1,403,882 docs (53.1%) - so just over half the corpus passes every filter, and none of the individual filters is narrow enough that we'd expect preFilter's during-traversal pruning to be cheap. Questions: is preFilter being slower than postFilter expected given our filter selectivity, topK=100, and index size, or does it point to a misconfiguration on our end? Since preFilter gives better recall but costs more, is there a way to get both, e.g. different topK, a different way to combine the preFilter clauses, or another Solr KNN filtering strategy? Any tuning approach you'd suggest before we commit to postFilter for production? Thanks, Sunny On Fri, Sep 11, 2026 at 5:48 PM Ciprian Dimofte - Opensolr.com via users < [email protected]> wrote: > Hi Satya, > > One correction to my previous mail first: the explicit preFilter local > param arrived in Solr 9.6, not 9.0. From 9.1 to 9.5 only the implicit form > exists. > > 1. Memory for three 2048 dim fields > > The arithmetic first: 3 x 2048 x 4 bytes = 24 KB of raw vectors per > document, so about 49 GB at 2M docs, six times what I computed last time. > The options, all native: > > vectorEncoding="BYTE" on DenseVectorField (Solr 9.3+). 4x smaller, about > 12 GB at 2M docs. You quantize to int8 yourself before indexing, and the > query vector with exactly the same scheme. > > ScalarQuantizedDenseVectorField (7 or 4 bits) and > BinaryQuantizedDenseVectorField (1 bit). Solr does the quantization at > index time and you keep sending floats. These are Solr 10.0 only. > > Fewer dimensions, but only if your model was trained Matryoshka style. > Then truncating to 1024 and renormalizing halves memory and distance > computation in one move. On a model not trained for it, truncation breaks > it. > > On the quality drop: we run float32 in production, so I have no number of > my own, and a number quoted without your model and your queries is a guess. > It is cheap to measure: take a few hundred real queries, use the float32 > top 10 as ground truth, and compute recall@10 of the compact version > against it, per field. > > 2. stored=true for atomic updates > > An atomic update does not save indexing work. Solr reads the stored > fields, applies your change and reindexes the entire document, vectors > included, so the HNSW insert happens anyway. What you get is not having to > resend the full document, and you pay for it in disk and page cache on > every document, permanently. > > It also means your three vector fields must be stored, because > DenseVectorField has no docValues. Otherwise an atomic update on a text > field silently drops the vectors. That is another 49 GB in stored fields, > and floats compress badly. > > At your size the usual design is that Solr is not the system of record. > Keep text and vectors in your source store (a database, object storage, a > compacted Kafka topic), rebuild the whole document on any change, and send > it complete. Then everything not returned in fl can be stored=false. For > you that is probably everything except displayid and title. > > 3. preFilter with {!bool} > > Implicit pre-filtering does not work in your setup. The docs are explicit: > fq is used as an implicit pre-filter only when knn is the main q. As a > subquery clause, which is what your three knn queries are inside {!bool}, > there is no implicit pre-filter. Right now each knn takes its global top K > and the fq cuts that afterwards. > > You need explicit preFilter on each knn (Solr 9.6+). It accepts a > parameter reference that is multivalued, so: > > ("imgq", "{!knn f=img_dot_emb topK=$topK preFilter=$kfq v=$vector_str}"), > ("textq", "{!knn f=text_dot_emb topK=$topK preFilter=$kfq v=$vector_str}"), > ("multiq", "{!knn f=multimodal_dot_emb topK=$topK preFilter=$kfq > v=$vector_str}"), > ("kfq", "has_img_dot_emb:true"), > ("kfq", "has_text_dot_emb:true"), > ("kfq", "has_multimodal_dot_emb:true"), > > Everything else stays as you have it, and you can keep the three fq lines > as well. > > 4. Min max normalization > > BM25 is unbounded and its range changes with every query and every boost. > Dot product sits in a narrow band. A weight multiplies that band but does > not widen it: two docs at 0.930 and 0.935 are still 0.005 apart after any > weight you pick, while their BM25 can differ by 8. So the vector leg cannot > reorder anything, and no weight fixes that. > > The method: > > a. Build the candidate set: lexical top N plus each vector leg's top K. > b. Normalize each leg separately within that set. Lexical: divide by the > highest lexical score, which gives 0 to 1 and keeps the ratios BM25 already > has. Vector: min max over the candidates that have a vector score, norm = > (score - min) / (max - min), because that is the leg squeezed into a narrow > band. A candidate missing from a leg gets 0 there. > c. final = w_lex * lex + w_img * img + w_text * text + w_multi * multi, > with weights summing to 1. > d. Sort by final and take rows. > > Normalize your three vector fields separately too. Image and text > embeddings rarely share a score band, so the same problem exists between > them. > > Stock Solr cannot do this inside one request. scale(query($q)) looks > tempting but takes min and max over the whole index, and counts > non-matching docs as 0.0, so min is always 0 and the narrow band stays > narrow. It also walks every document. In stock Solr the working path is > client side: one request per leg with fl=id,score and rows=N, fuse in your > application, then fetch the final page by id. > > We do this inside a single request with our own query parser, {!hybrid}, > on managed Solr: > https://opensolr.com/opensolr-platform-user-documentation/hybrid-search > > *Opensolr.com* > *Your Path to AI Search <https://opensolr.com>* > [email protected] > https://opensolr.com > VAT: RO-35410526 > > On 11 Sep 2026, at 15:02, Satya Nand <[email protected]> wrote: > > Satya Nand Kanodia > > >
