Hi all,
We benchmarked Solr KNN preFilter (filters evaluated inside the HNSW graph
traversal) against our current postFilter approach (filters applied as a
regular fq after the ANN search) to see if preFilter reduces query latency.
preFilter came out about 2x slower than postFilter. Wanted to check if this
is expected for our setup or if there's a better way to combine KNN search
with metadata filters.
10,000 queries each:
postFilter solr QTime median 84ms, avg 100.57ms, p95 143ms, max 6456ms.
preFilter median 188ms, avg 209.96ms, p95 228ms, max 13722ms. preFilter is
consistently around 2x slower across every percentile.
Actual postFilter request body we send (POST params, vector_str is the
2048-dim query embedding):
q={!bool should=$imgb should=$textb should=$multib}
defType=lucene
fl=displayid,title,score
rows=10
wt=json
imgq={!knn f=img_dot_emb topK=100 v=$vector_str}
imgb={!boost b=0.2 v=$imgq}
textq={!knn f=text_dot_emb topK=100 v=$vector_str}
textb={!boost b=0.4 v=$textq}
multiq={!knn f=multimodal_dot_emb topK=100 v=$vector_str}
multib={!boost b=0.4 v=$multiq}
vector_str=[<2048 floats>]
fq=has_img_dot_emb:true
fq=has_text_dot_emb:true
fq=has_multimodal_dot_emb:true
fq=CustTypeWt:[0 TO 699] OR CustTypeWt:(1114 1199 1099 1139 1219)
fq=productVisibilityFlag:1
Actual preFilter request body we send (same fields/weights, filters moved
into each knn clause instead of fq):
q={!bool should=$imgb should=$textb should=$multib}
defType=lucene
fl=displayid,title,score
rows=10
wt=json
imgq={!knn f=img_dot_emb topK=100 preFilter=$pf1 preFilter=$pf2
preFilter=$pfimg v=$vector_str}
imgb={!boost b=0.2 v=$imgq}
textq={!knn f=text_dot_emb topK=100 preFilter=$pf1 preFilter=$pf2
preFilter=$pftext v=$vector_str}
textb={!boost b=0.4 v=$textq}
multiq={!knn f=multimodal_dot_emb topK=100 preFilter=$pf1 preFilter=$pf2
preFilter=$pfmulti v=$vector_str}
multib={!boost b=0.4 v=$multiq}
vector_str=[<2048 floats>]
pf1=CustTypeWt:[0 TO 699] OR CustTypeWt:(1114 1199 1099 1139 1219)
pf2=productVisibilityFlag:1
pfimg=has_img_dot_emb:true
pftext=has_text_dot_emb:true
pfmulti=has_multimodal_dot_emb:true
Filter volume(2,644,090 total docs):
has_img_dot_emb:true matches 1,830,535 docs (69.2%). has_text_dot_emb:true
matches 1,771,756 docs (67.0%). has_multimodal_dot_emb:true matches
1,771,755 docs (67.0%). The CustTypeWt condition matches 2,613,219 docs
(98.8%) - barely selective. productVisibilityFlag:1 matches 2,311,558 docs
(87.4%). All 5 conditions combined (AND) match 1,403,882 docs (53.1%) - so
just over half the corpus passes every filter, and none of the individual
filters is narrow enough that we'd expect preFilter's during-traversal
pruning to be cheap.
Questions: is preFilter being slower than postFilter expected given our
filter selectivity, topK=100, and index size, or does it point to a
misconfiguration on our end? Since preFilter gives better recall but costs
more, is there a way to get both, e.g. different topK, a different way to
combine the preFilter clauses, or another Solr KNN filtering strategy? Any
tuning approach you'd suggest before we commit to postFilter for production?
Thanks,
Sunny
On Fri, Sep 11, 2026 at 5:48 PM Ciprian Dimofte - Opensolr.com via users <
[email protected]> wrote:
> Hi Satya,
>
> One correction to my previous mail first: the explicit preFilter local
> param arrived in Solr 9.6, not 9.0. From 9.1 to 9.5 only the implicit form
> exists.
>
> 1. Memory for three 2048 dim fields
>
> The arithmetic first: 3 x 2048 x 4 bytes = 24 KB of raw vectors per
> document, so about 49 GB at 2M docs, six times what I computed last time.
> The options, all native:
>
> vectorEncoding="BYTE" on DenseVectorField (Solr 9.3+). 4x smaller, about
> 12 GB at 2M docs. You quantize to int8 yourself before indexing, and the
> query vector with exactly the same scheme.
>
> ScalarQuantizedDenseVectorField (7 or 4 bits) and
> BinaryQuantizedDenseVectorField (1 bit). Solr does the quantization at
> index time and you keep sending floats. These are Solr 10.0 only.
>
> Fewer dimensions, but only if your model was trained Matryoshka style.
> Then truncating to 1024 and renormalizing halves memory and distance
> computation in one move. On a model not trained for it, truncation breaks
> it.
>
> On the quality drop: we run float32 in production, so I have no number of
> my own, and a number quoted without your model and your queries is a guess.
> It is cheap to measure: take a few hundred real queries, use the float32
> top 10 as ground truth, and compute recall@10 of the compact version
> against it, per field.
>
> 2. stored=true for atomic updates
>
> An atomic update does not save indexing work. Solr reads the stored
> fields, applies your change and reindexes the entire document, vectors
> included, so the HNSW insert happens anyway. What you get is not having to
> resend the full document, and you pay for it in disk and page cache on
> every document, permanently.
>
> It also means your three vector fields must be stored, because
> DenseVectorField has no docValues. Otherwise an atomic update on a text
> field silently drops the vectors. That is another 49 GB in stored fields,
> and floats compress badly.
>
> At your size the usual design is that Solr is not the system of record.
> Keep text and vectors in your source store (a database, object storage, a
> compacted Kafka topic), rebuild the whole document on any change, and send
> it complete. Then everything not returned in fl can be stored=false. For
> you that is probably everything except displayid and title.
>
> 3. preFilter with {!bool}
>
> Implicit pre-filtering does not work in your setup. The docs are explicit:
> fq is used as an implicit pre-filter only when knn is the main q. As a
> subquery clause, which is what your three knn queries are inside {!bool},
> there is no implicit pre-filter. Right now each knn takes its global top K
> and the fq cuts that afterwards.
>
> You need explicit preFilter on each knn (Solr 9.6+). It accepts a
> parameter reference that is multivalued, so:
>
> ("imgq", "{!knn f=img_dot_emb topK=$topK preFilter=$kfq v=$vector_str}"),
> ("textq", "{!knn f=text_dot_emb topK=$topK preFilter=$kfq v=$vector_str}"),
> ("multiq", "{!knn f=multimodal_dot_emb topK=$topK preFilter=$kfq
> v=$vector_str}"),
> ("kfq", "has_img_dot_emb:true"),
> ("kfq", "has_text_dot_emb:true"),
> ("kfq", "has_multimodal_dot_emb:true"),
>
> Everything else stays as you have it, and you can keep the three fq lines
> as well.
>
> 4. Min max normalization
>
> BM25 is unbounded and its range changes with every query and every boost.
> Dot product sits in a narrow band. A weight multiplies that band but does
> not widen it: two docs at 0.930 and 0.935 are still 0.005 apart after any
> weight you pick, while their BM25 can differ by 8. So the vector leg cannot
> reorder anything, and no weight fixes that.
>
> The method:
>
> a. Build the candidate set: lexical top N plus each vector leg's top K.
> b. Normalize each leg separately within that set. Lexical: divide by the
> highest lexical score, which gives 0 to 1 and keeps the ratios BM25 already
> has. Vector: min max over the candidates that have a vector score, norm =
> (score - min) / (max - min), because that is the leg squeezed into a narrow
> band. A candidate missing from a leg gets 0 there.
> c. final = w_lex * lex + w_img * img + w_text * text + w_multi * multi,
> with weights summing to 1.
> d. Sort by final and take rows.
>
> Normalize your three vector fields separately too. Image and text
> embeddings rarely share a score band, so the same problem exists between
> them.
>
> Stock Solr cannot do this inside one request. scale(query($q)) looks
> tempting but takes min and max over the whole index, and counts
> non-matching docs as 0.0, so min is always 0 and the narrow band stays
> narrow. It also walks every document. In stock Solr the working path is
> client side: one request per leg with fl=id,score and rows=N, fuse in your
> application, then fetch the final page by id.
>
> We do this inside a single request with our own query parser, {!hybrid},
> on managed Solr:
> https://opensolr.com/opensolr-platform-user-documentation/hybrid-search
>
> *Opensolr.com*
> *Your Path to AI Search <https://opensolr.com>*
> [email protected]
> https://opensolr.com
> VAT: RO-35410526
>
> On 11 Sep 2026, at 15:02, Satya Nand <[email protected]> wrote:
>
> Satya Nand Kanodia
>
>
>
