Hi Satya, One correction to my previous mail first: the explicit preFilter local param arrived in Solr 9.6, not 9.0. From 9.1 to 9.5 only the implicit form exists.
1. Memory for three 2048 dim fields
The arithmetic first: 3 x 2048 x 4 bytes = 24 KB of raw vectors per document,
so about 49 GB at 2M docs, six times what I computed last time. The options,
all native:
vectorEncoding="BYTE" on DenseVectorField (Solr 9.3+). 4x smaller, about 12 GB
at 2M docs. You quantize to int8 yourself before indexing, and the query vector
with exactly the same scheme.
ScalarQuantizedDenseVectorField (7 or 4 bits) and
BinaryQuantizedDenseVectorField (1 bit). Solr does the quantization at index
time and you keep sending floats. These are Solr 10.0 only.
Fewer dimensions, but only if your model was trained Matryoshka style. Then
truncating to 1024 and renormalizing halves memory and distance computation in
one move. On a model not trained for it, truncation breaks it.
On the quality drop: we run float32 in production, so I have no number of my
own, and a number quoted without your model and your queries is a guess. It is
cheap to measure: take a few hundred real queries, use the float32 top 10 as
ground truth, and compute recall@10 of the compact version against it, per
field.
2. stored=true for atomic updates
An atomic update does not save indexing work. Solr reads the stored fields,
applies your change and reindexes the entire document, vectors included, so the
HNSW insert happens anyway. What you get is not having to resend the full
document, and you pay for it in disk and page cache on every document,
permanently.
It also means your three vector fields must be stored, because DenseVectorField
has no docValues. Otherwise an atomic update on a text field silently drops the
vectors. That is another 49 GB in stored fields, and floats compress badly.
At your size the usual design is that Solr is not the system of record. Keep
text and vectors in your source store (a database, object storage, a compacted
Kafka topic), rebuild the whole document on any change, and send it complete.
Then everything not returned in fl can be stored=false. For you that is
probably everything except displayid and title.
3. preFilter with {!bool}
Implicit pre-filtering does not work in your setup. The docs are explicit: fq
is used as an implicit pre-filter only when knn is the main q. As a subquery
clause, which is what your three knn queries are inside {!bool}, there is no
implicit pre-filter. Right now each knn takes its global top K and the fq cuts
that afterwards.
You need explicit preFilter on each knn (Solr 9.6+). It accepts a parameter
reference that is multivalued, so:
("imgq", "{!knn f=img_dot_emb topK=$topK preFilter=$kfq v=$vector_str}"),
("textq", "{!knn f=text_dot_emb topK=$topK preFilter=$kfq v=$vector_str}"),
("multiq", "{!knn f=multimodal_dot_emb topK=$topK preFilter=$kfq
v=$vector_str}"),
("kfq", "has_img_dot_emb:true"),
("kfq", "has_text_dot_emb:true"),
("kfq", "has_multimodal_dot_emb:true"),
Everything else stays as you have it, and you can keep the three fq lines as
well.
4. Min max normalization
BM25 is unbounded and its range changes with every query and every boost. Dot
product sits in a narrow band. A weight multiplies that band but does not widen
it: two docs at 0.930 and 0.935 are still 0.005 apart after any weight you
pick, while their BM25 can differ by 8. So the vector leg cannot reorder
anything, and no weight fixes that.
The method:
a. Build the candidate set: lexical top N plus each vector leg's top K.
b. Normalize each leg separately within that set. Lexical: divide by the
highest lexical score, which gives 0 to 1 and keeps the ratios BM25 already
has. Vector: min max over the candidates that have a vector score, norm =
(score - min) / (max - min), because that is the leg squeezed into a narrow
band. A candidate missing from a leg gets 0 there.
c. final = w_lex * lex + w_img * img + w_text * text + w_multi * multi, with
weights summing to 1.
d. Sort by final and take rows.
Normalize your three vector fields separately too. Image and text embeddings
rarely share a score band, so the same problem exists between them.
Stock Solr cannot do this inside one request. scale(query($q)) looks tempting
but takes min and max over the whole index, and counts non-matching docs as
0.0, so min is always 0 and the narrow band stays narrow. It also walks every
document. In stock Solr the working path is client side: one request per leg
with fl=id,score and rows=N, fuse in your application, then fetch the final
page by id.
We do this inside a single request with our own query parser, {!hybrid}, on
managed Solr:
https://opensolr.com/opensolr-platform-user-documentation/hybrid-search
Opensolr.com
Your Path to AI Search <https://opensolr.com/>
[email protected]
https://opensolr.com <https://opensolr.com/>
VAT: RO-35410526
> On 11 Sep 2026, at 15:02, Satya Nand <[email protected]> wrote:
>
> Satya Nand Kanodia
smime.p7s
Description: S/MIME cryptographic signature
