I'm not sure the KNN search supports multiple sort conditions. I should do some deep dive in the Lucene code but I don't have time in the foreseeable short future. I can imagine that anyway it would only support 're-ranking' the retrieved topK by the additional score condition, this can not affect how the topK is retrieved, at best how it's sorted.
What would be really valuable is if you can reproduce the issue and even better if the issue can be reproduced using a solr test (org.apache.solr.search.neural.KnnQParserTest) Cheers -------------------------- *Alessandro Benedetti* Director @ Sease Ltd. *Apache Lucene/Solr Committer* *Apache Solr PMC Member* e-mail: a.benede...@sease.io *Sease* - Information Retrieval Applied Consulting | Training | Open Source Website: Sease.io <http://sease.io/> LinkedIn <https://linkedin.com/company/sease-ltd> | Twitter <https://twitter.com/seaseltd> | Youtube <https://www.youtube.com/channel/UCDx86ZKLYNpI3gzMercM7BQ> | Github <https://github.com/seaseltd> On Tue, 21 Jan 2025 at 13:04, Moll, Dr. Andreas <m...@juris.de.invalid> wrote: > Hi, > I want to inform you about a behavior change in SolR 9.6 (Lucene 9.10) vs. > SolR 9.7 (Lucene 9.11) for vector searches. > We heavily rely on vector searches for embeddings in combination with > filter queries on the parent documents. > Our queries in general looked like this: > select?q={ knn f=vector topK=2048}[...] > rows=100 > fq={ child of='childtype:root'}... > start=0 > sort=score desc,ID desc > With SolR 9.7 and higher, this results in ~10% of the queries producing > the following error: > java.lang.IllegalArgumentException: Doc id 27227879 doesn't match the query > at > org.apache.lucene.search.TopFieldCollector.populateScores(TopFieldCollector.java:478) > ~[?:?] > at > org.apache.solr.search.SolrIndexSearcher.populateScoresIfNeeded(SolrIndexSearcher.java:1812) > ~[?:?] > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:2001) > ~[?:?] > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1775) > ~[?:?] > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:772) > ~[?:?] > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:767) > ~[?:?] > After several days of debugging, I confirmed that the number of errors > correlates to the topK value: > > * k = 8 -> 44 errors > * k = 2048 -> 17 errors > * k = 16384 -> 1 error > I found a workaround for the issue by modifying the sort parameter to: > sort=score desc > With this change, our queries work like a charm again. The initial thought > of adding the ID desc sorting was to get more reproducible results, but it > is not strictly necessary for us. > Could you clarify if this change in SolR/Lucene was intended? If so, > perhaps you want to add documentation on vector queries that adding an > additional sorting might cause errors. > Best regards, > Dr. Andreas Moll > >
