b) The knn query results are the approximate nearest neighbors, but they might not be the best. We'd like to define some kind of cut-off value for knn document scores. Is this possible, and what would be a good day to do so? Implement a post-processing filter query with an frange on the score field?
FOr the cutoff is not that simple unfortunately, we spent some time at hackathons and conferences but never got something of my satisfaction, unfortunately you don't have a fair distribution of scores between 0 and 1, i.e. in some scenarios 0.9 is relevant in others 0.2 is relevant. Sometimes you have big gaps in score and still, the document is relevant. Long story short, I would need to study your case and think on a solution, nothing easy is out there. Cheers -------------------------- *Alessandro Benedetti* Director @ Sease Ltd. *Apache Lucene/Solr Committer* *Apache Solr PMC Member* e-mail: a.benede...@sease.io *Sease* - Information Retrieval Applied Consulting | Training | Open Source Website: Sease.io <http://sease.io/> LinkedIn <https://linkedin.com/company/sease-ltd> | Twitter <https://twitter.com/seaseltd> | Youtube <https://www.youtube.com/channel/UCDx86ZKLYNpI3gzMercM7BQ> | Github <https://github.com/seaseltd> On Tue, 17 Oct 2023 at 18:16, Alessandro Benedetti <a.benede...@sease.io> wrote: > Hi Mirko, > the topK is per shard. > Then shards * k results are aggregated. > Does it make sense? > > In regards to the debugging, it seems a bug, they all should be with a > score and within top-k > -------------------------- > *Alessandro Benedetti* > Director @ Sease Ltd. > *Apache Lucene/Solr Committer* > *Apache Solr PMC Member* > > e-mail: a.benede...@sease.io > > > *Sease* - Information Retrieval Applied > Consulting | Training | Open Source > > Website: Sease.io <http://sease.io/> > LinkedIn <https://linkedin.com/company/sease-ltd> | Twitter > <https://twitter.com/seaseltd> | Youtube > <https://www.youtube.com/channel/UCDx86ZKLYNpI3gzMercM7BQ> | Github > <https://github.com/seaseltd> > > > On Tue, 17 Oct 2023 at 10:41, Mirko Sertic <mirko.ser...@web.de> wrote: > >> Hey! >> >> Thank you for your help! >> >> We are running in cloud mode on GKE. Our index has 2 shards, and every >> shard has 2 replicas. The leader is a TLOG, the other replica is a PULL. >> >> Our main query is basically {!knn f=VECTOR_FIELD topK=10}[VECTOR DATA]. >> Thats it. >> >> I am really unsure how to debug this issue, Solr explain only gives some >> basic information. I really do not understand why I am getting more than >> topK=10 search results, and some of them do not match according to the >> explain data. >> >> Here is an example debug log of the parsed query: >> >> "rawquerystring" : "{!knn f=TEXTEMBEDDINGS_BESCHREIBUNG_768 >> topK=6}[-0.72818565,-0.41219202,0.5225889,0.53035444,0.76394194,0.7767894,0.82073385,0.95900697,-0.36803666,-0.21959017,0.7140104,-0.13439204,0.06747756,1.3333143,0.2797215,1.3020324,0.13869889,-0.10991136,0.21142228,-0.36915645,-0.15180133,-0.62384933,-0.51568866,0.23904686,-0.61362225,0.026911864,0.36433217,-0.6443799,0.86989623,-0.0058394694,0.360967,-0.39565668,0.4483029,-0.26269752,0.24516392,0.44815013,0.43040967,0.3587644,0.873287,-0.10215642,-0.25633913,-0.21385787,-0.5121543,-0.103228725,0.3895751,-0.49162507,-0.9432531,-0.5980949,0.63356507,-0.40455475,0.9550473,0.7282631,-0.031743232,0.439844,0.21040638,-0.36207297,0.22014715,-0.5069794,0.4957589,-0.30554834,-0.8615413,-0.4926672,-0.013558676,-0.041132603,0.65787005,0.30416572,0.18375356,0.6744972,-0.6142544,-0.18229552,-0.11830064,1.0248266,0.2171961,1.0895737,-0.027262354,-0.17439449,0.74050045,-0.8744221,-0.16733684,0.93963474,-0.4188998,-1.1815201,-0.4237022,-1.4382845,0.9899152,-0.17618449,0.8811465,-0.1435171,0.1467631,0.20906292,-0.19986387,-0.07172442,-0.5412453,-0.38141707,-0.8519122,0.57226634,0.27108946,0.47601715,-0.50961906,0.7443681,-0.46236074,0.44012043,-0.64782083,0.12072155,0.17669038,0.8894314,-0.26589298,-1.0005188,-0.38552156,-0.17849512,-0.41564974,0.46785104,0.07669882,0.098340474,-0.40940204,-0.22534837,-0.46240783,-0.6026122,-0.150852,-0.056922164,-0.5217692,0.2522323,-0.05158736,0.4368061,0.07538188,-0.15538858,-0.27512375,-7.942749,-0.5104796,0.3684156,0.17436846,0.33592805,0.30256566,0.3045547,-0.8253086,0.39595905,-0.63918144,0.023157855,-0.19350462,-1.1046336,0.24591433,-0.35902688,-0.80178124,0.60646725,-0.30873522,0.3227124,0.14232452,-0.11611875,0.36796737,-0.53184104,0.30364224,0.28488177,-1.0550518,-0.42094457,-0.12320968,-0.9765126,-0.15129066,0.001331389,-0.27246574,0.4749163,0.4177817,0.2973763,-0.29119635,-0.3972071,0.59532064,0.10756115,0.23748569,-0.70517355,1.201706,0.502714,-0.26869884,-0.053134415,-0.66120774,-0.0039092028,-0.64522946,-0.041144915,-0.05124693,-0.9726303,-0.99400455,-0.07065071,-0.78489596,0.80071807,-0.13281979,-0.006903738,0.5823942,-0.2669629,1.6502038,0.7482211,-0.6007372,0.66805166,-0.3942953,-0.5034327,0.27077916,0.15055649,0.10765681,-0.12882604,0.24207312,0.18053184,-0.21453774,0.45982376,-0.5745059,0.2006089,0.50371104,0.8538735,0.97700334,0.05624181,-0.132805,0.033950403,-0.18651533,-0.06351512,-0.7937849,-0.6627913,-0.3048031,0.53663576,0.17241986,0.36505452,-0.09964923,0.13906284,0.5434706,-0.36837682,-0.35080168,0.46206424,-0.20238751,-0.27848598,0.2480285,0.5826801,-0.48960462,0.7367855,1.0053717,0.17819782,0.38643655,-0.16025475,-0.35159552,-0.0024995431,-0.022663444,0.05895573,-0.19539028,-0.36216947,-0.06652315,-0.13567717,-0.68101376,-0.3490555,0.55224353,0.045317963,-0.8309715,-0.24363773,0.4269611,0.24411823,-0.06477823,0.80422884,0.35045695,0.20747428,-0.3475987,0.1297931,0.09917005,0.007867369,0.38570535,0.13428706,1.3826305,-0.8623333,-1.0942854,0.3748387,-0.5336166,-0.35972998,-0.10899716,0.278379,0.94615966,-0.93360096,-0.9554229,-0.28387132,-0.02770327,-0.9298182,0.45993057,-0.65919405,0.1475371,-0.34568012,1.209977,0.2802038,0.45577803,0.12152732,0.87355286,1.5883585,0.6779906,0.17098032,1.0125086,-0.55774075,-0.38352862,0.16628654,0.15557681,0.1861148,0.04379474,-0.7305415,-0.030636007,0.024982145,0.67828315,0.42840484,-0.053252134,-0.4504099,-0.20452605,-0.1081517,-0.14191566,0.07114916,-0.078530826,0.17243765,-0.46307424,-0.008914786,0.26029265,0.52127206,-0.35917446,-0.015462582,-0.28368863,-0.7689151,-0.263665,0.49229744,0.7343957,0.24539919,-0.2738248,-0.12916541,1.5979353,0.2385032,-0.5589315,0.1753772,-0.57815063,0.13694653,-0.20679545,0.78396624,-0.26194802,0.16952795,0.009313789,0.053571712,-0.50364834,0.19213165,0.020542603,0.3242613,-0.21481745,0.46329287,0.1883949,-0.53527766,0.19803612,-0.10915735,-0.53217936,0.5851483,0.8633852,-0.22306518,0.19917397,-0.48938546,-0.3489624,0.26114023,0.5178694,-0.49423835,0.22051883,0.9698861,-0.023483349,0.5058909,-0.19136174,-0.06710764,-0.50730103,-0.5724341,-0.07458467,0.72695184,0.36962357,0.12050099,0.16827017,-0.36094713,-0.22090997,0.5247373,0.5981514,0.11581371,-0.14274465,0.110738344,0.5218388,-0.38720345,0.9173073,-0.8510778,-0.22613077,-0.2748853,0.2863147,-0.3778913,-0.18058139,0.37012923,-0.5144021,-0.5551922,0.16914673,0.78963786,1.1504575,-0.7211657,-0.63688916,-0.075630635,-0.89228654,0.8719732,-0.16148148,-0.093216084,-0.0649338,0.40419587,-0.5091749,0.3257159,-0.38114238,-0.7489807,-0.14795141,0.070623375,1.0296395,-0.18232572,0.045388192,-0.5071435,-0.23788519,-0.60791194,-0.25087252,0.009456612,0.8095627,-0.43995717,-0.21105917,-0.83442384,0.7455656,-0.9289396,-0.26139355,0.33529952,-0.7209875,0.44793245,-0.11075375,0.35380912,-0.49433497,1.061043,0.24260058,1.277706,-1.0634571,-0.028423825,-0.6881516,-0.2748189,0.32650438,0.22231913,-0.6346664,-0.32749617,0.6449113,0.4030831,0.086880766,-0.34941757,-0.32754892,0.44228384,-1.0537914,-0.4920558,0.11018584,-0.4000114,0.19572248,-0.006614799,-0.1560454,-0.6138168,-0.2874727,0.08457174,0.852747,0.020211289,-0.46017483,0.05079806,0.61822194,0.16241436,-0.418595,-0.44193658,-0.23446727,0.08835063,-0.86544657,0.039239086,0.47766128,-0.31095138,-0.545759,-0.09576557,-0.5694708,-0.10043368,-0.2653149,-0.1599367,0.13423382,-0.1830842,0.046266567,-0.08192452,-0.75070643,-0.65771914,0.18250088,-0.36299148,0.33271846,-0.60936797,1.339671,-0.12343848,1.0515145,0.25372154,-0.4882932,-0.06304815,-0.17309606,0.30466563,0.08878487,-0.5156037,-0.33687803,0.5997892,0.1856752,0.5528808,0.5089958,-0.77673227,-0.43565074,-3.0061991,-0.22721101,1.1010107,1.285386,0.39106908,0.06541425,0.18982089,-0.33513412,-0.5287475,0.2362032,0.40598843,0.62325794,-0.8586378,-0.64406306,0.09112385,0.055086333,0.28247252,0.13180174,0.5969885,0.15263812,0.39325213,-0.45515132,-0.47813737,-0.33163494,0.3496457,0.39913225,-0.5750896,0.104484975,-0.69256973,0.18164606,0.09423768,0.045548365,0.51491827,-0.854532,0.40470743,-0.12821743,-0.22693269,-0.31501335,-0.53367645,0.4482267,-0.647523,-0.48177683,-0.45763886,0.31513563,0.115996234,-0.56461513,0.112646066,0.6897073,0.30277768,1.2858859,-0.47804725,0.36647198,0.8737555,0.89239925,-0.26573908,0.13890187,0.8274238,0.6068263,0.21134727,-0.5331567,-0.40894493,-0.20580018,-0.4116305,-8.852631E-4,0.71050125,0.18508287,0.3889124,-0.38231707,-0.7620664,-0.26053023,0.5970558,0.12187529,-0.35657012,-0.20820218,-0.4604194,-0.36569384,0.08593381,0.31331047,-0.12055105,0.12107455,0.45513165,-0.0023682665,0.3980597,0.1632141,0.33392814,-0.050680444,0.13038154,0.13847367,1.3034295,0.2744402,0.124595374,-0.53905815,0.35776177,-0.13275874,0.018385967,-0.16347891,-0.19726902,-0.4961019,0.07333634,0.52688265,0.20600657,0.14087541,-0.21112329,0.5135732,0.4315345,-0.020893982,0.41702232,0.07464256,-0.2845659,-0.13949631,-0.013015106,0.40522456,0.041121256,1.1591781,0.043223336,0.39874303,-0.30360883,-0.27349794,-0.04964445,-0.5729313,0.14069785,-0.33394682,0.9343837,-0.9350466,-0.8602237,-0.29167423,-1.5445148,0.040775828,-0.018334387,0.42681727,-0.1458294,0.49805698,0.27569738,-0.10979689,-0.5429115,-0.020050785,0.31627277,-0.6220587,0.061334964,0.13807471,0.34479836,0.11312207,0.796787,-0.48607716,-0.14010525,-0.61075526,0.6672587,1.6726193,0.4950563,0.41054416,0.2522166,-0.95681673,-1.0119423,-0.4870938,0.48430035,-0.38044646,0.59904295,0.23687701,0.93788594,-0.22263275,0.07065558,-0.4562675,-0.08655307,0.69646436,-0.5812278,-0.30097616,-0.9931224,0.23148312,0.13233739,0.74470276,0.014401446,0.08970716,0.23342986,-0.4664313,0.054693338,0.2778716,0.4019597,-0.97575116,0.058492173,-0.5017976,0.09864781,-0.3473021,-0.4823157,0.13100009,0.4488969,-0.1672722,-1.0326554,-0.18146743,-0.349057,0.61888343,-0.18784277,-0.53318554,-0.2559318,-0.6182706,0.24759102,-0.71451706,0.7098109,0.53027123,0.36208203,0.23936976,-0.10664157,0.3863506,0.42203853,0.1808002,-0.22444467,1.0584201,-0.29779503,0.43873647,0.2835624,-0.76047546,0.37234184,0.2044634,0.5471627,0.4247879,0.4856389,-0.7011678,0.40757468,0.1918484,0.54874665,-0.1279645,0.5678282,0.02382508,0.18181127,-0.42465243,0.20996757,-0.48161522,-0.53782755,-0.1985728,-0.12512843,-0.47403875,0.44477355,-0.05491643,0.81035346,0.3633467,0.5024226,-0.07482373,-0.7337039,0.0675221,0.3952929,0.45677915,-0.29935122,-0.42017272,-0.80567694,0.2868536,-0.40015128,-0.060443062,-0.21671702,0.18321274,-0.20112215,-0.50728095,0.3698101,0.56871736,-0.0904237,-0.43735075,-0.79018253,-0.0739331,0.49475756,0.014863159,-0.47107005,0.14020085,0.28982484,0.32168245,-0.56485337,0.17436136,-0.0050138533,-0.9533212,0.036251593,-0.33072975,-0.87562966,-0.4379939,0.8052754,0.30534235,0.20573251]", >> >> "querystring" : "{!knn f=TEXTEMBEDDINGS_BESCHREIBUNG_768 >> topK=6}[-0.72818565,-0.41219202,0.5225889,0.53035444,0.76394194,0.7767894,0.82073385,0.95900697,-0.36803666,-0.21959017,0.7140104,-0.13439204,0.06747756,1.3333143,0.2797215,1.3020324,0.13869889,-0.10991136,0.21142228,-0.36915645,-0.15180133,-0.62384933,-0.51568866,0.23904686,-0.61362225,0.026911864,0.36433217,-0.6443799,0.86989623,-0.0058394694,0.360967,-0.39565668,0.4483029,-0.26269752,0.24516392,0.44815013,0.43040967,0.3587644,0.873287,-0.10215642,-0.25633913,-0.21385787,-0.5121543,-0.103228725,0.3895751,-0.49162507,-0.9432531,-0.5980949,0.63356507,-0.40455475,0.9550473,0.7282631,-0.031743232,0.439844,0.21040638,-0.36207297,0.22014715,-0.5069794,0.4957589,-0.30554834,-0.8615413,-0.4926672,-0.013558676,-0.041132603,0.65787005,0.30416572,0.18375356,0.6744972,-0.6142544,-0.18229552,-0.11830064,1.0248266,0.2171961,1.0895737,-0.027262354,-0.17439449,0.74050045,-0.8744221,-0.16733684,0.93963474,-0.4188998,-1.1815201,-0.4237022,-1.4382845,0.9899152,-0.17618449,0.8811465,-0.1435171,0.1467631,0.20906292,-0.19986387,-0.07172442,-0.5412453,-0.38141707,-0.8519122,0.57226634,0.27108946,0.47601715,-0.50961906,0.7443681,-0.46236074,0.44012043,-0.64782083,0.12072155,0.17669038,0.8894314,-0.26589298,-1.0005188,-0.38552156,-0.17849512,-0.41564974,0.46785104,0.07669882,0.098340474,-0.40940204,-0.22534837,-0.46240783,-0.6026122,-0.150852,-0.056922164,-0.5217692,0.2522323,-0.05158736,0.4368061,0.07538188,-0.15538858,-0.27512375,-7.942749,-0.5104796,0.3684156,0.17436846,0.33592805,0.30256566,0.3045547,-0.8253086,0.39595905,-0.63918144,0.023157855,-0.19350462,-1.1046336,0.24591433,-0.35902688,-0.80178124,0.60646725,-0.30873522,0.3227124,0.14232452,-0.11611875,0.36796737,-0.53184104,0.30364224,0.28488177,-1.0550518,-0.42094457,-0.12320968,-0.9765126,-0.15129066,0.001331389,-0.27246574,0.4749163,0.4177817,0.2973763,-0.29119635,-0.3972071,0.59532064,0.10756115,0.23748569,-0.70517355,1.201706,0.502714,-0.26869884,-0.053134415,-0.66120774,-0.0039092028,-0.64522946,-0.041144915,-0.05124693,-0.9726303,-0.99400455,-0.07065071,-0.78489596,0.80071807,-0.13281979,-0.006903738,0.5823942,-0.2669629,1.6502038,0.7482211,-0.6007372,0.66805166,-0.3942953,-0.5034327,0.27077916,0.15055649,0.10765681,-0.12882604,0.24207312,0.18053184,-0.21453774,0.45982376,-0.5745059,0.2006089,0.50371104,0.8538735,0.97700334,0.05624181,-0.132805,0.033950403,-0.18651533,-0.06351512,-0.7937849,-0.6627913,-0.3048031,0.53663576,0.17241986,0.36505452,-0.09964923,0.13906284,0.5434706,-0.36837682,-0.35080168,0.46206424,-0.20238751,-0.27848598,0.2480285,0.5826801,-0.48960462,0.7367855,1.0053717,0.17819782,0.38643655,-0.16025475,-0.35159552,-0.0024995431,-0.022663444,0.05895573,-0.19539028,-0.36216947,-0.06652315,-0.13567717,-0.68101376,-0.3490555,0.55224353,0.045317963,-0.8309715,-0.24363773,0.4269611,0.24411823,-0.06477823,0.80422884,0.35045695,0.20747428,-0.3475987,0.1297931,0.09917005,0.007867369,0.38570535,0.13428706,1.3826305,-0.8623333,-1.0942854,0.3748387,-0.5336166,-0.35972998,-0.10899716,0.278379,0.94615966,-0.93360096,-0.9554229,-0.28387132,-0.02770327,-0.9298182,0.45993057,-0.65919405,0.1475371,-0.34568012,1.209977,0.2802038,0.45577803,0.12152732,0.87355286,1.5883585,0.6779906,0.17098032,1.0125086,-0.55774075,-0.38352862,0.16628654,0.15557681,0.1861148,0.04379474,-0.7305415,-0.030636007,0.024982145,0.67828315,0.42840484,-0.053252134,-0.4504099,-0.20452605,-0.1081517,-0.14191566,0.07114916,-0.078530826,0.17243765,-0.46307424,-0.008914786,0.26029265,0.52127206,-0.35917446,-0.015462582,-0.28368863,-0.7689151,-0.263665,0.49229744,0.7343957,0.24539919,-0.2738248,-0.12916541,1.5979353,0.2385032,-0.5589315,0.1753772,-0.57815063,0.13694653,-0.20679545,0.78396624,-0.26194802,0.16952795,0.009313789,0.053571712,-0.50364834,0.19213165,0.020542603,0.3242613,-0.21481745,0.46329287,0.1883949,-0.53527766,0.19803612,-0.10915735,-0.53217936,0.5851483,0.8633852,-0.22306518,0.19917397,-0.48938546,-0.3489624,0.26114023,0.5178694,-0.49423835,0.22051883,0.9698861,-0.023483349,0.5058909,-0.19136174,-0.06710764,-0.50730103,-0.5724341,-0.07458467,0.72695184,0.36962357,0.12050099,0.16827017,-0.36094713,-0.22090997,0.5247373,0.5981514,0.11581371,-0.14274465,0.110738344,0.5218388,-0.38720345,0.9173073,-0.8510778,-0.22613077,-0.2748853,0.2863147,-0.3778913,-0.18058139,0.37012923,-0.5144021,-0.5551922,0.16914673,0.78963786,1.1504575,-0.7211657,-0.63688916,-0.075630635,-0.89228654,0.8719732,-0.16148148,-0.093216084,-0.0649338,0.40419587,-0.5091749,0.3257159,-0.38114238,-0.7489807,-0.14795141,0.070623375,1.0296395,-0.18232572,0.045388192,-0.5071435,-0.23788519,-0.60791194,-0.25087252,0.009456612,0.8095627,-0.43995717,-0.21105917,-0.83442384,0.7455656,-0.9289396,-0.26139355,0.33529952,-0.7209875,0.44793245,-0.11075375,0.35380912,-0.49433497,1.061043,0.24260058,1.277706,-1.0634571,-0.028423825,-0.6881516,-0.2748189,0.32650438,0.22231913,-0.6346664,-0.32749617,0.6449113,0.4030831,0.086880766,-0.34941757,-0.32754892,0.44228384,-1.0537914,-0.4920558,0.11018584,-0.4000114,0.19572248,-0.006614799,-0.1560454,-0.6138168,-0.2874727,0.08457174,0.852747,0.020211289,-0.46017483,0.05079806,0.61822194,0.16241436,-0.418595,-0.44193658,-0.23446727,0.08835063,-0.86544657,0.039239086,0.47766128,-0.31095138,-0.545759,-0.09576557,-0.5694708,-0.10043368,-0.2653149,-0.1599367,0.13423382,-0.1830842,0.046266567,-0.08192452,-0.75070643,-0.65771914,0.18250088,-0.36299148,0.33271846,-0.60936797,1.339671,-0.12343848,1.0515145,0.25372154,-0.4882932,-0.06304815,-0.17309606,0.30466563,0.08878487,-0.5156037,-0.33687803,0.5997892,0.1856752,0.5528808,0.5089958,-0.77673227,-0.43565074,-3.0061991,-0.22721101,1.1010107,1.285386,0.39106908,0.06541425,0.18982089,-0.33513412,-0.5287475,0.2362032,0.40598843,0.62325794,-0.8586378,-0.64406306,0.09112385,0.055086333,0.28247252,0.13180174,0.5969885,0.15263812,0.39325213,-0.45515132,-0.47813737,-0.33163494,0.3496457,0.39913225,-0.5750896,0.104484975,-0.69256973,0.18164606,0.09423768,0.045548365,0.51491827,-0.854532,0.40470743,-0.12821743,-0.22693269,-0.31501335,-0.53367645,0.4482267,-0.647523,-0.48177683,-0.45763886,0.31513563,0.115996234,-0.56461513,0.112646066,0.6897073,0.30277768,1.2858859,-0.47804725,0.36647198,0.8737555,0.89239925,-0.26573908,0.13890187,0.8274238,0.6068263,0.21134727,-0.5331567,-0.40894493,-0.20580018,-0.4116305,-8.852631E-4,0.71050125,0.18508287,0.3889124,-0.38231707,-0.7620664,-0.26053023,0.5970558,0.12187529,-0.35657012,-0.20820218,-0.4604194,-0.36569384,0.08593381,0.31331047,-0.12055105,0.12107455,0.45513165,-0.0023682665,0.3980597,0.1632141,0.33392814,-0.050680444,0.13038154,0.13847367,1.3034295,0.2744402,0.124595374,-0.53905815,0.35776177,-0.13275874,0.018385967,-0.16347891,-0.19726902,-0.4961019,0.07333634,0.52688265,0.20600657,0.14087541,-0.21112329,0.5135732,0.4315345,-0.020893982,0.41702232,0.07464256,-0.2845659,-0.13949631,-0.013015106,0.40522456,0.041121256,1.1591781,0.043223336,0.39874303,-0.30360883,-0.27349794,-0.04964445,-0.5729313,0.14069785,-0.33394682,0.9343837,-0.9350466,-0.8602237,-0.29167423,-1.5445148,0.040775828,-0.018334387,0.42681727,-0.1458294,0.49805698,0.27569738,-0.10979689,-0.5429115,-0.020050785,0.31627277,-0.6220587,0.061334964,0.13807471,0.34479836,0.11312207,0.796787,-0.48607716,-0.14010525,-0.61075526,0.6672587,1.6726193,0.4950563,0.41054416,0.2522166,-0.95681673,-1.0119423,-0.4870938,0.48430035,-0.38044646,0.59904295,0.23687701,0.93788594,-0.22263275,0.07065558,-0.4562675,-0.08655307,0.69646436,-0.5812278,-0.30097616,-0.9931224,0.23148312,0.13233739,0.74470276,0.014401446,0.08970716,0.23342986,-0.4664313,0.054693338,0.2778716,0.4019597,-0.97575116,0.058492173,-0.5017976,0.09864781,-0.3473021,-0.4823157,0.13100009,0.4488969,-0.1672722,-1.0326554,-0.18146743,-0.349057,0.61888343,-0.18784277,-0.53318554,-0.2559318,-0.6182706,0.24759102,-0.71451706,0.7098109,0.53027123,0.36208203,0.23936976,-0.10664157,0.3863506,0.42203853,0.1808002,-0.22444467,1.0584201,-0.29779503,0.43873647,0.2835624,-0.76047546,0.37234184,0.2044634,0.5471627,0.4247879,0.4856389,-0.7011678,0.40757468,0.1918484,0.54874665,-0.1279645,0.5678282,0.02382508,0.18181127,-0.42465243,0.20996757,-0.48161522,-0.53782755,-0.1985728,-0.12512843,-0.47403875,0.44477355,-0.05491643,0.81035346,0.3633467,0.5024226,-0.07482373,-0.7337039,0.0675221,0.3952929,0.45677915,-0.29935122,-0.42017272,-0.80567694,0.2868536,-0.40015128,-0.060443062,-0.21671702,0.18321274,-0.20112215,-0.50728095,0.3698101,0.56871736,-0.0904237,-0.43735075,-0.79018253,-0.0739331,0.49475756,0.014863159,-0.47107005,0.14020085,0.28982484,0.32168245,-0.56485337,0.17436136,-0.0050138533,-0.9533212,0.036251593,-0.33072975,-0.87562966,-0.4379939,0.8052754,0.30534235,0.20573251]", >> >> "parsedquery" : >> "KnnVectorQuery(KnnVectorQuery:TEXTEMBEDDINGS_BESCHREIBUNG_768[-0.72818565,...][6])", >> >> "parsedquery_toString" : >> "KnnVectorQuery:TEXTEMBEDDINGS_BESCHREIBUNG_768[-0.72818565,...][6]", >> "QParser" : "KnnQParser", >> >> Am 17.10.2023 um 10:05 schrieb Alessandro Benedetti: >> > What's your full Solr query? >> > Are you on SolrCloud or single Solr node? >> > -------------------------- >> > *Alessandro Benedetti* >> > Director @ Sease Ltd. >> > *Apache Lucene/Solr Committer* >> > *Apache Solr PMC Member* >> > >> > e-mail:a.benede...@sease.io >> > >> > >> > *Sease* - Information Retrieval Applied >> > Consulting | Training | Open Source >> > >> > Website: Sease.io<http://sease.io/> >> > LinkedIn<https://linkedin.com/company/sease-ltd> | Twitter >> > <https://twitter.com/seaseltd> | Youtube >> > <https://www.youtube.com/channel/UCDx86ZKLYNpI3gzMercM7BQ> | Github >> > <https://github.com/seaseltd> >> > >> > >> > On Tue, 17 Oct 2023 at 09:45, Mirko Sertic<mirko.ser...@web.de> wrote: >> > >> >> To correct me, there was a typo. I meant: >> >> >> >> If I specify topK=6, I get numFound=12, but only some of them match the >> >> top 6 >> >> >> >> >> >> Am 17.10.2023 um 09:31 schrieb Mirko Sertic: >> >>> Hi! >> >>> >> >>> To keep you updated, here are some observations regarding the >> >>> numFound/resultset size and DenseVectorQueries: >> >>> >> >>> If I specity topK=10, I get numFound=20, but only some of them match >> the >> >>> top 10 >> >>> >> >>> If I specify topK=8, I get numFound=16, but only some of them match >> the >> >>> top 8 >> >>> >> >>> If I specify topK=6, I get numFound=8, but only some of them match the >> >>> top 6 >> >>> >> >>> So the numFound seems always to be the double of topK. Might there be >> a >> >>> correlation with sharding? Our collection has two shards, so does this >> >>> double the results? I would't expect that, but that might be the only >> >>> thing relating to a constant 2 in our setup. >> >>> >> >>> Mirko >> >>> >> >>> >> >>> Am 16.10.2023 um 14:46 schrieb Mirko Sertic: >> >>>> Hi@all >> >>>> >> >>>> We are using Solr 9.1.1, and are trying usecases with DenseVector >> Fields >> >>>> and knnQueries in mind. >> >>>> >> >>>> During our tests, we see the following results and are trying to >> figure >> >>>> out what is going on: >> >>>> >> >>>> a) We use the following main query : {!knn f=VECTOR_FIELD >> >>>> topK=10}[VECTOR DATA]. We use it as a main query because we want to >> >>>> apply the distance function to the document score. However, when I >> try >> >>>> do debug and explain the search results, I am getting more than >> topK=10 >> >>>> result documents, some are marked as match = true with "within top >> 10", >> >>>> others are marked as match = false with "not in top 10". I'd expect >> that >> >>>> only matched documents are part of the search result, but there are >> 20 >> >>>> result documents, but only 5 of them are matched. Did I miss >> something? >> >>>> >> >>>> b) The knn query results are the approximate nearest neighbors, but >> they >> >>>> might not be the best. We'd like to define some kind of cut-off value >> >>>> for knn document scores. Is this possible, and what would be a good >> day >> >>>> to do so? Implement a post-processing filter query with an frange on >> the >> >>>> score field? >> >>>> >> >>>> Thank you all, >> >>>> >> >>>> Mirko >> >>>> >> >
