Re: Solr 9.2 join query returns no results
Hi there,
If we are talking about SolrCloud. Up to 9.2 let to join "from" fully
replicated single shard collection "to" a many shards collection.
Starting from 9.3 Solr can join from many shards to many shards
collections, but route fields should be the fields, which are used in join
query.
AffinityFactoryPlugin.withCollectionShards may be used to collocate
corresponding collection shards.
On Wed, Jul 19, 2023 at 11:03 PM Ron Haines wrote:
> Well, that's interesting. I was under the impression that the 'join'
> required the 'joining' docs to be located on the same shard. I'd actually
> experienced fewer matches with the join, until I made sure the docs to
> 'join to' were located on the same shard as the docs that were doing the
> 'joining'. Maybe I'm confused...it would make my life a bit easier, if the
> 'same shard' requirement were not necessary.
>
> Ron Haines
>
> On Wed, Jul 19, 2023 at 3:11 PM Charles Sanders
> wrote:
>
> > Thanks for the reply Ron. You bring up a good question and something I
> > need to consider. However in the solr 7 system the records are not on
> the
> > same shard. They are spread across 2 shards.
> >
> > Currently I'm working on a small test instance so I know exactly what
> data
> > I have. I will investigate the multiple shards question.
> >
> > Thanks
> >
> > On Wed, Jul 19, 2023 at 2:57 PM Ron Haines wrote:
> >
> > > After your migration, did your documents that should 'join' end up
> > located
> > > on the same shard? I believe that is a requirement.
> > >
> > > Ron Haines
> > >
> > > On Wed, Jul 19, 2023 at 1:04 PM Charles Sanders
> > > wrote:
> > >
> > > > Thanks again Mikhail for your reply. Yes there are absolutely
> matches
> > > for
> > > > the subquery. This query should return 1 result. I'm working off a
> > > small
> > > > test collection.
> > > >
> > > > Here are the results of a debug.
> > > >
> > > > {
> > > > "responseHeader":{
> > > > "zkConnected":true,
> > > > "status":0,
> > > > "QTime":3,
> > > > "params":{
> > > > "q":"{!join from=image_id
> > > > to=image_record_id}(registry:\"registry.access.redhat.com\" AND
> > > > repository:\"rhel6/rhel\" AND published:true)&rows=100&start=0",
> > > > "indent":"true",
> > > > "q.op":"AND",
> > > > "debugQuery":"true",
> > > > "useParams":""}},
> > > >
> > > >
> > >
> >
> "response":{"numFound":0,"start":0,"maxScore":0.0,"numFoundExact":true,"docs":[]
> > > > },
> > > > "debug":{
> > > > "track":{
> > > > "rid":"localhost-96",
> > > > "EXECUTE_QUERY":{
> > > > "
> > > >
> > >
> >
> http://localhost:7574/solr/rhcc_shard2_replica_n6/|http://localhost:8983/solr/rhcc_shard2_replica_n2/
> > > > ":{
> > > > "QTime":"1",
> > > > "ElapsedTime":"2",
> > > > "RequestPurpose":"GET_TOP_IDS,SET_TERM_STATS",
> > > > "NumFound":"0",
> > > > "Response":"{responseHeader={zkConnected=true, status=0,
> > > > QTime=1, params={df=text, distrib=false, qt=select, debug=[false,
> > > > timing, track], fl=[id, score], shards.purpose=16388, start=0,
> > > > fsv=true, q.op=AND,
> > > > shard.url=
> > > >
> > >
> >
> http://localhost:7574/solr/rhcc_shard2_replica_n6/|http://localhost:8983/solr/rhcc_shard2_replica_n2/
> > > > ,
> > > > rows=10, rid=localhost-96, version=2, q={!join from=image_id
> > > > to=image_record_id}(registry:\"registry.access.redhat.com\" AND
> > > > repository:\"rhel6/rhel\" AND published:true)&rows=100&start=0,
> > > > omitHeader=false, requestPurpose=GET_TOP_IDS,SET_TERM_STATS,
> > > > NOW=1689786155982, isShard=true, wt=javabin, debugQuery=false,
> > > > useParams=}},
> > > >
> response={numFound=0,numFoundExact=true,start=0,maxScore=0.0,docs=[]},
> > > > sort_values={}, debug={join={{!join from=image_id
> > > > to=image_record_id}+(+registry:registry.access.redhat.com
> > > > +repository:rhel6/rhel +published:T) +text:&rows=100&start=0={time=0,
> > > > fromSetSize=0, toSetSize=0, fromTermCount=0, fromTermTotalDf=0,
> > > > fromTermDirectCount=0, fromTermHits=0, fromTermHitsTotalDf=0,
> > > > toTermHits=0, toTermHitsTotalDf=0, toTermDirectCount=0,
> > > > smallSetsDeferred=0, toSetDocsAdded=0}}, timing={time=1.0,
> > > > prepare={time=0.0, query={time=0.0}, facet={time=0.0},
> > > > facet_module={time=0.0}, mlt={time=0.0}, highlight={time=0.0},
> > > > stats={time=0.0}, expand={time=0.0}, terms={time=0.0},
> > > > debug={time=0.0}}, process={time=0.0, query={time=0.0},
> > > > facet={time=0.0}, facet_module={time=0.0}, mlt={time=0.0},
> > > > highlight={time=0.0}, stats={time=0.0}, expand={time=0.0},
> > > > terms={time=0.0}, debug={time=0.0}"},
> > > > "
> > > >
> > >
> >
> http://localhost:8983/solr/rhcc_shard1_replica_n1/|http://localhost:7574/solr/rhcc_shard1_replica_n4/
> > > > ":{
> > > > "QTime":"0",
> > > > "ElapsedTime":"2",
> > > > "RequestPurpose":"GET_TOP_IDS,SET_TERM_STATS",
> > > > "NumFound":"0",
> > > >
Re: Solr 9.2 join query returns no results
Thanks for the help on this issue. It appears to be related to a
multi-shard collection. I will either work with a single shard collection
or change the way the data is indexed so as to not need a join query.
Thanks
On Thu, Jul 20, 2023 at 5:09 AM Mikhail Khludnev wrote:
> Hi there,
> If we are talking about SolrCloud. Up to 9.2 let to join "from" fully
> replicated single shard collection "to" a many shards collection.
> Starting from 9.3 Solr can join from many shards to many shards
> collections, but route fields should be the fields, which are used in join
> query.
> AffinityFactoryPlugin.withCollectionShards may be used to collocate
> corresponding collection shards.
>
> On Wed, Jul 19, 2023 at 11:03 PM Ron Haines wrote:
>
> > Well, that's interesting. I was under the impression that the 'join'
> > required the 'joining' docs to be located on the same shard. I'd
> actually
> > experienced fewer matches with the join, until I made sure the docs to
> > 'join to' were located on the same shard as the docs that were doing the
> > 'joining'. Maybe I'm confused...it would make my life a bit easier, if
> the
> > 'same shard' requirement were not necessary.
> >
> > Ron Haines
> >
> > On Wed, Jul 19, 2023 at 3:11 PM Charles Sanders
> > wrote:
> >
> > > Thanks for the reply Ron. You bring up a good question and something I
> > > need to consider. However in the solr 7 system the records are not on
> > the
> > > same shard. They are spread across 2 shards.
> > >
> > > Currently I'm working on a small test instance so I know exactly what
> > data
> > > I have. I will investigate the multiple shards question.
> > >
> > > Thanks
> > >
> > > On Wed, Jul 19, 2023 at 2:57 PM Ron Haines wrote:
> > >
> > > > After your migration, did your documents that should 'join' end up
> > > located
> > > > on the same shard? I believe that is a requirement.
> > > >
> > > > Ron Haines
> > > >
> > > > On Wed, Jul 19, 2023 at 1:04 PM Charles Sanders >
> > > > wrote:
> > > >
> > > > > Thanks again Mikhail for your reply. Yes there are absolutely
> > matches
> > > > for
> > > > > the subquery. This query should return 1 result. I'm working off
> a
> > > > small
> > > > > test collection.
> > > > >
> > > > > Here are the results of a debug.
> > > > >
> > > > > {
> > > > > "responseHeader":{
> > > > > "zkConnected":true,
> > > > > "status":0,
> > > > > "QTime":3,
> > > > > "params":{
> > > > > "q":"{!join from=image_id
> > > > > to=image_record_id}(registry:\"registry.access.redhat.com\" AND
> > > > > repository:\"rhel6/rhel\" AND published:true)&rows=100&start=0",
> > > > > "indent":"true",
> > > > > "q.op":"AND",
> > > > > "debugQuery":"true",
> > > > > "useParams":""}},
> > > > >
> > > > >
> > > >
> > >
> >
> "response":{"numFound":0,"start":0,"maxScore":0.0,"numFoundExact":true,"docs":[]
> > > > > },
> > > > > "debug":{
> > > > > "track":{
> > > > > "rid":"localhost-96",
> > > > > "EXECUTE_QUERY":{
> > > > > "
> > > > >
> > > >
> > >
> >
> http://localhost:7574/solr/rhcc_shard2_replica_n6/|http://localhost:8983/solr/rhcc_shard2_replica_n2/
> > > > > ":{
> > > > > "QTime":"1",
> > > > > "ElapsedTime":"2",
> > > > > "RequestPurpose":"GET_TOP_IDS,SET_TERM_STATS",
> > > > > "NumFound":"0",
> > > > > "Response":"{responseHeader={zkConnected=true, status=0,
> > > > > QTime=1, params={df=text, distrib=false, qt=select, debug=[false,
> > > > > timing, track], fl=[id, score], shards.purpose=16388, start=0,
> > > > > fsv=true, q.op=AND,
> > > > > shard.url=
> > > > >
> > > >
> > >
> >
> http://localhost:7574/solr/rhcc_shard2_replica_n6/|http://localhost:8983/solr/rhcc_shard2_replica_n2/
> > > > > ,
> > > > > rows=10, rid=localhost-96, version=2, q={!join from=image_id
> > > > > to=image_record_id}(registry:\"registry.access.redhat.com\" AND
> > > > > repository:\"rhel6/rhel\" AND published:true)&rows=100&start=0,
> > > > > omitHeader=false, requestPurpose=GET_TOP_IDS,SET_TERM_STATS,
> > > > > NOW=1689786155982, isShard=true, wt=javabin, debugQuery=false,
> > > > > useParams=}},
> > > > >
> > response={numFound=0,numFoundExact=true,start=0,maxScore=0.0,docs=[]},
> > > > > sort_values={}, debug={join={{!join from=image_id
> > > > > to=image_record_id}+(+registry:registry.access.redhat.com
> > > > > +repository:rhel6/rhel +published:T)
> +text:&rows=100&start=0={time=0,
> > > > > fromSetSize=0, toSetSize=0, fromTermCount=0, fromTermTotalDf=0,
> > > > > fromTermDirectCount=0, fromTermHits=0, fromTermHitsTotalDf=0,
> > > > > toTermHits=0, toTermHitsTotalDf=0, toTermDirectCount=0,
> > > > > smallSetsDeferred=0, toSetDocsAdded=0}}, timing={time=1.0,
> > > > > prepare={time=0.0, query={time=0.0}, facet={time=0.0},
> > > > > facet_module={time=0.0}, mlt={time=0.0}, highlight={time=0.0},
> > > > > stats={time=0.0}, expand={time=0.0}, terms={time=0.0},
> > > > > debug={time=0.0}}, process={time=0.0, query={
upgrade to 8.6 to 9.2
I performed upgrade from 8.6 to 9.2. Solr starts but the console no longer renders. Are there particular areas I should look at that would cause this? I have verified that /etc/default/solr.in.sh is configured correctly. This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.
Re: knn parser not working as expected
Thanks. If I move the knn parser syntax and value to fq param and make q as
*:*, it works and starts giving relevant results instantly.
curl -X POST -H "Content-Type: application/json" -d '{
"params": {
"q": "*:*", "fq": "{!knn f=dense_vector
topK=1}[0.06525743007659912,0.015727980062365532,0.003069591475650668,-0.016254400834441185,0.003478930564597249,-0.02475954219698906,0.020238326862454414,0.010255611501634121,0.05522076040506363,0.020635411143302917,0.05825875699520111,-0.05110647529363632,-0.04696913808584213,0.05991407483816147,-0.0003015052934642881,0.03625837340950966,-0.044656239449977875,-0.06582673639059067,-0.06842341274023056,-0.022927379235625267,0.048230838030576706,-0.12659960985183716,-0.019311215728521347,-0.04432906210422516,0.03600681200623512,0.010301047936081886,0.08415472507476807,0.04727723449468613,-0.0584205724298954,-0.045265913009643555,0.012285877950489521,0.0034233061596751213,-0.00982636958360672,-0.013216182589530945,-0.038882751017808914,-0.05872005969285965,-0.029350444674491882,0.04930287227034569,0.0022274062503129244,0.01728842593729496,-0.08762819767,-0.045831114053726196,0.072530098259449,0.03804686293005943,0.0021682181395590305,-0.05424166098237038,-0.004494055639952421,0.05843663960695267,0.058729417622089386,0.016252348199486732,0.0019551776349544525,-0.012190568260848522,-0.08235936611890793,-0.003848800901323557,0.028969185426831245,0.047798849642276764,-0.04074695333838463,-0.10175333172082901,0.06699151545763016,-0.06788542866706848,-0.01607389748096466,0.07294511049985886,0.007754810154438019,0.039606861770153046,0.07451225817203522,-0.02967959391212,0.014015864580869675,0.08055979013442993,0.0010412412229925394,0.13284511864185333,-0.013288799673318863,-0.05446619912981987,-0.03510258346796036,-0.12459734082221985,-0.017629574984312057,-0.04287091642618179,-0.019087448716163635,0.027409998700022697,-0.040427371859550476,-0.1713477075099945,-0.0035959691740572453,0.01750982739031315,-0.06452985852956772,0.10622204840183258,-0.06865541636943817,0.06022517383098602,0.03378240391612053,0.02320132404565811,0.02072194404900074,0.03390982002019882,0.0051648980006575584,0.05843415856361389,-0.07012602686882019,0.046549294143915176,0.005304296966642141,0.09183698892593384,0.060101959854364395,-0.031673040241003036,0.03126641735434532,0.10213921219110489,0.07624002546072006,-0.09995660930871964,0.03316718339920044,-0.040208760648965836,-0.016963355243206024,-0.01603076048195362,-0.00566966412588954,0.0570228286087513,0.006566803902387619,0.028397461399435997,-0.03737075999379158,-0.03357473015785217,-0.05060608312487602,0.0882791057229042,0.14182551205158234,0.01651209406554699,0.047577112913131714,-0.028357332572340965,-0.12397051602602005,0.03264006972312927,0.030581200495362282,0.025287700816988945,-0.08509892970323563,0.032361947267,-0.06732083112001419,0.0193667970597744,0.07096285372972488,-5.732041797079612e-33,0.033934514969587326,0.029480531811714172,-0.024119360372424126,0.03248802572488785,0.060654137283563614,-0.04089922457933426,-0.06845896691083908,0.015865417197346687,-0.03816983848810196,0.12768638134002686,-0.047979939728975296,0.01888129487633705,0.01966758444905281,-0.021792754530906677,-0.00209379056468606,-0.060791824012994766,0.07595516741275787,-0.05137578397989273,-0.020345840603113174,0.02730456180870533,-0.08421282470226288,0.0052170781418681145,-0.0396740548312664,0.013655638322234154,0.043763574212789536,0.0368662029504776,-0.021710995584726334,0.03603581339120865,0.04991370812058449,-0.007524373475462198,0.033250145614147186,0.0669487863779068,-0.012807670049369335,-0.08904062211513519,-0.04803512617945671,-0.0461772084236145,0.018098553642630577,0.01096352282911539,0.0617918036878109,0.014066621661186218,-0.03305654972791672,-0.08129353821277618,-0.025270603597164154,0.03537251427769661,0.06029881164431572,0.06169535592198372,0.0355769582092762,0.03534447401762009,-0.047377053648233414,0.053076375275850296,-0.019250469282269478,-0.03837420791387558,-0.00834209006279707,0.031550273299217224,0.004682184662669897,0.0590718574821949,0.0326957181096077,-0.041941817849874496,-0.04179370403289795,-0.010403091087937355,0.11914990842342377,-0.049126915633678436,0.015761952847242355,-0.012162514962255955,-0.05942496284842491,0.04794146493077278,-0.06834675371646881,-0.03294386342167854,0.02242257259786129,0.0774146020412445,-0.1095564718246,0.023828692734241486,0.054935190826654434,0.0202674251049757,-0.057155776768922806,-0.009578827768564224,-0.051850661635398865,0.09117215871810913,-0.07315851002931595,-0.0019339871359989047,-0.05835318937897682,-0.058747921139001846,-0.05519327148795128,-0.014699703082442284,-0.0020833320450037718,-0.05721793323755264,0.055632084608078,0.006448595318943262,0.0034963993821293116,-0.031087594106793404,-0.09541762620210648,0.03679275885224342,-0.012651922181248665,-0.038976479321718216,-0.013171667233109474,3.5278054836115376e-33,-0.057552605867385864,0.15763822197914124,0.0087867574766278
