Re: Question on solr filter syntax

[Mikhail Khludnev]

Hello, Yash.
The syntax you come up with makes sense. It's important to don't prepend it
with space; probably bracket may be redundant; you can also resolve syntax
ambiguity with local params.
fq={!cache=false v="filter(planName:all_features) OR zipCode:1234"}
debugQuery=true shows how it's parsed.
Another esoteric options:
https://solr.apache.org/guide/solr/latest/query-guide/block-join-query-parser.html#all-parents-syntax
fq={!cache=false v=$refq}&refq=zipCode:1234 {!parent
which=planName:all_features}


On Thu, May 25, 2023 at 8:04 PM Wei  wrote:

> Send on behalf of my colleague as his email seems not showing up in the
> list.
>
> Hi Team,
>
>
> I was reading about special filter syntax in Solr documentation -
>
>
> https://solr.apache.org/guide/8_3/the-standard-query-parser.html#differences-between-lucenes-classic-query-parser-and-solrs-standard-query-parser
>
>
> Support for a special filter(…) syntax to indicate that some query
> clauses should be cached in the filter cache (as a constant score
> boolean query). This allows sub-queries to be cached and re-used in
> other queries. For example inStock:true will be cached and re-used in
> all three of the queries below:
>
> q=features:songs OR filter(inStock:true)
>
>
> My question is how do I disable cache for part of OR filter query and
> enable for part of OR filter query?
>
>
> For e.g. - Current FQ -
>
> fq: planName:all_features OR zipCode:1234 (in this case entire FQ is
> cached)
>
>
> I don't want to cache the entire FQ given it's very unique , but I
> want to cache the first part (plan_name:all_features) to improve
> performance . How do I achieve this?
>
>
> My understanding is the following should work? Is this correct?
>
> fq: {!cache=false}(filter(planName:all_features) OR zipCode:1234)
>
>
> Will the above syntax result in planName:all_features to be cached in
> Filter Cache? Otherwise how can I achieve this?
>
>
> I am on Solr 8.4
>
>
> Thanks,
>
> Yash
>


-- 
Sincerely yours
Mikhail Khludnev
https://t.me/MUST_SEARCH
A caveat: Cyrillic!

Re: Vector Search Parser(!vp) : calculate cosine

[Alessandro Benedetti]

Hi Rajani,
the issues you mentioned are very old and superseded by more recent works
in the vector-based search domain.
The  'knn' query parser that Kumar detailed is the official way of doing
vector search in Solr.
Function queries for vector similarities and the possibility of better
Hybrid search with Learning To Rank and vector-similarity features are
coming soon!

Some blogs that could help:
- https://sease.io/2023/01/apache-solr-neural-search-tutorial.html
- https://sease.io/2022/01/apache-solr-neural-search.html
- https://sease.io/2022/01/apache-solr-neural-search-knn-benchmark.html

Cheers

--
*Alessandro Benedetti*
Director @ Sease Ltd.
*Apache Lucene/Solr Committer*
*Apache Solr PMC Member*

e-mail: a.benede...@sease.io


*Sease* - Information Retrieval Applied
Consulting | Training | Open Source

Website: Sease.io 
LinkedIn  | Twitter
 | Youtube
 | Github



On Thu, 25 May 2023 at 22:12, rajani m  wrote:

> That also invokes knn, correct? I just need the vector math response, a
> cosine value of the query vector and document vector. For example, the
> streaming api endpoint vector-math.html#dot-product-and-cosine-similarity
> <
> https://solr.apache.org/guide/7_5/vector-math.html#dot-product-and-cosine-similarity
> >
> looks like one however it is a streaming api, it would have been nice if
> this was also supported by standard query parser or a function query.
>
>
>
> On Thu, May 25, 2023 at 3:34 PM kumar gaurav  wrote:
>
> > HI Rajani
> >
> > Read here :
> >
> >
> https://solr.apache.org/guide/solr/latest/query-guide/dense-vector-search.html
> >
> > The knn query parser can be used .
> >
> > &q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
> >
> >
> > Thanks
> > Kumar Gaurav
> >
> >
> >
> > On Fri, 26 May 2023 at 00:14, Rajani Maski 
> wrote:
> >
> > > Hi Solr Users,
> > >
> > >Is there a vector search parser that allows computing distance
> > between a
> > > document's vector field and query vector passed as query param?  The
> > jiras
> > > SOLR-14397 
> > SOLR-12890
> > >   suggests vector
> > search
> > > parser "!vp" and the other "cosine" as functions but I couldn't get it
> to
> > > work on solr 9.1.1 version, is this supported yet, one of the jira
> state
> > is
> > > still "open"?
> > >
> > > What are other alternatives to calculate cosine between vector fields
> in
> > > solr vs query vector sent as query param?
> > >
> > >
> > > Thanks,
> > > Rajani
> > >
> >
>

Re: Deleting document on wrong shard?

[Chris Hostetter]

: Hmm. That doesn’t seem consistent with the format change added in 
: SOLR-5980, where each ID can have a _route_ attribute.
: 
: { "id":"ID" , "_route_":"route”}

I think you may be getting confused between the syntactic sugar of sending 
a single delete command as the entire JSON payload + the "syntactic sugar" 
of sending multiple IDs in a single delete command vs the full JSON 
update command syntax,

this...

{ "delete":"x" }

...is just syntactic sugar for...

{ 
  "delete":{"id":"x"}
}


While this...


{ "delete":["a","b","c"] }

...is just syntactic sugar for...

{ 
  "delete":{"id":"a"},
  "delete":{"id":"b"},
  "delete":{"id":"c"}
}

But when using the full syntax multiple commands can be included in a 
single request, and each command can include options (like _version_ and 
_route_)...

{ 
  "delete": { "id":"x", "_route_":"a" }, 
  "delete": { "id":"y", "_route_":"b" },
  "add": { ... } 
}

(I'm guessing the docs would make more sense if the note about including 
"_version_" with deletes was moved up by the example of the full command 
syntax, and the examples of the "simple delete-by-id" syntactic sugar were 
moved below that ... if anyone wnats to submit a patch)


-Hoss
http://www.lucidworks.com/

Re: Deleting document on wrong shard?

[Walter Underwood]

I wouldn’t call it semantic sugar, more like a different compact format. The 
compact format also avoids duplicate keys, which are legal in JSON but hard to 
create in some systems.

The XML command format is working fine.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On May 26, 2023, at 10:47 AM, Chris Hostetter  
> wrote:
> 
> 
> : Hmm. That doesn’t seem consistent with the format change added in 
> : SOLR-5980, where each ID can have a _route_ attribute.
> : 
> : { "id":"ID" , "_route_":"route”}
> 
> I think you may be getting confused between the syntactic sugar of sending 
> a single delete command as the entire JSON payload + the "syntactic sugar" 
> of sending multiple IDs in a single delete command vs the full JSON 
> update command syntax,
> 
> this...
> 
> { "delete":"x" }
> 
> ...is just syntactic sugar for...
> 
> { 
>  "delete":{"id":"x"}
> }
> 
> 
> While this...
> 
> 
> { "delete":["a","b","c"] }
> 
> ...is just syntactic sugar for...
> 
> { 
>  "delete":{"id":"a"},
>  "delete":{"id":"b"},
>  "delete":{"id":"c"}
> }
> 
> But when using the full syntax multiple commands can be included in a 
> single request, and each command can include options (like _version_ and 
> _route_)...
> 
> { 
>  "delete": { "id":"x", "_route_":"a" }, 
>  "delete": { "id":"y", "_route_":"b" },
>  "add": { ... } 
> }
> 
> (I'm guessing the docs would make more sense if the note about including 
> "_version_" with deletes was moved up by the example of the full command 
> syntax, and the examples of the "simple delete-by-id" syntactic sugar were 
> moved below that ... if anyone wnats to submit a patch)
> 
> 
> -Hoss
> http://www.lucidworks.com/

Re: Deleting document on wrong shard?

[dmitri maziuk]

On 2023-05-26 1:19 PM, Walter Underwood wrote:

I wouldn’t call it semantic sugar, more like a different compact format. The 
compact format also avoids duplicate keys, which are legal in JSON but hard to 
create in some systems.


It's not they're hard to create, it's that you can't create a data 
structure (incl. a JavaScript Object: the "JSO" in JSON) with them -- 
and then marshall it into JSON.


The only way to create a Solr-style "JSON" string is to mash it by hand. 
String mashing is considered a bad idea in general and especially bad 
when proper facilities are available in your language.


OT $.02
Dima

Re: Question on Solr filter syntax

[Chris Hostetter]

: For e.g. - Current FQ -
: fq: planName:all_features OR zipCode:1234 (in this case entire FQ is cached)
: 
: I don't want to cache the entire FQ given it's very unique , but I
: want to cache the first part (plan_name:all_features) to improve
: performance . How do I achieve this?
: 
: 
: My understanding is the following should work? Is this correct?
: 
: fq: {!cache=false}(filter(planName:all_features) OR zipCode:1234)

Correct.

In small scale testing you can see it work by looking at your cache 
metrics -- I've included a `bin/solr -e techproducts` example below my 
sig...


-Hoss
http://www.lucidworks.com/




hossman@slate:~$ curl -sS 
'http://localhost:8983/solr/admin/metrics?omitHeader=true&key=solr.core.techproducts:CACHE.searcher.filterCache'
{
  "metrics":{
"solr.core.techproducts:CACHE.searcher.filterCache":{
  "lookups":0,
  "hits":0,
  "hitratio":1.0,
  "inserts":0,
  "evictions":0,
  "size":0,
  "warmupTime":0,
  "ramBytesUsed":448,
  "maxRamMB":-1,
  "cumulative_lookups":0,
  "cumulative_hits":0,
  "cumulative_hitratio":1.0,
  "cumulative_inserts":0,
  "cumulative_evictions":0}}}
hossman@slate:~$ curl --globoff -sS 
'http://localhost:8983/solr/techproducts/select?q=*:*&rows=0&omitHeader=true&fq={!cache=false}(filter(inStock:true)+OR+id:IW-02)'
{
  "response":{"numFound":18,"start":0,"numFoundExact":true,"docs":[]
  }}
hossman@slate:~$ curl -sS 
'http://localhost:8983/solr/admin/metrics?omitHeader=true&key=solr.core.techproducts:CACHE.searcher.filterCache'
{
  "metrics":{
"solr.core.techproducts:CACHE.searcher.filterCache":{
  "lookups":1,
  "hits":0,
  "hitratio":0.0,
  "inserts":1,
  "evictions":0,
  "size":1,
  "warmupTime":0,
  "ramBytesUsed":1744,
  "maxRamMB":-1,
  "cumulative_lookups":1,
  "cumulative_hits":0,
  "cumulative_hitratio":0.0,
  "cumulative_inserts":1,
  "cumulative_evictions":0}}}
hossman@slate:~$ curl --globoff -sS 
'http://localhost:8983/solr/techproducts/select?q=*:*&rows=0&omitHeader=true&fq={!cache=false}(filter(inStock:true)+OR+id:GB18030TEST)'
{
  "response":{"numFound":17,"start":0,"numFoundExact":true,"docs":[]
  }}
hossman@slate:~$ curl -sS 
'http://localhost:8983/solr/admin/metrics?omitHeader=true&key=solr.core.techproducts:CACHE.searcher.filterCache'
{
  "metrics":{
"solr.core.techproducts:CACHE.searcher.filterCache":{
  "lookups":2,
  "hits":1,
  "hitratio":0.5,
  "inserts":1,
  "evictions":0,
  "size":1,
  "warmupTime":0,
  "ramBytesUsed":1744,
  "maxRamMB":-1,
  "cumulative_lookups":2,
  "cumulative_hits":1,
  "cumulative_hitratio":0.5,
  "cumulative_inserts":1,
  "cumulative_evictions":0}}}