Re: Highlighting: echo matching query text?

[Alessandro Benedetti]

Hi Stephen,
so you want to show in the UI the original token in the inverted index that
caused the match?
This relies on the text analysis configured Solr side and it would be far
from being intuitive to your final user.

With stemming you made the perfect example:
The stem of the term potentially is not even a term at all in the language
associated with the field.
Not sure if showing the token from the index would help at all from the
explainability side (in comparison to now showing it).
Unless the user is informed about the entire text analysis chain(including
the query time text analysis and he/she understands that), I guess showing
the token will just complicate even more the situation:
*e.g.*
q: arguing
D1 : argues (Match argu)

Without knowing what index and query time means, what stemming means, and
that stemming was applied at query time and at indexing time, I am not
entirely sure it's going to add that much to the final user experience.

Aside from my personal observations, I don't think there's anything in the
super stratified highlighting module, so you should be able to pick some of
the implementations and customize it.

Cheers
--
Alessandro Benedetti
Apache Lucene/Solr Committer
Director, R&D Software Engineer, Search Consultant

www.sease.io


On Tue, 30 Nov 2021 at 19:00, Stephen Lewis Bianamara <
stephen.bianam...@gmail.com> wrote:

> Hi SOLR Community,
>
> I am investigating some different options with highlighting, and one
> feature I wanted to build would require matching a highlighted match back
> to the original matchin token. I couldn't find a way to do that in the
> documentation, so I'm guessing that it doesn't exist yet. The application
> for this would be to leverage solr to understand the query -> field
> matching with many field types with varying matching rules.
>
> The simplest example of what I'd like would be something like this in
> english: A document like
>
>{ "id": "test", "comment_en": "we like dogs" }
>
> ..and a query like "dog OR cat". I'd like highlighting to be able to return
> something like this:
>
> {"highlighting": {
>"text_en": {
>"comment_en": ["we like *dogs*" ]}}}
>
> So the essence is that I don't need to know anything about english matching
> rules (in this case plurals) outside of solr to know how it came to the
> conclusion that this document was a match.
>
> Has anyone come up with a solution to this before? Does anyone know an
> existing feature request for this if not?
>
> Thanks!
> Stephen
>

Is the showItems argument for fieldValueCache used for anything?

[Andy Lester]

It looks to me like the showItems argument for the fieldValueCache is not used. 
 I can’t find any documentation of it, although it was mentioned in the 
changelog for v8.1.0.

I looked through the source and I can’t see where the value is used. It gets a 
default value, but never seems to be used, based on my understanding of the 
source code.

$ grep -w showItems -R .
./solr/core/src/java/org/apache/solr/core/SolrConfig.java:
args.put("showItems", "-1");
./solr/core/src/java/org/apache/solr/search/SolrCache.java:  String 
SHOW_ITEMS_PARAM = "showItems";
./solr/core/src/test-files/solr/configsets/exitable-directory/conf/solrconfig.xml:
showItems="0" />
./solr/server/solr/configsets/_default/conf/solrconfig.xml: 
   showItems="32" />
./solr/server/solr/configsets/sample_techproducts_configs/conf/solrconfig.xml:  
  showItems="32" />
./solr/CHANGES.txt:* SOLR-13432: Add .toString methods to BitDocSet and 
SortedIntDocSet so that enabling "showItems" on the filter caches

$ grep -w SHOW_ITEMS_PARAM -R .
./solr/core/src/java/org/apache/solr/search/SolrCache.java:  String 
SHOW_ITEMS_PARAM = "showItems”;

Is it a leftover from long ago?  If so, I’ll put in a patch to remove its last 
vestiges from the source and from the sample config files.

Thanks,
Andy

Re: Is the showItems argument for fieldValueCache used for anything?

[Mikhail Khludnev]

Hi, Andy.

It seems it's gone https://issues.apache.org/jira/browse/SOLR-15762
I will miss showItems, it was really useful a long ago.

On Fri, Dec 3, 2021 at 6:12 PM Andy Lester  wrote:

> It looks to me like the showItems argument for the fieldValueCache is not
> used.  I can’t find any documentation of it, although it was mentioned in
> the changelog for v8.1.0.
>
> I looked through the source and I can’t see where the value is used. It
> gets a default value, but never seems to be used, based on my understanding
> of the source code.
>
> $ grep -w showItems -R .
> ./solr/core/src/java/org/apache/solr/core/SolrConfig.java:
> args.put("showItems", "-1");
> ./solr/core/src/java/org/apache/solr/search/SolrCache.java:  String
> SHOW_ITEMS_PARAM = "showItems";
> ./solr/core/src/test-files/solr/configsets/exitable-directory/conf/solrconfig.xml:
>   showItems="0" />
> ./solr/server/solr/configsets/_default/conf/solrconfig.xml:
> showItems="32" />
> ./solr/server/solr/configsets/sample_techproducts_configs/conf/solrconfig.xml:
>   showItems="32" />
> ./solr/CHANGES.txt:* SOLR-13432: Add .toString methods to BitDocSet and
> SortedIntDocSet so that enabling "showItems" on the filter caches
>
> $ grep -w SHOW_ITEMS_PARAM -R .
> ./solr/core/src/java/org/apache/solr/search/SolrCache.java:  String
> SHOW_ITEMS_PARAM = "showItems”;
>
> Is it a leftover from long ago?  If so, I’ll put in a patch to remove its
> last vestiges from the source and from the sample config files.
>
> Thanks,
> Andy



-- 
Sincerely yours
Mikhail Khludnev

Re: Is the showItems argument for fieldValueCache used for anything?

[Andy Lester]

> On Dec 3, 2021, at 9:39 AM, Mikhail Khludnev  wrote:
> 
> It seems it's gone https://issues.apache.org/jira/browse/SOLR-15762 
> 
> I will miss showItems, it was really useful a long ago.

I don’t understand how that ticket relates.  I don’t see any mention of 
showitems in it.

Andy

Not able to write solr logs in json format

[Kakolu, Karthik]

Hi,

Trying to write solr.log in json format but unsuccessful.
Any pointers on the same would be appreciated.

Also JsonLayout in 8.10 was not able to get it working as suggested in the solr 
document.


  
  


  
  


Thanks
Karthik

Re: Not able to write solr logs in json format

[Andy Lester]

> On Dec 3, 2021, at 9:55 AM, Kakolu, Karthik  
> wrote:
> 
> Trying to write solr.log in json format but unsuccessful.


How exactly are you unsuccessful?

Do you get logs but they aren’t in JSON as expected? If so, what format are 
they in?

Are you able to write logs in other formats than JSON?

Do you get any logs at all?

Re: Is the showItems argument for fieldValueCache used for anything?

[Mikhail Khludnev]

The first comment mentioned FastLRUCache deprecation, and IIRC it was the
only impl which supported showItems.
However, I can't find a ticket dedicated for its removal. So, forget.

On Fri, Dec 3, 2021 at 6:42 PM Andy Lester  wrote:

>
>
> > On Dec 3, 2021, at 9:39 AM, Mikhail Khludnev  wrote:
> >
> > It seems it's gone https://issues.apache.org/jira/browse/SOLR-15762 <
> https://issues.apache.org/jira/browse/SOLR-15762>
> > I will miss showItems, it was really useful a long ago.
>
> I don’t understand how that ticket relates.  I don’t see any mention of
> showitems in it.
>
> Andy



-- 
Sincerely yours
Mikhail Khludnev

Stream Query result Cacheable ?

[sambasivarao giddaluri]

Hi All,

Wanted to check if the result of stream query is internally cached by solr .
example , if we hit the same query with /stream will the response be fast ?

because if we run the same query /query or /select the response if faster
the next time we hit .

Regards
sam

How to enforce client hostname verification when using mutual TLS in Solr?

[S G]

Hello,

Does anyone know how to enforce client hostname verification when using
mutual TLS in Solr?
Without that feature, any host with a certificated could connect to Solr
using m-TLS and there is no authentication (the primary use-case of m-TLS).

Also, if Solr is secured by m-TLS, is there a way to extend the same to
zookeeper as well ?

Thanks,

SolrCloud S3 Backup Status doesn't return the correct value for IndexSizeMB

[Ricardo Ruiz]

Hi!
I'm trying to get the Index size of my SolrCloud backups, but the problem
is that It doesn't matter the size of my backup, the value for
"indexSizeMB" is always 0.0, even if the STATUS is completed and the backup
files are saved successfully in S3.

Is this a bug or am I doing something wrong?

The steps I do are:
- Backup my collection with an async Id
- Check the status of my request with REQUESTSTATUS
 until
it has a completed status.

This is a backup without any documents indexed.
{
"responseHeader": {
"status": 0,
"QTime": 4
},
"success": {
...
},
...
"response": [
"collection",
"collectionName",
"numShards",
2,
"backupId",
0,
"indexVersion",
"8.10.1",
"startTime",
"2021-12-04T06:10:25.198262Z",
"indexSizeMB",
0.0
],
"status": {
"state": "completed",
"msg": "found [backup12041510] in completed tasks"
}
}

This is a backup with 7000+ documents indexed, around 40MB according to S3
{
"responseHeader": {
"status": 0,
"QTime": 4
},
"success": {
...
},
...
"response": [
"collection",
"livedoornews",
"numShards",
2,
"backupId",
3,
"indexVersion",
"8.10.1",
"startTime",
"2021-12-04T06:37:32.563195Z",
"indexSizeMB",
0.0
],
"status": {
"state": "completed",
"msg": "found [backup12041537] in completed tasks"
}
}

Thank you in advance!!
Richard