Sv: issue with f..qf in solr 9.4
Just for clarification: this works in 9.3 (like 9.1.1), and not 9.4.
Thanks for reacting to this, I appreciate it. I'll keep an eye on SOLR-16916.
Regards,
/Noah
--
Noah Torp-Smith (n...@dbc.dk)
Fra: Chris Hostetter
Sendt: 24. oktober 2023 19:32
Til: users@solr.apache.org
Emne: Re: issue with f..qf in solr 9.4
(thank you for using an example query that works against the techproducts
example! .. makes it very easy to reproduce)
At the *qparser* level, what you are doing is still working in 9.4..
hossman@slate:~/lucene/solr [j11] [tags/releases/solr/9.4.0] $ curl -sS
'http://localhost:8983/solr/techproducts/select?omitHeader=true&fl=id&defType=edismax&q=all:belkin&f.all.qf=compName_s+id+address_s'
{
"response":{
"numFound":1,
"start":0,
"numFoundExact":true,
"docs":[{
"id":"belkin"
}]
}
}
...the error you are getting only seems to happen when using the JSON
Request API. (as in your email)
Below is the ERROR log w/stacktrace that i get when i try your request
(FWIW: including solr error log messages in email questions about request
errors is always a great way to help devs answer your questions)
The main thing that jumps out at me is that the edismax parser isn't
involved -- it appears to have decided the LuceneQParser should be used?
Which makes me speculate that something broke in how the "params block" is
parsed when using the JSON Request API?
Skimming CHANGES.txt looking for mentions of JSON Request API changes led
me to this...
* SOLR-16916: Use of the JSON Query DSL should ignore the defType parameter
(Christina Chortaria, Max Kadel, Ryan Laddusaw, Jane Sandberg, David Smiley)
...i'm having a hard time wrapping my head around the jira comments ...
the CHANGES.txt entry is written like the point of the issue was to
intentionally 'ignore' defType in the JSON Query API, but the comments in
the Jira read like using 'defType' was broken in 7.2 and this issue
"fixed" it so it would work again starting in 9.4? ... the commit itself
only shows testing what happens if 'defType=edismax' is defined as a
request handler default.
I'm not really sure what's going on here or what the intent was ... i've
posted a comment in the jira...
https://issues.apache.org/jira/browse/SOLR-16916
2023-10-24 16:57:07.705 ERROR (qtp1535026957-24) [ x:techproducts
t:localhost-24] o.a.s.h.RequestHandlerBase Client exception =>
org.apache.solr.common.SolrException: undefined field all
at
org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1478)
org.apache.solr.common.SolrException: undefined field all
at
org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1478)
~[?:?]
at
org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:500)
~[?:?]
at
org.apache.lucene.analysis.DelegatingAnalyzerWrapper$DelegatingReuseStrategy.getReusableComponents(DelegatingAnalyzerWrapper.java:83)
~[?:?]
at
org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:184) ~[?:?]
at
org.apache.lucene.util.QueryBuilder.createFieldQuery(QueryBuilder.java:256)
~[?:?]
at
org.apache.solr.parser.SolrQueryParserBase.newFieldQuery(SolrQueryParserBase.java:527)
~[?:?]
at
org.apache.solr.parser.QueryParser.newFieldQuery(QueryParser.java:68)
~[?:?]
at
org.apache.solr.parser.SolrQueryParserBase.getFieldQuery(SolrQueryParserBase.java:1140)
~[?:?]
at
org.apache.solr.parser.SolrQueryParserBase.handleBareTokenQuery(SolrQueryParserBase.java:856)
~[?:?]
at org.apache.solr.parser.QueryParser.Term(QueryParser.java:454)
~[?:?]
at org.apache.solr.parser.QueryParser.Clause(QueryParser.java:293)
~[?:?]
at org.apache.solr.parser.QueryParser.Query(QueryParser.java:173)
~[?:?]
at
org.apache.solr.parser.QueryParser.TopLevelQuery(QueryParser.java:143)
~[?:?]
at
org.apache.solr.parser.SolrQueryParserBase.parse(SolrQueryParserBase.java:274)
~[?:?]
at
org.apache.solr.search.LuceneQParser.parse(LuceneQParser.java:51) ~[?:?]
at org.apache.solr.search.QParser.getQuery(QParser.java:188)
~[?:?]
at
org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:172)
~[?:?]
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:431)
~[?:?]
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:226)
~[?:?]
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2901)
~[?:?]
: Date: Tue, 24 Oct 2023 12:01:20 +
: From: Noah Torp-Smith
: Reply-To: users@solr.apache.org
: To: "users@solr.apache.org"
: Subject: issue with f..qf in solr 9.4
:
: Hello,
:
: When I spin up the techproducts example in solr 9.1.1, I am able to send this
to the /query endpoint and get a reasonable response:
:
: {
: "query": "+all:belkin",
: "fields": "id compName_s",
: "offset": 0,
: "limit": 10,
:
SolR 9.4.0 and zookeepr version
Hi , which version of zookeeper is certified / suggested for solr 9.4.0 ? Thank you and regards -- Isabella Trevisan
Retry strategy in Solrj 8
When using CloudSolrClient builder, the default HttpClient appears to be created with 3 retries. However, it looks like BaseCloudSolrClient already retries 5 times so we end up with retries on top of retries. Is there a reason to keep both of these retries configured or is it safe to disable the underlying HttpClient retries and still maintain reliability?
deleting documents based on matching value of the field
Hi, Could someone provide me a few examples on the syntax for delete a few documents based on matching (or actually partial matching) of a value of a a field. for example, delete all document is: *:* I have some of the documents that has a kv pairs liked: "my_field":"value1" "my_field":"value2" "my_field":"value3" only the document with value2 needs to be deleted. What is the syntax to delete by partial matching value of a field? Thanks.
Re: Solr Heap usage
Thank you Rajani and Arrieta, I will read the link. Ishan also reached out to me earlier on with some insights , and am analyzing them as well. I did take a heap dump using VisualVM and analyzing the details, after loading through the visualVM tool. thanks Surya On Tue, Oct 24, 2023 at 9:18 PM Arrieta, Alejandro < aarri...@perrinsoftware.com> wrote: > Hello, > > this link may help you, it has examples: > https://risdenk.github.io/2017/12/18/ambari-infra-solr-ranger.html > > Kind Regards, > Alejandro Arrieta > > On Tue, Oct 24, 2023 at 9:18 PM rajani m wrote: > > > When you see memory filling up on the visualvm gui, request a heap dump. > > The heap dump file gets stored on the node's /tmp location. Use the > eclipse > > memory analyzer tool (MAT) to load the downloaded ".dmp" file, the MAT > > analyzes the heap and generates a report of objects on the heap. > > > > On Mon, Oct 23, 2023 at 1:48 PM Surya R wrote: > > > > > Hi. We have a solr index with around 2.5M documents , running solr > 8.11 . > > > We have bunch of fields that we faceet/sort/highlight on. Heap is 8GB, > > out > > > of 12 GB RAM > > > > > > These fields were not defined with docValues=true due to some legacy > > design > > > issues , but if we were to rewrite the app, we would definitely have > the > > > docvalues enabled for those fields that participate in > > > facet/sort/highlight. > > > > > > Problem: > > > I do see garbage collection ( G1GC) kicking in when the traffic is > > moderate > > > to high, with a "pause full" event that takes around 2seconds to > > complete. > > > This event happens more often when the traffic is high. > > > > > > I see this GC event happening when the humongous regions go beyond > 1000. > > > The humongous region size is 4M > > > > > > I took a heap dump and analyzed with VisualVM, but I couldnt pinpoint > > what > > > objects are using the 'humougous region' . > > > > > > My question is, using the VisualVM, is it possible to pinpoint which / > > what > > > objects are using the 'humongous region' ? > > > > > > If for example I have an solr indexed field called "answerSnippet" , > if > > > that is the culprit, Is it possible to see the field answerSnippet in > the > > > VisualVM heap dump output? > > > > > > All i see is byte[] with 55% heap occupancy, but dont know what objects > > > constitute that byte[] > > > > > > Can anybody throw some light on this ? I really appreciate the help. > > > > > > > > > thanks > > > -- Surya > > > > > > -- -- Surya
Question on "useColdSearcher"
Hello Solr Experts, I have a question on SOLR core behavior I noticed , post 8.x upgrade. We use the simple leader/follower design which holds around 2 million documents per core. Say the name of the core is "customers" which holds all the 2 million customer details, when I restart the solr core, and when i query the customers core, it takes around 30 seconds to respond for the first time. But when i see this message "registered customers searcher", something in these lines in the solr.log, the queries start responding. I started playing with this setting " *false* " in the solr config.xml, when i made it "*true*", which implies to use the cold searcher, I get response immediately after a core restart. But if i leave the default, which is "*false*" it takes a long time. So my question is, internally did something change from Solr 6.x to Solr 8.x onwards ? Even in solr 6 the default was "false" but the core responded immediately after a restart. I noticed this behavior only after upgrade to Solr 8.x Changing this setting to "true" in solr 8 solrconfig.xml, made the core respond immediately. Is that ok to turn on to "true" ? Which means, it could be taking a hit serving queries NOT from the cache. Note: I also made the warmupcount to ZERO in the solrconfig, still I dont see a difference with the detault settings. Can anyone please throw some light on this setting/behavior ? thanks Surya
Re: deleting documents based on matching value of the field
Hello Vince, we use something in this format to delete by "id", basically try changing the query to a pattern and test in your sandbox. We use leader/follower architecture. curl "http://solr-host:8983/solr/core/deletes?commit=true"; -H "Content-Type:text/xml" --data-binary 'id:12233' In case the id is negative, you have to escape the minus sign. We have a shell script that deletes ids in bulk once in a while and the above statement works perfectly fine. Our requirement is to delete by id, but I am confident the pattern would work too. Surya On Wed, Oct 25, 2023 at 10:44 AM Vince McMahon < sippingonesandze...@gmail.com> wrote: > Hi, > > Could someone provide me a few examples on the syntax for delete a few > documents based on matching (or actually partial matching) of a value of a > a field. > > for example, delete all document is: *:* > > I have some of the documents that has a kv pairs liked: > "my_field":"value1" > "my_field":"value2" > "my_field":"value3" > > only the document with value2 needs to be deleted. > > What is the syntax to delete by partial matching value of a field? > > Thanks. > -- -- Surya
Re: SolR 9.4.0 and zookeepr version
On 10/25/23 05:45, Isabella Trevisan wrote: which version of zookeeper is certified / suggested for solr 9.4.0 ? Solr 9.4.0 comes with the ZK 3.9.0 client. The ZK project guarantees that 3.9.x will work with 3.8.0 through 3.10.x. A wider version range might actually work, that is just their guarantee. The latest stable release of ZK right now is 3.8.3, that is the version I would recommend for the server. Thanks, Shawn
Re: Question on "useColdSearcher"
On 10/25/23 09:05, Surya R wrote: I started playing with this setting " *false* " in the solr config.xml, when i made it "*true*", which implies to use the cold searcher, I get response immediately after a core restart. But if i leave the default, which is "*false*" it takes a long time. It sounds like you probably have a bunch of slow queries in a firstSearcher listener config. Maybe you did not have those slow queries in the 6.x config, so the searcher was available right away. Or maybe the 6.x config was different in some other way that made those warming queries complete quickly. Generally, I would probably run with useColdSearcher set to true, even you have a number of Solr servers behind a load balancer. That way the searcher will be usable immediately, even if it is not warmed yet by the queries in the firstSearcher listener. Thanks, Shawn
question on docker example from guide
I am trying to follow the example on running solr via docker. the example is from https://solr.apache.org/guide/solr/latest/deployment-guide/solr-in-docker.html what is the last two part of the command highlighted. docker run -d -v "$PWD/indata:/var/solr" -p 8983:8983 --name my_solr *solr solr-precreate* how to fix the problem when I run the above command, it will exit 1. root@abc:/mnt/c/Users/abcd/Downloads/wsl# docker run -d -v "$PWD/indata:/var/solr" -p 8983:8983 --name my_solr solr solr-precreate 0608bf3e16cd2f7dd1ec00db7e04f76e9fd8499c47eb87033d781d146c267f5e root@abc:/mnt/c/Users/abcd/Downloads/wsl# docker container ls -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 0608bf3e16cd solr "docker-entrypoint.s…" 5 seconds ago Exited (1) 5 seconds ago my_solr
Re: question on docker example from guide
ok, so those are commands. I can't find them. I'm running into many problems to get any of the examples to start. Is there a clean version of the docker to learn solr? Is there any YouTube videos can help me jump start? My primary job is to dev an app and I'm in a tight deadline. Running solr will not help me keeping my job. Please help.
RE: question on docker example from guide
This single simple command should work and you should be able to see an empty Solr running if there isn’t any problem with your Docker: docker run -p 8983:8983 solr:9.1 What OS and docker version are you using? Sent from Mail for Windows From: Vince McMahon Sent: Thursday, October 26, 2023 5:43 AM To: users@solr.apache.org Subject: Re: question on docker example from guide ok, so those are commands. I can't find them. I'm running into many problems to get any of the examples to start. Is there a clean version of the docker to learn solr? Is there any YouTube videos can help me jump start? My primary job is to dev an app and I'm in a tight deadline. Running solr will not help me keeping my job. Please help.
