Solr partial highlighting
Hi, I’m facing issue when it comes to partial highlighting in the response. I have enabled the highlighting in schema.xml. For example there is a document starts with "Test", currently when I search for "Test", the document's "Test" part gets highlighted but when I search "Tes" it doesn't get highlighted. Could you help me find the issue? Thanks Regards, Thiviyaraj
streaming expressions - sharding memory usage
Hi, I am working currently on implementing sharding on current Solr Cloud Cluster. Main idea is to be able to scale horizontally. At the moment, without sharding we have all collections sitting on all servers. We have as well pretty heavy streaming expressions returning many ids. Average of 300,000 ids to join. After doing sharding I see a huge increase on CPU and memory usage. Making queries way slower comparing sharding to not sharding. I guess that's expected bacuase the joins need to send data across servers over network. Any thoughs on best practices here. I guess a possible approach is to split shards in more. Regards Sergio
deleteById for multiple ids with route parameter
Hi all, We are using solr cloud (version 8.10). We are using deleteById which takes the id and route value to delete the id from the specific shard. deleteById takes only one value of id. I want to ask if there is any way to send multiple ids to deleteById with route parameter and if can we send multiple update requests to one solr request.
Handling OOM error
Hi all, We faced an out-of-memory error on 3 nodes of a Solr cloud cluster. There is an oom_solr script in the solr binary to kill the solr on out-of-memory error. How this script works and is there any setting we need to do for this?
Re: deleteById for multiple ids with route parameter
Leave off the routing and send multiple IDs. Solr Cloud will route then to the correct shards for you. This is just as fast as Solr Cloud reading the route parameter and sending it to the right shard. The whole point of Solr Cloud is that it manages shards and replicas for you. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On May 9, 2023, at 2:24 AM, Anjali Maurya > wrote: > > Hi all, > > We are using solr cloud (version 8.10). We are using deleteById which takes > the id and route value to delete the id from the specific shard. deleteById > takes only one value of id. > > I want to ask if there is any way to send multiple ids to deleteById with > route parameter and if can we send multiple update requests to one solr > request.
Re: Handling OOM error
In solr cloud cluster, we got OOM error for 3 node. When I checked the logs, oom_solr script was executed for only 2 nodes. I want to ask why it was executed for 2 nodes and not for 3rd node when we got OOM error for 3 node. How can we be sure about the oom_solr script or it there any setting we need to do for script to run? On Mon, 8 May, 2023, 11:05 am Anjali Maurya, wrote: > Hi all, > We faced an out-of-memory error on 3 nodes of a Solr cloud cluster. There > is an oom_solr script in the solr binary to kill the solr on out-of-memory > error. > How this script works and is there any setting we need to do for this? >
deleteById for multiple ids with route parameter
Hi all, We are using solr cloud (version 8.10). We are using deleteById which takes the id and route value to delete the id from the specific shard. deleteById takes only one value of id. I want to ask if there is any way to send multiple ids to deleteById with route parameter and if can we send multiple update requests to one solr request.
Re: Handling OOM error
On 5/8/23 18:36, Anjali Maurya wrote: In solr cloud cluster, we got OOM error for 3 node. When I checked the logs, oom_solr script was executed for only 2 nodes. I want to ask why it was executed for 2 nodes and not for 3rd node when we got OOM error for 3 node. How can we be sure about the oom_solr script or it there any setting we need to do for script to run? The oom_solr script only runs for Java OutOfMemoryError exceptions, and it does not work on Windows. If Solr dies for some other reason, including the OOM killer built into the operating system, then the oom script included with Solr will not run. Starting with Solr 9.2, the OOME handling is a lot better, and it works on Windows too. It's handled by Java now, not a script. But the same caveat applies for the operating system OOM killer -- there's no way Solr can detect that. Thanks, Shawn
Re: Debug time spent in aggregating the search results
What could be the reason for the missing metric? Where else can I look for
that datapoint? I am trying to figure out the time spent in aggregating the
query results.
On Thu, May 4, 2023 at 9:37 PM Rajani Maski wrote:
>
> Solr version* 9.1.1*
>
>
> Query -
>
> solr/admin/metrics?nodes=all&wt=json&prefix=QUERY./select.distrib.requestTimes
>
> {
> "responseHeader":{
> "status":0,
> "QTime":7},
> "ip:8983_solr":{
> "responseHeader":{
> "status":0,
> "QTime":2},
> "metrics":{}},
> "ip:8983_solr":{
> "responseHeader":{
> "status":0,
> "QTime":2},
> "metrics":{}},
> "ip:8983_solr":{
> "responseHeader":{
> "status":0,
> "QTime":2},
> "metrics":{}}}
>
>
> Query without "distrib"
> solr/admin/metrics?nodes=all&wt=json&prefix=QUERY./select.requestTimes
>
>
> {
> "responseHeader":{
> "status":0,
> "QTime":10},
> "10.146.38.84:8983_solr":{
> "responseHeader":{
> "status":0,
> "QTime":5},
> "metrics":{
> "solr.core.test.shard1.replica_n1":{
> "QUERY./select.requestTimes":{
> "count":403,
> "meanRate":4.906764808325113E-4,
> "1minRate":2.964393875E-314,
> "5minRate":1.4821969375E-313,
> "15minRate":4.44659081257E-313,
> "min_ms":0.0,
> "max_ms":0.0,
> "mean_ms":0.0,
> "median_ms":0.0,
> "stddev_ms":0.0,
> "p75_ms":0.0,
> "p95_ms":0.0,
> "p99_ms":0.0,
> "p999_ms":0.0}},
>
>
> On Thu, May 4, 2023 at 2:51 PM Chris Hostetter
> wrote:
>
>>
>> : Do I need to enable something to retrieve this metric? I tried to
>> query it
>> :
>> http://localhost:8983/solr/admin/metrics?wt=json&prefix=QUERY./select.distrib.requestTimes
>> :
>> : but there is no "distrib" in the response, there are only select
>>
>> Hrm what version of solr are you running?
>>
>>
>> Here's 8.11 ...
>>
>> $ ./bin/solr -e cloud -noprompt
>> ...
>> $ curl -sS '
>> http://localhost:8983/solr/admin/metrics?nodes=all&wt=json&prefix=QUERY./select.distrib.requestTimes
>> '
>> {
>> "responseHeader":{
>> "status":0,
>> "QTime":35},
>> "127.0.1.1:7574_solr":{
>> "responseHeader":{
>> "status":0,
>> "QTime":23},
>> "metrics":{
>> "solr.core.gettingstarted.shard2.replica_n6":{
>> "QUERY./select.distrib.requestTimes":{
>> "count":0,
>> "meanRate":0.0,
>> "1minRate":0.0,
>> "5minRate":0.0,
>> "15minRate":0.0,
>> "min_ms":0.0,
>> "max_ms":0.0,
>> "mean_ms":0.0,
>> "median_ms":0.0,
>> "stddev_ms":0.0,
>> "p75_ms":0.0,
>> "p95_ms":0.0,
>> "p99_ms":0.0,
>> "p999_ms":0.0}},
>> "solr.core.gettingstarted.shard1.replica_n2":{
>> "QUERY./select.distrib.requestTimes":{
>> "count":0,
>> "meanRate":0.0,
>> "1minRate":0.0,
>> "5minRate":0.0,
>> "15minRate":0.0,
>> "min_ms":0.0,
>> "max_ms":0.0,
>> "mean_ms":0.0,
>> "median_ms":0.0,
>> "stddev_ms":0.0,
>> "p75_ms":0.0,
>> "p95_ms":0.0,
>> "p99_ms":0.0,
>> "p999_ms":0.0,
>> "127.0.1.1:8983_solr":{
>> "responseHeader":{
>> "status":0,
>> "QTime":2},
>> "metrics":{
>> "solr.core.gettingstarted.shard2.replica_n4":{
>> "QUERY./select.distrib.requestTimes":{
>> "count":0,
>> "meanRate":0.0,
>> "1minRate":0.0,
>> "5minRate":0.0,
>> "15minRate":0.0,
>> "min_ms":0.0,
>> "max_ms":0.0,
>> "mean_ms":0.0,
>> "median_ms":0.0,
>> "stddev_ms":0.0,
>> "p75_ms":0.0,
>> "p95_ms":0.0,
>> "p99_ms":0.0,
>> "p999_ms":0.0}},
>> "solr.core.gettingstarted.shard1.replica_n1":{
>> "QUERY./select.distrib.requestTimes":{
>> "count":0,
>> "meanRate":0.0,
>> "1minRate":0.0,
>> "5minRate":0.0,
>> "15minRate":0.0,
>> "min_ms":0.0,
>> "max_ms":0.0,
>> "mean_ms":0.0,
>> "median_ms":0.0,
>> "stddev_ms":0.0,
>> "p75_ms":0.0,
>> "p95_ms":0.0,
>> "p99_ms":0.0,
>> "p999_ms":0.0}
>> $ curl -sS '
>> http://localhost:8983/solr/gettingstarted/select?omitHeader=true&q=*:*'
>> {
>>
>>
>> "response":{"numFound":0,"start":0,"maxScore":0.0,"numFoundExact":true,"docs":[]
>> }}
>> $ curl -sS '
>> http://localhost:8983/solr/admin/metrics?nodes=all&wt=json&prefix=QUERY./select.distrib.requestTimes
>> '
>> {
>> "responseHeader":{
>> "status":0,
>> "QTime":25},
>> "127.0.1.1:7574_solr":{
>> "responseHeader":{
>> "status":0,
>> "QTime":8},
>> "metrics":{
>> "solr.core.gettingstarted.shard2.replica_n6":{
>> "QUERY./s
RE: deleteById for multiple ids with route parameter
Yes Anjali, I have used UpdateRequest. deleteById (List) . I don’t face any issues. Not sure what you mean when you say route value. Id is unique. Thanks Subhasis Patra 240-755-2601 subhasis.pa...@e2open.com -Original Message- From: Anjali Maurya Sent: Sunday, May 7, 2023 11:56 PM To: users@solr.apache.org Subject: deleteById for multiple ids with route parameter PHISH ALERT! CHECK VALIDITY IF CLICKING, SHARING, RESPONDING Hi all, We are using solr cloud (version 8.10). We are using deleteById which takes the id and route value to delete the id from the specific shard. deleteById takes only one value of id. I want to ask if there is any way to send multiple ids to deleteById with route parameter and if can we send multiple update requests to one solr request.
Re: streaming expressions - sharding memory usage
Can you share the expressions? Then we can discuss where the sharding comes into play. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, May 9, 2023 at 1:17 PM Sergio García Maroto wrote: > Hi, > > I am working currently on implementing sharding on current Solr Cloud > Cluster. > Main idea is to be able to scale horizontally. > > At the moment, without sharding we have all collections sitting on all > servers. > We have as well pretty heavy streaming expressions returning many ids. > Average of 300,000 ids to join. > > After doing sharding I see a huge increase on CPU and memory usage. > Making queries way slower comparing sharding to not sharding. > > I guess that's expected bacuase the joins need to send data across servers > over network. > > Any thoughs on best practices here. I guess a possible approach is to split > shards in more. > > Regards > Sergio >
