Re: Solr: Get leaderness status of local node
On 5/8/2021 8:05 PM, lamine lamine wrote: I only want the code be run once per shard. One way to guarantee that is to do it in the leader, as there is always one leader per shard. I don't want to run it in all the replicas. The code to run is "external" it doesn't touch any document. What I am saying is that when you define the processor chain, include DistributedUpdateProcessor in it. It gets added by SolrCloud even if you don't include it, so it's better to have control over its placement. And then place your update processor in the list *BEFORE* DistributedUpdateProcessor. This should accomplish your goals automatically, with no code required. Thanks, Shawn
Re: Solr: Get leaderness status of local node
If the true requirement is merely to process the document once, then I agree with Shawn's solution. You needn't concern yourself with knowing who the leader is. If somehow it's important that it be guaranteed to execute on the leader in particular, then take inspiration from some existing URPs. I'm thinking this: https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/update/processor/SkipExistingDocumentsProcessorFactory.java#L217 (method isLeader). Also note that this particular URP implements RunAlways, thus its order can be before or after DURP as you please. ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Sun, May 9, 2021 at 12:23 PM Shawn Heisey wrote: > On 5/8/2021 8:05 PM, lamine lamine wrote: > > I only want the code be run once per shard. One way to guarantee that > is to do it in the leader, as there is always one leader per shard. I don't > want to run it in all the replicas. The code to run is "external" it > doesn't touch any document. > > What I am saying is that when you define the processor chain, include > DistributedUpdateProcessor in it. It gets added by SolrCloud even if > you don't include it, so it's better to have control over its placement. > And then place your update processor in the list *BEFORE* > DistributedUpdateProcessor. > > This should accomplish your goals automatically, with no code required. > > Thanks, > Shawn >
Re: childFilter Solr 6.6
Hi,
childFilter is guaranteed to return the documents in the order it was
provided to Solr.
If you want to sort or do other interesting things, consider [subQuery]
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley
On Thu, Apr 29, 2021 at 3:55 PM Roopa Rao wrote:
> Hi -
> I am on version Solr 6.6 and using parent and child construct
>
> Query as follows:
> [child parentFilter='field1:value1' childFilter='((field1:(value1)) AND
> ({!terms f=field2}value2))' limit=1]
>
> Here I want to sort this children by field3 and return only the top 1
> element (hence limit=1) . Is there a way to do this?
>
> I notice that when the children are returned, it is being returned in the
> order it was indexed. Is this guaranteed?
> If it is going to return the order in which is indexed that will work in my
> case too, as I index the field3 data in the order which it should be shown.
>
> Thanks,
> Roopa
>
Re: -Ddisable.configEdit=true does not work
I was not able to reproduce the problem. I did this with the
"techproducts" example (thus standalone mode), and ran that curl command,
and got this error:
"responseHeader":{
"status":403,
"QTime":10044},
"error":{
"metadata":[
"error-class","org.apache.solr.common.SolrException",
"root-error-class","org.apache.solr.common.SolrException"],
"msg":" solrconfig editing is not enabled due to disable.configEdit",
"code":403}}
I was running in a debugger with breakpoints, which is why it took 10
seconds :-) I was also using main branch (eventual 9.0).
I ran Solr 8.8 as well, this time in SolrCloud mode, created a "demo"
collection, and I got the same error message as above.
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley
On Thu, May 6, 2021 at 2:52 PM Bogdan Korchevskiy <
bogdan.korchevs...@gmail.com> wrote:
> -Ddisable.configEdit=true does not work
>
Solr JVM Heap becomes full and stops when we try to restart
Hi everyone,
We have 3 cluster solr running in 3 different machines with an index size
of 300 GB.
RAM: 300 GB per node
Heap - Xms: 240GB Xmx: 300GB
Index size: 300GB
GC_TUNE="-XX:+UseG1GC
-XX:InitiatingHeapOccupancyPercent=45
-XX:ConcGCThreads=6
-XX:ParallelGCThreads=30
-XX:G1ReservePercent=20
${solr.autoCommit.maxTime:40}
false
${solr.autoSoftCommit.maxTime:-1}
[image: image.png]
*Our cloud servers suddenly stopped yesterday. When we try to restart, our
JVM heap size goes to max of 300 GB just in few seconds and we get the
following message before stopping automatically.*
Heap before GC invocations=0 (full 0):
garbage-first heap total 251658240K, used 360448K [0x7eba8000,
0x7eba8200f000, 0x7f058000)
region size 32768K, 12 young (393216K), 0 survivors (0K)
Metaspace used 20504K, capacity 21158K, committed 21248K, reserved
22528K
2021-05-10T05:31:59.511+: 3.036: [GC pause (Metadata GC Threshold)
(young) (initial-mark)
Desired survivor size 805306368 bytes, new threshold 15 (max 15)
{Heap before GC invocations=11 (full 0):
garbage-first heap total 288849920K, used 20398080K [0x7eba8000,
0x7eba82011378, 0x7f058000)
region size 32768K, 440 young (14417920K), 54 survivors (1769472K)
Metaspace used 58413K, capacity 61495K, committed 61696K, reserved
63488K
2021-05-10T05:33:15.477+: 79.002: [GC pause (G1 Evacuation Pause)
(young)
Desired survivor size 922746880 bytes, new threshold 1 (max 15)
- age 1: 1043976736 bytes, 1043976736 total
- age 2: 766998080 bytes, 1810974816 total
, 0.4319767 secs]
[Parallel Time: 408.3 ms, GC Workers: 30]
[GC Worker Start (ms): Min: 79002.5, Avg: 79003.0, Max: 79003.6, Diff:
1.2]
[Ext Root Scanning (ms): Min: 0.1, Avg: 0.8, Max: 2.7, Diff: 2.6, Sum:
23.7]
[Update RS (ms): Min: 0.0, Avg: 1.7, Max: 3.1, Diff: 3.1, Sum: 51.7]
[Processed Buffers: Min: 0, Avg: 3.8, Max: 17, Diff: 17, Sum: 113]
[Scan RS (ms): Min: 13.9, Avg: 15.8, Max: 16.7, Diff: 2.8, Sum: 474.0]
[Code Root Scanning (ms): Min: 0.0, Avg: 0.1, Max: 2.1, Diff: 2.1,
Sum: 4.3]
[Object Copy (ms): Min: 385.5, Avg: 387.5, Max: 390.6, Diff: 5.1, Sum:
11624.2]
[Termination (ms): Min: 0.1, Avg: 0.5, Max: 0.9, Diff: 0.9, Sum: 13.8]
[Termination Attempts: Min: 1, Avg: 82.1, Max: 172, Diff: 171, Sum:
2464]
[GC Worker Other (ms): Min: 0.0, Avg: 0.1, Max: 0.4, Diff: 0.4, Sum:
3.6]
[GC Worker Total (ms): Min: 405.9, Avg: 406.5, Max: 407.3, Diff: 1.4,
Sum: 12195.3]
[GC Worker End (ms): Min: 79409.4, Avg: 79409.5, Max: 79409.8, Diff:
0.4]
[Code Root Fixup: 0.1 ms]
[Code Root Purge: 0.0 ms]
[Clear CT: 6.7 ms]
[Other: 16.9 ms]
[Choose CSet: 0.0 ms]
[Ref Proc: 5.2 ms]
[Ref Enq: 0.0 ms]
[Redirty Cards: 9.2 ms]
[Humongous Register: 0.3 ms]
[Humongous Reclaim: 0.0 ms]
[Free CSet: 0.7 ms]
Please help to solve this issue!
Thanks in advance!
Regards!
Vigz
