Re: Modifying maxFormContentSize in Apache Solr service.

[Kaushal Shriyan]

On Tue, Oct 18, 2022 at 11:28 PM Kaushal Shriyan 
wrote:

>
>
> On Fri, Oct 14, 2022 at 11:13 PM Kaushal Shriyan 
> wrote:
>
>>
>>
>> On Fri, Oct 14, 2022 at 12:19 AM Kaushal Shriyan <
>> kaushalshri...@gmail.com> wrote:
>>
>>>
>>>
>>> On Wed, Oct 12, 2022 at 9:24 PM Kaushal Shriyan <
>>> kaushalshri...@gmail.com> wrote:
>>>
 Hi,

 I am running Apache Solr 8.11.2 on CentOS Linux release 7.9.2009 (Core)
 trying to modify /opt/solr/server/etc/jetty.xml to set the Attribute
 *maxFormContentSize* as per the below path

 [root@etc]# ls -l
 total 76
 -rw-r--r-- 1 root root  1950 May 13 03:21 jetty-gzip.xml
 -rw-r--r-- 1 root root  3630 May 13 03:21 jetty-https8.xml
 -rw-r--r-- 1 root root  3698 Oct 12 17:54 jetty-https.xml
 -rw-r--r-- 1 root root  2686 Oct 12 17:53 jetty-http.xml
 -rw-r--r-- 1 root root  1850 May 13 03:21 jetty-requestlog.xml
 -rw-r--r-- 1 root root  2248 May 13 03:21 jetty-ssl.xml
 -rw-r--r-- 1 root root 11646 Oct 12 17:51 jetty.xml
 -rw-r--r-- 1 root root 11823 May 13 03:21 security.policy
 -rw-r--r-- 1 root root  1279 May 13 03:21 security.properties
 -rw-r--r-- 1 root root 24426 May 13 03:21 webdefault.xml
 [root@etc]#

 I am adding the below attribute in /opt/solr/server/etc/jetty.xml to
 increase maxFormContentSize

 
   org.eclipse.jetty.server.Request.maxFormContentSize
   150
 

 When i restart the solr service it fails with the below message.

 [root@etc]# service solr restart
 Sending stop command to Solr running on port 8983 ... waiting up to 180
 seconds to allow Jetty process 4311 to stop gracefully.
 Waiting up to 180 seconds to see Solr running on port 8983 [\]  Still
 not seeing Solr listening on 8983 after 180 seconds!
 2022-10-12 10:28:25.469 INFO  (main) [   ] o.e.j.u.log Logging
 initialized @2083ms to org.eclipse.jetty.util.log.Slf4jLog
 2022-10-12 10:28:25.601 WARN  (main) [   ] o.e.j.x.XmlConfiguration
 Config error at 
 >>> class="com.codahale.metrics.jetty9.InstrumentedQueuedThreadPool" name="registry">
 >>> class="com.codahale.metrics.SharedMetricRegistries">solr.jetty
   
   
 2022-10-12 10:28:25.601 WARN  (main) [   ] o.e.j.x.XmlConfiguration  =>
 java.lang.IllegalStateException: Element 'Arg' not skipped
 at
 org.eclipse.jetty.xml.XmlConfiguration$JettyXmlConfiguration.configure(XmlConfiguration.java:507)
 java.lang.IllegalStateException: Element 'Arg' not skipped
 at
 org.eclipse.jetty.xml.XmlConfiguration$JettyXmlConfiguration.configure(XmlConfiguration.java:507)
 ~[jetty-xml-9.4.44.v20210927.jar:9.4.44.v20210927]
 at
 org.eclipse.jetty.xml.XmlConfiguration$JettyXmlConfiguration.configure(XmlConfiguration.java:468)
 ~[jetty-xml-9.4.44.v20210927.jar:9.4.44.v20210927]
 at
 org.eclipse.jetty.xml.XmlConfiguration.configure(XmlConfiguration.java:380)
 ~[jetty-xml-9.4.44.v20210927.jar:9.4.44.v20210927]
 at
 org.eclipse.jetty.xml.XmlConfiguration.lambda$main$3(XmlConfiguration.java:1893)
 ~[jetty-xml-9.4.44.v20210927.jar:9.4.44.v20210927]
 at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
 at
 org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1857)
 ~[jetty-xml-9.4.44.v20210927.jar:9.4.44.v20210927]
 at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 ~[?:?]
 at
 jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 ~[?:?]
 at
 jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[?:?]
 at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
 at org.eclipse.jetty.start.Main.invokeMain(Main.java:218)
 ~[start.jar:9.4.44.v20210927]
 at org.eclipse.jetty.start.Main.start(Main.java:491)
 ~[start.jar:9.4.44.v20210927]
 at org.eclipse.jetty.start.Main.main(Main.java:77)
 ~[start.jar:9.4.44.v20210927]
 [root@etc]#

 Am I missing anything? Please guide me to modify maxFormContentSize in
 solr service. Thanks in advance.

 Best Regards,

 Kaushal

>>>
>>> Hi,
>>>
>>> Checking in again if someone can pitch in for my earlier post to this
>>> mailing list. Thanks in advance.
>>>
>>> Best Regards,
>>>
>>> Kaushal
>>>
>>
>> Hi,
>>
>> I have configured the below settings in /opt/solr/server/etc/jetty.xml
>>
>> 
>> https
>> > default="8443" />
>> > name="solr.jetty.output.buffer.size" default="*62768*" />
>> > name="solr.jetty.output.aggregation.size" default="*81920*" />
>> > name="solr.jetty.request.header.size" default="*81920*" />
>> > name="solr.jetty.response.header.size" default="*81920*" />
>> > name="solr.jetty.send.server.version" default="false" />
>> > name="solr.jetty.send.date.header" default="false" />
>> client_loop: send di

How to return location name in search result?

[Gurmit Teotia]

I've defined following field in my schema to store latitude and longitude-



and "location" is of following field type


I'm storing multiple latitude and longitude pairs in loc_coords field and
using a "geofilt" filter successfully able to retrieve the documents close
to a given point. I'm looking for your advice on how I can store the
location names, corresponding to latitude/longitude pairs and return them
in search result e.g. if "loc_coords" has following lat/long coordinates
for "London" and "New York"

["51.5072,0.1276", "40.7128,74.0060"]

In my search query when I'm looking for documents *near *to "New York" then
the above document will be returned  and in the search result I also want
to return the location name "New York".

I'm considering following two approaches to store location names and return
them in search result-

1. Store location names in a string array field ( call it location_names)
in the same order as lng/lat pair in location_coords field and somehow find
the index of matching lng/lat pair and use the index on location_names
field to retrieve the location name. This is just logical thinking. I'm not
sure how I can achieve it in Solr.

2. Store the data as GeoJson in solr because GeoJson allows to store extra
information along with coordinates as mentioned here- https://geojson.org/.
But I'm not sure how I'm going to get the location name of matching
coording in the search result.

Thank you for the help.

GS

Cannot get nested objects to index

[Matthew Castrigno]

Hello Community, Thank you for taking the time to read my message.

I am attempting to index (update) a document with a nested object. In reading 
the documentation is seems to state that the fields of the nested object do not 
need to be and should not be added to schema.xml.  I am getting error that the 
field of the nested object is undefined, and I am not sure what I am doing 
wrong. Any insights appreciated. Thank you.

[cid:8383c74f-7d38-4694-8eb8-bf71f699027a]
[cid:c6791e41-de3d-460e-b193-500a5ade35ad]

--
"This message is intended for the use of the person or entity to which it is 
addressed and may contain information that is confidential or privileged, the 
disclosure of which is governed by applicable law. If the reader of this 
message is not the intended recipient, you are hereby notified that any 
dissemination, distribution, or copying of this information is strictly 
prohibited. If you have received this message by error, please notify us 
immediately and destroy the related message."

SolrCore is loading

[dmitri maziuk]

Hi all,

I've a weird problem with a fresh standalone install of 8.11.2 on a 
winders VM/Corretto JDK 11: it starts up and is getting into a loop 
where the one existing core is up for a bit, and then it's throwing a 
"503 Core is loading", then it's up again, and so on. The only error I 
see in the log (at debug level) is "can't find configsetprops.json, 
assuming default properties", that doesn't look relevant.


The user running the service has "full control" on the entire solr-8.11 
directory tree and I can't think of anything else to check.


Any ideas what could cause that?

TIA
Dima

Re: SolrCore is loading

[Rahul Goswami]

Hi Dima,
Do you observe anything unusual (blocked thread or something else) in the
thread dump when this happens? Or can you share one?
Maybe through jstack or the /solr/admin/info/threads endpoint.

Rahul

On Mon, Oct 24, 2022 at 5:17 PM dmitri maziuk 
wrote:

> Hi all,
>
> I've a weird problem with a fresh standalone install of 8.11.2 on a
> winders VM/Corretto JDK 11: it starts up and is getting into a loop
> where the one existing core is up for a bit, and then it's throwing a
> "503 Core is loading", then it's up again, and so on. The only error I
> see in the log (at debug level) is "can't find configsetprops.json,
> assuming default properties", that doesn't look relevant.
>
> The user running the service has "full control" on the entire solr-8.11
> directory tree and I can't think of anything else to check.
>
> Any ideas what could cause that?
>
> TIA
> Dima
>

Re: Cannot get nested objects to index

[Matthew Castrigno]

Images I tried to add to original post.

[url=https://postimg.cc/944b6QWZ][img]https://i.postimg.cc/25GtB1Mc/update.png[/img][/url]
[url=https://postimg.cc/WdYSYkPs][img]https://i.postimg.cc/SKk5s6TY/schema.png[/img][/url]

From: Matthew Castrigno 
Sent: Monday, October 24, 2022 2:10 PM
To: users@solr.apache.org 
Subject: Cannot get nested objects to index

Hello Community, Thank you for taking the time to read my message. I am 
attempting to index (update) a document with a nested object. In reading the 
documentation is seems to state that the fields of the nested object do not 
need to be and
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside the St. Luke's email system.

ZjQcmQRYFpfptBannerEnd
Hello Community, Thank you for taking the time to read my message.

I am attempting to index (update) a document with a nested object. In reading 
the documentation is seems to state that the fields of the nested object do not 
need to be and should not be added to schema.xml.  I am getting error that the 
field of the nested object is undefined, and I am not sure what I am doing 
wrong. Any insights appreciated. Thank you.

[cid:8383c74f-7d38-4694-8eb8-bf71f699027a]
[cid:c6791e41-de3d-460e-b193-500a5ade35ad]

"This message is intended for the use of the person or entity to which it is 
addressed and may contain information that is confidential or privileged, the 
disclosure of which is governed by applicable law. If the reader of this 
message is not the intended recipient, you are hereby notified that any 
dissemination, distribution, or copying of this information is strictly 
prohibited. If you have received this message by error, please notify us 
immediately and destroy the related message."

--
"This message is intended for the use of the person or entity to which it is 
addressed and may contain information that is confidential or privileged, the 
disclosure of which is governed by applicable law. If the reader of this 
message is not the intended recipient, you are hereby notified that any 
dissemination, distribution, or copying of this information is strictly 
prohibited. If you have received this message by error, please notify us 
immediately and destroy the related message."

I cannot get nested objects to index - with image links

[Matthew Castrigno]

Hello Community, Thank you for taking the time to read my message.

I am attempting to index (update) a document with a nested object.

In reading the documentation is seems to state that the fields of the nested 
object do not need to be and should not be added

to schema.xml.  I am getting error that the field of the nested object is 
undefined, and I am not sure what I am doing wrong.

Any insights appreciated. Thank you

https://i.postimg.cc/25GtB1Mc/update.png

https://i.postimg.cc/SKk5s6TY/schema.png

--
"This message is intended for the use of the person or entity to which it is 
addressed and may contain information that is confidential or privileged, the 
disclosure of which is governed by applicable law. If the reader of this 
message is not the intended recipient, you are hereby notified that any 
dissemination, distribution, or copying of this information is strictly 
prohibited. If you have received this message by error, please notify us 
immediately and destroy the related message."

Solr 9 Custom ibraries

[Ed K]

Hello everyone,
Wondering if anyone who has migrated to Solr 9.0 is using the custom libraries 
? I have some custom libraries that I've used with 8.11.2 but not seeing that 
functionality on Solr 9.
Any info is greatly appreciated.
Thanks~ed

Re: SolrCore is loading

[dmitri maziuk]

On 2022-10-24 4:21 PM, Rahul Goswami wrote:

Hi Dima,
Do you observe anything unusual (blocked thread or something else) in the
thread dump when this happens? Or can you share one?
Maybe through jstack or the /solr/admin/info/threads endpoint.


Nothing stands out.

So far the only consistent thing is, when I run a data importer script 
(that works w/o problems on solr-8.7) it errors out in the middle of the 
run with Solr either dropping connection in the middle of a POST or 
refusing one for the next POST. :(


There seems to be a
'''
2022-10-24 22:03:49.566 ERROR 
(recoveryExecutor-17-thread-1-processing-x:searchwhs) [   x:searchwhs] 
o.a.s.u.UpdateLog java.io.EOFException => java.io.EOFException
  at 
org.apache.solr.common.util.FastInputStream.readFully(FastInputStream.java:179)

java.io.EOFException: null
'''
on tlog replay but not every time.

Dima

Re: Solr 9 and the Affinity placement plugin

[Jonathan Tan]

For anybody else who comes across this, for easy access, here's the wiki
page where this was first discussed:
https://cwiki.apache.org/confluence/display/SOLR/SIP-8+Autoscaling+policy+engine+V2

And here's the issue where the implementations were discussed:
https://issues.apache.org/jira/browse/SOLR-14613





On Fri, Oct 21, 2022 at 4:03 AM Houston Putman  wrote:

> I agree with Jan. Though I have seen this exact usage multiple times, so it
> will likely be beneficial to the community.
>
> - Houston
>
> On Thu, Oct 13, 2022 at 3:34 AM Jan Høydahl  wrote:
>
> > Hi,
> >
> > In Solr 9 the focus is on a rich Java API that will allow you to create
> > your own placement plugin that can do pretty much anything.
> > The affinity plugin tries to give a good OOTB experience for most use
> > cases, but does not aim to be the one everyone uses.
> > You may want to fork the affinity plugin and extend it with your needs.
> If
> > you believe your extension is useful to a larger audience you may
> consider
> > opening a PR with your changes.
> >
> > Jan
> >
> > > 12. okt. 2022 kl. 16:05 skrev Jonathan Tan :
> > >
> > > Hi!
> > >
> > > In Solr 8.6, I used the autoscaling configuration to set it up so that
> we
> > > had 2 `nodeTypes`, one for indexing, the other for querying.
> > > And our collections were set up to be TLOG + PULL replicas only, and
> we'd
> > > use the autoscaling to place the TLOG on the indexing nodeTypes, and
> the
> > > PULL on the querying nodeTypes. (Essentially replicating the older
> > > master/slave setup of pre-SolrCloud)
> > >
> > > Now we've come to Solr 9, and I can't really find a clear way to do the
> > > same thing.
> > >
> > >
> >
> https://solr.apache.org/guide/solr/latest/configuration-guide/replica-placement-plugins.html#affinityplacementfactory
> > >
> > > I've looked through this, and it seems to imply that the
> > > `collectionNodeType` configuration will allow me to place specific
> > > collections on specific nodeTypes. But I can't find a way to place
> > specific
> > > replica types on specific nodeTypes...
> > >
> > > Is this mechanism no longer existing?
> > >
> > > Thank you
> > > Jonathan
> >
> >
>

SOLVED(-ish) Re: SolrCore is loading

[dmitri maziuk]

OK so this has to do with trying to run solr service as windows domain 
user. It works w/o problems (fingers crossed) when run as "LocalSystem" 
account, runs into this core reloading loop when run as a domain users, 
and giving the domain user "full control" on solr directory tree does 
not fix that.


It's a good thing we're changing providers and will soon be able to 
ditch windows for at least solr. We hope.


Dima

Re: Solr 9 and the Affinity placement plugin

[Maxbone]

Hi there,


Thank you for shopping with us and reaching out!


Unfortunately, it seems we have been included in this communication thread as 
an error. As this is affecting our customer's ticket inquiries in our queue, we 
would like to request to be removed from all communications effective 
immediately.


Please provide confirmation.


All the best,


JOSH

Director of Customer Experience & Retail Stores


[cid:fc6728bd-824a-4629-ab8a-814f23dca428]


Los Angeles | New York City

maxbone.com

@themaxbone



From: Jonathan Tan 
Sent: Monday, October 24, 2022 3:24 PM
To: users@solr.apache.org 
Subject: Re: Solr 9 and the Affinity placement plugin

For anybody else who comes across this, for easy access, here's the wiki
page where this was first discussed:
https://cwiki.apache.org/confluence/display/SOLR/SIP-8+Autoscaling+policy+engine+V2

And here's the issue where the implementations were discussed:
https://issues.apache.org/jira/browse/SOLR-14613





On Fri, Oct 21, 2022 at 4:03 AM Houston Putman  wrote:

> I agree with Jan. Though I have seen this exact usage multiple times, so it
> will likely be beneficial to the community.
>
> - Houston
>
> On Thu, Oct 13, 2022 at 3:34 AM Jan Høydahl  wrote:
>
> > Hi,
> >
> > In Solr 9 the focus is on a rich Java API that will allow you to create
> > your own placement plugin that can do pretty much anything.
> > The affinity plugin tries to give a good OOTB experience for most use
> > cases, but does not aim to be the one everyone uses.
> > You may want to fork the affinity plugin and extend it with your needs.
> If
> > you believe your extension is useful to a larger audience you may
> consider
> > opening a PR with your changes.
> >
> > Jan
> >
> > > 12. okt. 2022 kl. 16:05 skrev Jonathan Tan :
> > >
> > > Hi!
> > >
> > > In Solr 8.6, I used the autoscaling configuration to set it up so that
> we
> > > had 2 `nodeTypes`, one for indexing, the other for querying.
> > > And our collections were set up to be TLOG + PULL replicas only, and
> we'd
> > > use the autoscaling to place the TLOG on the indexing nodeTypes, and
> the
> > > PULL on the querying nodeTypes. (Essentially replicating the older
> > > master/slave setup of pre-SolrCloud)
> > >
> > > Now we've come to Solr 9, and I can't really find a clear way to do the
> > > same thing.
> > >
> > >
> >
> https://solr.apache.org/guide/solr/latest/configuration-guide/replica-placement-plugins.html#affinityplacementfactory
> > >
> > > I've looked through this, and it seems to imply that the
> > > `collectionNodeType` configuration will allow me to place specific
> > > collections on specific nodeTypes. But I can't find a way to place
> > specific
> > > replica types on specific nodeTypes...
> > >
> > > Is this mechanism no longer existing?
> > >
> > > Thank you
> > > Jonathan
> >
> >
>

Re: SOLVED(-ish) Re: SolrCore is loading

[Rahul Goswami]

That’s kind of strange that the EOFException doesn’t occur with the change
to LocalSystem. But glad it works anyway.
For what it’s worth, we run Solr on Windows as a service under the Network
Service account which is less privileged than LocalSystem.

Rahul

On Mon, Oct 24, 2022 at 6:32 PM dmitri maziuk 
wrote:

> OK so this has to do with trying to run solr service as windows domain
> user. It works w/o problems (fingers crossed) when run as "LocalSystem"
> account, runs into this core reloading loop when run as a domain users,
> and giving the domain user "full control" on solr directory tree does
> not fix that.
>
> It's a good thing we're changing providers and will soon be able to
> ditch windows for at least solr. We hope.
>
> Dima
>
>