Re: solr backups failing

[Vivaldi]

Are you using HDFS to backup or shared network directory? In the case of shared 
directory the same location should be writable (accessible and has write 
permissions) from all solr nodes. 


> On 28 Mar 2023, at 19:22, Houston Putman  wrote:
> 
> Can you provide more information, such as the request you send to solr and
> logs that might explain the IOException?
> 
> - Houston
> 
>> On Wed, Mar 22, 2023 at 10:51 AM r ohara  wrote:
>> 
>> Hi all,
>> We are using solrcloud 8.11 with 10 shards. We've been getting errors
>> trying to back up our index using the collections API. Here's the error we
>> see in some of the nodes when we run the REQUESTSTATUS api.
>> 
>> Failed to backup core=collection_02_replica_t21 because
>> java.io.IOException: Input/output error
>> 
>> We ran it successfully once, but since then at least one node, but not
>> always the same one fails. It is a large index with almost 100 million
>> documents and 2T total in size. What could cause this error?
>> 
>> Thanks in advance
>> 

Authentication problems

[MEXANIK]

I logged in using the john_sl user in Solr UI, and sent a request to read
the collection, but I can read some of the collections, but I don't want
them to be read.

If I replace the admin user's rights from admin_rwx to admin_x, then the
john_sl user can't read either

How do I make it so that john_sl cannot read collections with admin_x
rights, but at the same time so that the admin user has admin_rwx rights??

Help!!

Example security.json http://replit.com/@irkuev666/Test#data.json

Re: Authentication problems

[MEXANIK]

Apache Solr version 8.11.0

ср, 29 мар. 2023 г., 15:27 MEXANIK :

> I logged in using the john_sl user in Solr UI, and sent a request to read
> the collection, but I can read some of the collections, but I don't want
> them to be read.
>
> If I replace the admin user's rights from admin_rwx to admin_x, then the
> john_sl user can't read either
>
> How do I make it so that john_sl cannot read collections with admin_x
> rights, but at the same time so that the admin user has admin_rwx rights??
>
> Help!!
>
> Example security.json http://replit.com/@irkuev666/Test#data.json
>

RE: Jetty 11 Upgrade

[Rao, Vanishree]

Hey team,

Could you please reply?

Thanks & Regards,
Vanishree Rao
(Vaa-nee Shree Ra-ao)
Developer, Application Development
vanishree@transunion.com
M: +91 8097034710
Pronouns: She/Her
[TULogo-blue-rgb-120px-01]

This email including, without limitation, the attachments, if any, accompanying 
this email, may contain information which is confidential or privileged and 
exempt from disclosure under applicable law.
The information is for the use of the intended recipient. If you are not the 
intended recipient, be aware that any disclosure, copying, distribution, review 
or use of the contents of this email, and/or its attachments, is without 
authorization and is prohibited. If you have received this email in error, 
please notify us by reply email immediately and destroy all copies of this 
email and its attachments.

From: Rao, Vanishree
Sent: Monday, March 27, 2023 10:40 AM
To: solr-u...@lucene.apache.org
Subject: Jetty 11 Upgrade

Hello Team,

Could you please confirm when is it planned to complete this Jetty 11.x 
Upgrade: https://issues.apache.org/jira/browse/SOLR-16441 ?

Thanks & Regards,
Vanishree Rao
(Vaa-nee Shree Ra-ao)
Developer, Application Development
vanishree@transunion.com
M: +91 8097034710
Pronouns: She/Her
[TULogo-blue-rgb-120px-01]

This email including, without limitation, the attachments, if any, accompanying 
this email, may contain information which is confidential or privileged and 
exempt from disclosure under applicable law.
The information is for the use of the intended recipient. If you are not the 
intended recipient, be aware that any disclosure, copying, distribution, review 
or use of the contents of this email, and/or its attachments, is without 
authorization and is prohibited. If you have received this email in error, 
please notify us by reply email immediately and destroy all copies of this 
email and its attachments.

Recall: Jetty 11 Upgrade

[Rao, Vanishree]

Rao, Vanishree would like to recall the message, "Jetty 11 Upgrade".

Re: Authentication problems

[Jan Høydahl]

Permissions are evaluated in order from top to bottom. 
The first "read" permission found requires roles "admin_ro", "admin_rwx", 
"solr-internal-traffic", so that should be selected.

Do you have any logs that can shed light over what happens?

Jan

> 29. mar. 2023 kl. 14:27 skrev MEXANIK :
> 
> I logged in using the john_sl user in Solr UI, and sent a request to read
> the collection, but I can read some of the collections, but I don't want
> them to be read.
> 
> If I replace the admin user's rights from admin_rwx to admin_x, then the
> john_sl user can't read either
> 
> How do I make it so that john_sl cannot read collections with admin_x
> rights, but at the same time so that the admin user has admin_rwx rights??
> 
> Help!!
> 
> Example security.json http://replit.com/@irkuev666/Test#data.json

Re: LDAP authentication with Solr 9.1.1 or later

[Kevin Risden]

I thought there was a way to configure the user/group filter for the users
using Hadoop authentication. It was something
like hadoop.security.group.mapping.ldap - but I haven't looked at it in
years. I can't seem to find a reference right now.

Kevin Risden


On Sat, Mar 25, 2023 at 2:02 AM Shawn Heisey  wrote:

> On 3/24/23 21:03, Shawn Heisey wrote:
> > On 3/24/23 20:39, Kevin Risden wrote:
> >> I personally haven't tried it but I would guess it takes the
> >> principal/username and you can map it to whatever roles you want using
> >> "user-role":
> >>
> >>
> https://solr.apache.org/guide/solr/latest/deployment-guide/rule-based-authorization-plugin.html#configuration-syntax
> One more question about the authentication part:  The other ldap
> configurations in use at this site specify an ldap group that the users
> must be a member of in order to get authenticated at all.
>
> I searched HadoopAuthPlugin.java for group and found nothing.  Is there
> a way to specify that?
>
> Thanks,
> Shawn
>

Setting up a new Kubernetes Cluster in Production with Azure Kubernetes Service

[Dipu Divakaran]

Team,

I am trying to find the VM Node Configuration required for running an Azure 
Kubernetes Production Cluster. Trying to deploy a .Net Service, Apache Solr, 
Apache ZooKeeper and Rabbit MQ containers inside.

I may have to use around 4500 Indexes.
Maximum data storage need will be 300 GB.
Maximum document count cab be around 6 million
Average Index size can be 240 MB
Max Index size can be up to 6GB
average document size 3.5 kb

This is the kind of metrics I am trying to use. Data/Index growth expected in 
future will be around 10% per year.

If someone have prior experience in setting up something similar in PROD using 
Azure or AWS using Kubernetes, Please share me your experience.

Thans
Dipu Divakaran

This e-mail is intended solely for the person or entity to which it is 
addressed and may contain confidential and/or privileged information. Any 
review, dissemination, copying, printing or other use of this e-mail by persons 
or entities other than the addressee is prohibited. If you have received this 
e-mail in error, please contact the sender immediately and delete this e-mail 
and any attachments from any device.

Is there a tutorial for Solr 9.2 and Kubernetes

[Phill Campbell]

I have been looking for an example of how to use Solr 9 with Kubernetes.
All of the examples I have found are using Solr 8 and Solr 8’s autoscaling.

Since autoscaling is deprecated in Solr 9 I was hoping to find guidance for 
Solr 9.

Re: Is there a tutorial for Solr 9.2 and Kubernetes

[Deepak Vohra]

 If the only issue is autoscaling being removed,   use the new API for 
pluggable Replica Placement Plugins that replaces the auto-scaling framework.
https://solr.apache.org/guide/solr/latest/configuration-guide/replica-placement-plugins.html
On Wednesday, March 29, 2023 at 04:29:25 p.m. EDT, Phill Campbell 
 wrote:  
 
 I have been looking for an example of how to use Solr 9 with Kubernetes.
All of the examples I have found are using Solr 8 and Solr 8’s autoscaling.

Since autoscaling is deprecated in Solr 9 I was hoping to find guidance for 
Solr 9.


  

Re: Is there a tutorial for Solr 9.2 and Kubernetes

[Deepak Vohra]

 Apache Solr Operator  manages Solr resources in KubernetesDoesn't it support 
version 9 yet?https://solr.apache.org/operator/
8.6+ is listed as supported, which should include 9.2. 
https://apache.github.io/solr-operator/docs/upgrade-notes.html

On Wednesday, March 29, 2023 at 04:29:25 p.m. EDT, Phill Campbell 
 wrote:  
 
 I have been looking for an example of how to use Solr 9 with Kubernetes.
All of the examples I have found are using Solr 8 and Solr 8’s autoscaling.

Since autoscaling is deprecated in Solr 9 I was hoping to find guidance for 
Solr 9.


  

Can't start solr with 9.2.0

[Ricardo Ruiz]

Hi!

I'm building and provisioning Solr images using
geerlingguy/ansible-role-solr
 for versions 9.1.1,
9.0.0, 8.11.2, and the newest version 9.2.0.

My configuration works for the other three versions, but when I try to
start the service for 9.2.0, the start process fails and keeps restarting
over and over.

>From the logs, this is what I can see (please see the attached file).

I'm not sure what could have changed in this new version, or if this is a
problem with the Ansible role, but any insight would be appreciated.

Thanks,
Ricardo Ruiz
```log
ERROR StatusConsoleListener Unable to locate appender "MainLogFile" for logger 
config "root"
ERROR StatusConsoleListener Unable to locate appender "SlowLogFile" for logger 
config "org.apache.solr.core.SolrCore.SlowRequest"
2023-03-29 21:44:50.444 INFO  (main) [] o.e.j.s.Server jetty-10.0.13; built: 
2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 
11.0.18+10-LTS
2023-03-29 21:44:50.950 WARN  (main) [] o.e.j.u.DeprecationWarning Using 
@Deprecated Class org.eclipse.jetty.servlet.listener.ELContextCleaner
2023-03-29 21:44:50.995 INFO  (main) [] o.a.s.s.CoreContainerProvider Using 
logger factory org.apache.logging.slf4j.Log4jLoggerFactory
2023-03-29 21:44:51.004 INFO  (main) [] o.a.s.s.CoreContainerProvider  ___  
_   Welcome to Apache Solr™ version 9.2.0
2023-03-29 21:44:51.004 INFO  (main) [] o.a.s.s.CoreContainerProvider / __| 
___| |_ _   Starting in standalone mode on port 8983
2023-03-29 21:44:51.004 INFO  (main) [] o.a.s.s.CoreContainerProvider \__ \/ _ 
\ | '_|  Install dir: /opt/solr-9.2.0
2023-03-29 21:44:51.004 INFO  (main) [] o.a.s.s.CoreContainerProvider 
|___/\___/_|_|Start time: 2023-03-29T21:44:51.004480Z
2023-03-29 21:44:51.009 INFO  (main) [] o.a.s.s.CoreContainerProvider Solr 
started with "-XX:+CrashOnOutOfMemoryError" that will crash on any 
OutOfMemoryError exception. The cause of the OOME will be logged in the crash 
file at the following path: /var/solr/logs/jvm_crash_18081.log
java.lang.reflect.InvocationTargetException
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.eclipse.jetty.start.Main.invokeMain(Main.java:229)
at org.eclipse.jetty.start.Main.start(Main.java:527)
at org.eclipse.jetty.start.Main.main(Main.java:76)
Caused by: java.security.AccessControlException: access denied 
("java.io.FilePermission" "/mnt/instance-data/solr/logs" "read")
at 
java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at 
java.base/java.security.AccessController.checkPermission(AccessController.java:897)
at 
java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
at 
java.base/java.lang.SecurityManager.checkRead(SecurityManager.java:661)
at java.base/java.io.File.isDirectory(File.java:856)
at 
org.eclipse.jetty.util.RolloverFileOutputStream.setFile(RolloverFileOutputStream.java:237)
at 
org.eclipse.jetty.util.RolloverFileOutputStream.(RolloverFileOutputStream.java:180)
at 
org.eclipse.jetty.util.RolloverFileOutputStream.(RolloverFileOutputStream.java:143)
at 
org.eclipse.jetty.server.RequestLogWriter.doStart(RequestLogWriter.java:192)
at 
org.eclipse.jetty.server.AsyncRequestLogWriter.doStart(AsyncRequestLogWriter.java:92)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:121)
at 
org.eclipse.jetty.server.CustomRequestLog.doStart(CustomRequestLog.java:483)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171)
at org.eclipse.jetty.server.Server.start(Server.java:470)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:121)
at 
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89)
at org.eclipse.jetty.server.Server.doStart(Server.java:415)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93)
at 
org.eclipse.jetty.xml.XmlConfiguration.lambda$main$4(XmlConfiguration.java:1880)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at 
org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1818)
   

Rid generation in solr cloud logs

[Anjali Maurya]

Hi all,
We have seen that rid is not coming in logs if we are querying on a single
shard collection. But if we are querying on a multiple shards collection
then rid is coming in the logs.

Is there any specific reason for this?

Re: Can't start solr with 9.2.0

[Shawn Heisey]

On 3/29/2023 7:10 PM, Ricardo Ruiz wrote:
My configuration works for the other three versions, but when I try to 
start the service for 9.2.0, the start process fails and keeps 
restarting over and over.


 From the logs, this is what I can see (please see the attached file).

I'm not sure what could have changed in this new version, or if this is 
a problem with the Ansible role, but any insight would be appreciated.


This is the relevant line from the log:

Caused by: java.security.AccessControlException: access denied 
("java.io.FilePermission" "/mnt/instance-data/solr/logs" "read")


Is any part of /mnt/instance-data/solr/logs a symlink, or is Solr 
started with a directory setting that has a symlink to that location? 
Does the user that is running Solr have read/write permission to that 
location?


The reason that I ask about symlinks is that Solr 9 starts with a 
security manager that restricts what directories it can access.  We've 
already seen and fixed problems with symlinks for the install directory, 
similar problems could exist for the data directories too.  Avoiding 
symlinks for Solr's data directories would be a good idea.  We'd like to 
know about any problems there so we can fix them in a future version.


Thanks,
Shawn

Re: Rid generation in solr cloud logs

[Shawn Heisey]

On 3/29/2023 9:09 PM, Anjali Maurya wrote:

We have seen that rid is not coming in logs if we are querying on a single
shard collection. But if we are querying on a multiple shards collection
then rid is coming in the logs.

Is there any specific reason for this?


SolrCloud adds the rid parameter when it accesses another core via HTTP. 
 If there is one server and the collection has only one shard with no 
redundancy, then Solr will never need to talk to another core to satisfy 
the request, which means that the rid parameter will not be added.


I do not know how the value of the parameter is chosen or whether the 
value has any significance to anything outside of SolrCloud internals.


Thanks,
Shawn