Re: Rule-Based Authorization Plugin not working as described in the documentation

[Sebastien VERDET]

Hello,

thanks for the reply, even if it makes me sad. I could not find any 
reference to this limitation anywhere in the documentation, and the 
parameter/attribute does not generate any error in this case (and the 
rule is just ignored without notice). Or maybe it's just in the only 
piece of documentation I haven't read yet regarding authentication and 
authorization in solr ?


--
Regards,
_
*Sebastien VERDET*
Linux Systems and Applications Integrator – PID

*LINKBYNET*
*Web : https://www.linkbynet.com **
*_

*Before printing this e-mail, think about environment.*


On 2021/07/13 16:00:31, Thomas Corthals  wrote:
> Hello Sebastien,>
>
> Unfortunately rules can only have a collection scope in SolrCloud, 
not a>

> core scope in standalone Solr.>
>
> Thomas>
>
> Op di 13 jul. 2021 om 16:17 schreef Sebastien VERDET 


> >:>
>
> > Hello everybody,>
> >>
> > I'm trying to deploy the "RBAP" plugin on a really basic solr 8.8.2>
> > installation that was done using the Ansible galaxy module by Jeff>
> > Geerling : https://galaxy.ansible.com/geerlingguy/solr>
> >>
> > This is pretty straightforward : standalone, started through systemd.>
> > I've kept the default 'collection1' core and created a 'collection2'>
> > core to perform my testing of permissions/roles, following the>
> > documentation. Basically, I'm trying to create rules with only one>
> > collection scope, and two users having permissions only on one of the>
> > cores/collection. And for now, i'm focusing on read permissions as 
other>

> > will follow naturally.>
> >>
> > But this is not going as expected. In the permission ordering section>
> > (>
> > 
https://solr.apache.org/guide/8_8/rule-based-authorization-plugin.html#permission-ordering-and-resolution),> 


> >>
> > it states that rules mentioning explicit collection names are taken>
> > first. Here is my security.json resulting of API calls to create 
users,>

> > permissions and roles :>
> >>
> > {>
> > "authentication": {>
> > "class": "solr.BasicAuthPlugin",>
> > "blockUnknown": true,>
> > "credentials": {>
> > "solr": "X== YY=",>
> > "seb": "= BBB=",>
> > "Osman": "== =">
> > },>
> > "": {>
> > "v": 0>
> > }>
> > },>
> > "authorization": {>
> > "class": "solr.RuleBasedAuthorizationPlugin",>
> > "permissions": [>
> > {>
> > "name": "security-edit",>
> > "role": "admin",>
> > "index": 1>
> > },>
> > {>
> > "name": "read",>
> > "role": [>
> > "admin",>
> > "Osman">
> > ],>
> > "index": 2>
> > },>
> > {>
> > "name": "update",>
> > "role": [>
> > "admin">
> > ],>
> > "index": 3>
> > },>
> > {>
> > "name": "coll_read",>
> > "role": [>
> > "read_sve">
> > ],>
> > "collection": [>
> > "collection2">
> > ],>
> > "path": "/select",>
> > "index": 4>
> > }>
> > ],>
> > "user-role": {>
> > "solr": "admin",>
> > "seb": [>
> > "read_sve">
> > ],>
> > "Osman": [>
> > "Osman">
> > ]>
> > },>
> > "": {>
> > "v": 0>
> > }>
> > }>
> > }>
> >>
> > I'm querying collection2 with user seb :>
> >>
> > curl -u seb:xxx -H "Content-Type: application/json">
> > "http://127.0.0.1:12001/solr/collection2/select?q=*delete*";>
> >>
> > I get a 403 error, with corresponding message in solr.log :>
> >>
> > 2021-07-13 13:23:53.934 INFO (qtp208684473-17) [ x:collection2]>
> > o.a.s.s.RuleBasedAuthorizationPluginBase This resource is 
configured to>

> > have a permission {>
> > "name":"read",>
> > "role":[>
> > "admin",>
> > "Osman"],>
> > "index":2}, The principal>
> > 
org.apache.solr.security.BasicAuthPlugin$BasicAuthUserPrincipal@54a23304[username=seb,pwd=*]> 


> >>
> > does not have the right role>
> >>
> > It takes the "global rule" instead of the collection specific one. 
I'ved>
> > try to change orders (as there are indexes), always with api calls 
(by>
> > deleting/recreating), even restarting the solr service between 
changes,>

> > to no luck. "Global" permissions assignation works correctly, but as>
> > soon as we want to focus on one collection, we are in the dark.>
> >>
> > Many searches with different terms leads to almost no resources 
around>
> > this plugin, so it's complicated to tackle. Did anyone already hav 
this>

> > problem, and can share how it can be worked on ?>
> >>
> >>
> > -->
> >>
> > Regards,>
> > _>
> > *Sebastien VERDET*>
> > Linux Systems and Applications Integrator – PID>
> > >
> > *LINKBYNET*>
> > *Web : https://www.linkbynet.com **>
> > *_>
> >>
> > *Before printing this e-mail, think about environment.*>
> >>
> >>
>

Lucene grouping on non-doc value field.

[Modassar Ather]

Hi,

I need to implement grouping search using Lucene APIs on fields which are
non-doc value fields.

The index is created using Solr-6.5.1 and spread across 12 shards.
The group field's definition is as follows.


I tried to use Lucene group related APIs but it is failing as the
fields are not indexed as docValues.

Kindly provide your inputs/pointers on how I can achieve it outside the
context of Solr using Lucene APIs.

Thanks,
Modassar

Lucene grouping on non-doc value field.

[Modassar Ather]

Hi,

I need to implement grouping search using Lucene APIs on fields which are
non-doc value fields.

The index is created using Solr-6.5.1 and spread across 12 shards.
The group field's definition is as follows.


I tried to use Lucene group related APIs but it is failing as the
fields are not indexed as docValues.

Kindly provide your inputs/pointers on how I can achieve it outside the
context of Solr using Lucene APIs.

Thanks,
Modassar

Re: Rule-Based Authorization Plugin not working as described in the documentation

[Thomas Corthals]

The docs are technically correct in consistently talking about "collection"
and not "core" in this regard. They could make it a bit clearer that it
doesn't apply equally to cores though. Just like you, I assumed they would
be interchangeable and found out the hard way.

I asked about core specific rules recently:
http://mail-archives.apache.org/mod_mbox/solr-users/202105.mbox/%3ccabewpvezzmf5kkaekoik7o-uvxiqzi43e-j7thsf0p213gy...@mail.gmail.com%3e

Thomas

Op do 15 jul. 2021 om 09:01 schreef Sebastien VERDET :

> Hello,
>
> thanks for the reply, even if it makes me sad. I could not find any
> reference to this limitation anywhere in the documentation, and the
> parameter/attribute does not generate any error in this case (and the
> rule is just ignored without notice). Or maybe it's just in the only
> piece of documentation I haven't read yet regarding authentication and
> authorization in solr ?
>
> --
> Regards,
> _
> *Sebastien VERDET*
> Linux Systems and Applications Integrator – PID
> 
> *LINKBYNET*
> *Web : https://www.linkbynet.com **
> *_
>
> *Before printing this e-mail, think about environment.*
>
>
> On 2021/07/13 16:00:31, Thomas Corthals  wrote:
>  > Hello Sebastien,>
>  >
>  > Unfortunately rules can only have a collection scope in SolrCloud,
> not a>
>  > core scope in standalone Solr.>
>  >
>  > Thomas>
>  >
>  > Op di 13 jul. 2021 om 16:17 schreef Sebastien VERDET
> 
>  > >:>
>  >
>  > > Hello everybody,>
>  > >>
>  > > I'm trying to deploy the "RBAP" plugin on a really basic solr 8.8.2>
>  > > installation that was done using the Ansible galaxy module by Jeff>
>  > > Geerling : https://galaxy.ansible.com/geerlingguy/solr>
>  > >>
>  > > This is pretty straightforward : standalone, started through systemd.>
>  > > I've kept the default 'collection1' core and created a 'collection2'>
>  > > core to perform my testing of permissions/roles, following the>
>  > > documentation. Basically, I'm trying to create rules with only one>
>  > > collection scope, and two users having permissions only on one of the>
>  > > cores/collection. And for now, i'm focusing on read permissions as
> other>
>  > > will follow naturally.>
>  > >>
>  > > But this is not going as expected. In the permission ordering section>
>  > > (>
>  > >
>
> https://solr.apache.org/guide/8_8/rule-based-authorization-plugin.html#permission-ordering-and-resolution),>
>
>
>  > >>
>  > > it states that rules mentioning explicit collection names are taken>
>  > > first. Here is my security.json resulting of API calls to create
> users,>
>  > > permissions and roles :>
>  > >>
>  > > {>
>  > > "authentication": {>
>  > > "class": "solr.BasicAuthPlugin",>
>  > > "blockUnknown": true,>
>  > > "credentials": {>
>  > > "solr": "X== YY=",>
>  > > "seb": "= BBB=",>
>  > > "Osman": "== =">
>  > > },>
>  > > "": {>
>  > > "v": 0>
>  > > }>
>  > > },>
>  > > "authorization": {>
>  > > "class": "solr.RuleBasedAuthorizationPlugin",>
>  > > "permissions": [>
>  > > {>
>  > > "name": "security-edit",>
>  > > "role": "admin",>
>  > > "index": 1>
>  > > },>
>  > > {>
>  > > "name": "read",>
>  > > "role": [>
>  > > "admin",>
>  > > "Osman">
>  > > ],>
>  > > "index": 2>
>  > > },>
>  > > {>
>  > > "name": "update",>
>  > > "role": [>
>  > > "admin">
>  > > ],>
>  > > "index": 3>
>  > > },>
>  > > {>
>  > > "name": "coll_read",>
>  > > "role": [>
>  > > "read_sve">
>  > > ],>
>  > > "collection": [>
>  > > "collection2">
>  > > ],>
>  > > "path": "/select",>
>  > > "index": 4>
>  > > }>
>  > > ],>
>  > > "user-role": {>
>  > > "solr": "admin",>
>  > > "seb": [>
>  > > "read_sve">
>  > > ],>
>  > > "Osman": [>
>  > > "Osman">
>  > > ]>
>  > > },>
>  > > "": {>
>  > > "v": 0>
>  > > }>
>  > > }>
>  > > }>
>  > >>
>  > > I'm querying collection2 with user seb :>
>  > >>
>  > > curl -u seb:xxx -H "Content-Type: application/json">
>  > > "http://127.0.0.1:12001/solr/collection2/select?q=*delete*";>
>  > >>
>  > > I get a 403 error, with corresponding message in solr.log :>
>  > >>
>  > > 2021-07-13 13:23:53.934 INFO (qtp208684473-17) [ x:collection2]>
>  > > o.a.s.s.RuleBasedAuthorizationPluginBase This resource is
> configured to>
>  > > have a permission {>
>  > > "name":"read",>
>  > > "role":[>
>  > > "admin",>
>  > > "Osman"],>
>  > > "index":2}, The principal>
>  > >
> org.apache.solr.security.BasicAuthPlugin$BasicAuthUserPrincipal@54a23304[username=seb,pwd=*]>
>
>
>  > >>
>  > > does not have the right role>
>  > >>
>  > > It takes the "global rule" instead of the collection specific one.
> I'ved>
>  > > try to change orders (as there are indexes), always with api calls
> (by>
>  > > deleting/recreating), even restarting the solr service between
> changes,>
>  > > to no luck. "Global" permissions assignation w

Incorrect status in CLUSTERSTATUS

[Bram Van Dam]

Howdy,

Solr 7.7, in Cloud, with ~400 cores spread over 6 chunky servers.

We've noticed that CLUSTERSTATUS occasionally contains incorrect
information. This mostly seems to happen when things are pear shaped
(couple of servers down, shards recovering, that sort of thing).

Here's an example where a shard is supposedly active, while its only two
replicas are down. Because ... two wrongs .. make a right? Ahem.

"foobar01":{
   "range":null,
   "state":"active", > WTF?
   "replicas":{
  "core_node362":{
 "core":"foobar_foobar01_replica_n361",
 "base_url":"http://1.2.3.4:8180/solr";,
 "node_name":"1.2.3.4:8180_solr",
 "state":"down", > WTF?
 "type":"NRT",
 "force_set_state":"false"
  },
  "core_node364":{
 "core":"foobar_foobar01_replica_n363",
 "base_url":"http://5.6.7.8:8180/solr";,
 "node_name":"5.6.7.8:8180_solr",
 "state":"down", > WTF?
 "type":"NRT",
 "force_set_state":"false"
  }
   }
}

Is this a bug? Couldn't find anything relevant on Jira. Is there any way
we can force solr to compute a correct cluster status? I don't think we
can reproduce this reliably, but it's nevertheless an annoyance.

Thanks,

 - Bram

MaxConnections and MaxconnectionsPerHost. (Solr 4 and Solr 8)

[Reej M]

Hi Team,

Want an advise on this property maxConnections and maxconnectionsperhost.
We were  using solr 4 and on searching in google I found the default max 
connections per host is 20 in solr4 , and it is 10in solr 8. Can you please 
help me to understand the below
1. So in Solr 4, only 20 requests per hosts are allowed at a time (document 
does not have a mention about this)
2. If Solr 8 allows 10 connections per host and will that not slow down the 
system.
maximumPoolSize - The maximum size of the threadpool servicing requests. 
Default is unlimited - So if the requesting application is WebLogic, so it 
depends on the WebLogic thread pool size?

Kindly advise if we need to limit all these values in solr8 so it does not slow 
down the system, because we are moving to solr 8 and the WebLogic Server is 
going down within 30 minutes and we dont see any errors in WebLogic side or 
solr side, so we think the connections are the issue , but solr 4 we didn’t 
have any issues with the connection

Thanks
Reej

Re: Incorrect status in CLUSTERSTATUS

[Cassandra Targett]

It is probably more of a design flaw than an outright bug, and I think it was 
addressed with https://issues.apache.org/jira/browse/SOLR-15300 (released in 
8.9).

As explained in that issue, that `state` flag was only meant to reflect the 
state during SPLITSHARD operations, but that’s actually a really rare thing and 
if we’re going to have a `state` flag it would be better if it reflected the 
real state on an ongoing basis.

Cassandra
On Jul 15, 2021, 4:52 AM -0500, Bram Van Dam , wrote:
> Howdy,
>
> Solr 7.7, in Cloud, with ~400 cores spread over 6 chunky servers.
>
> We've noticed that CLUSTERSTATUS occasionally contains incorrect
> information. This mostly seems to happen when things are pear shaped
> (couple of servers down, shards recovering, that sort of thing).
>
> Here's an example where a shard is supposedly active, while its only two
> replicas are down. Because ... two wrongs .. make a right? Ahem.
>
> "foobar01":{
> "range":null,
> "state":"active", > WTF?
> "replicas":{
> "core_node362":{
> "core":"foobar_foobar01_replica_n361",
> "base_url":"http://1.2.3.4:8180/solr";,
> "node_name":"1.2.3.4:8180_solr",
> "state":"down", > WTF?
> "type":"NRT",
> "force_set_state":"false"
> },
> "core_node364":{
> "core":"foobar_foobar01_replica_n363",
> "base_url":"http://5.6.7.8:8180/solr";,
> "node_name":"5.6.7.8:8180_solr",
> "state":"down", > WTF?
> "type":"NRT",
> "force_set_state":"false"
> }
> }
> }
>
> Is this a bug? Couldn't find anything relevant on Jira. Is there any way
> we can force solr to compute a correct cluster status? I don't think we
> can reproduce this reliably, but it's nevertheless an annoyance.
>
> Thanks,
>
> - Bram

Solr operator ConfigMap support

[Joel Bernstein]

I've been digging into the ConfigMap support in the Solr operator. Based on
the documentation only the solr.xml and log4j2.xml seem to be supported by
the Solr operator. A couple questions about this:

1) Am I correct in my reading of the documentation or is there broader
support for config files?

2) I also see this in the SolrCloud CRD:


*"If unspecified, each key-value pair in the Data field of the referenced
ConfigMap will be projected into the volume as a file whose name is the key
and content is the value. If specified, the listed keys will be projected
into the specified paths, and unlisted keys will not be present. If a key
is specified which is not present in the ConfigMap, the volume setup will
error unless it is marked optional. Paths must be relative and may not
contain the '..' path or start with '..'."*

This seems to imply broader support, but the ConfigMap docs reference a
different part of the CRD under "configMapOptions".

Thanks!

Joel Bernstein
http://joelsolr.blogspot.com/

CDCR replacement -- Timelines

[Natarajan, Rajeswari]

Hi ,
Saw in Solr Documentations that Solr community is working to identify the best 
recommended replacement in time for 9.0.
What are the timelines for 9.0 release . We are currently using CDCR and have 
now started looking at the alternatives.


Thanks,
Rajeswari

Re: Solr operator ConfigMap support

[Houston Putman]

Hey Joel,

   1. You are correct, only solr.xml and log4j2.xml are currently supported.
   2. Are you looking at the
   "SolrCloud.spec.customSolrKubeOptions.podOptions.volumes" section?
   If so, then yes, this is a way to mount whatever extra volumes you want
   to add to your Solr container,
   including a configMap if you so desire. However, that doesn't mean that
   Solr will do anything with the data
   you have loaded into the container. It will sit there and wait for you
   to do something with it. In case of the new option
   of loading configSets from disk, this might be useful for that whenever
   the Solr feature is merged and released.
   This option has nothing to do with Solr, but is just a way for you to
   customize your pod, i.e. adding extra volumes.

The broad answer here is that the Solr Operator doesn't want to be in the
business of managing internal Solr resources like a Config Set.
We have already removed Collection and Alias support from the operator
because it wasn't a good fit for the project. (I can expand on this if you
would like)

Our stance currently is to encourage users to use the ConfigSet API
 for any configSet
management they want to do, much like we encourage
users to use the Collections API for managing Collections & Aliases.

- Houston

On Thu, Jul 15, 2021 at 9:58 AM Joel Bernstein  wrote:

> I've been digging into the ConfigMap support in the Solr operator. Based on
> the documentation only the solr.xml and log4j2.xml seem to be supported by
> the Solr operator. A couple questions about this:
>
> 1) Am I correct in my reading of the documentation or is there broader
> support for config files?
>
> 2) I also see this in the SolrCloud CRD:
>
>
> *"If unspecified, each key-value pair in the Data field of the referenced
> ConfigMap will be projected into the volume as a file whose name is the key
> and content is the value. If specified, the listed keys will be projected
> into the specified paths, and unlisted keys will not be present. If a key
> is specified which is not present in the ConfigMap, the volume setup will
> error unless it is marked optional. Paths must be relative and may not
> contain the '..' path or start with '..'."*
>
> This seems to imply broader support, but the ConfigMap docs reference a
> different part of the CRD under "configMapOptions".
>
> Thanks!
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>

Re: Solr operator ConfigMap support

[Joel Bernstein]

Thanks Houston, this makes perfect sense.

Best.

Joel Bernstein
http://joelsolr.blogspot.com/


On Thu, Jul 15, 2021 at 1:21 PM Houston Putman 
wrote:

> Hey Joel,
>
>1. You are correct, only solr.xml and log4j2.xml are currently
> supported.
>2. Are you looking at the
>"SolrCloud.spec.customSolrKubeOptions.podOptions.volumes" section?
>If so, then yes, this is a way to mount whatever extra volumes you want
>to add to your Solr container,
>including a configMap if you so desire. However, that doesn't mean that
>Solr will do anything with the data
>you have loaded into the container. It will sit there and wait for you
>to do something with it. In case of the new option
>of loading configSets from disk, this might be useful for that whenever
>the Solr feature is merged and released.
>This option has nothing to do with Solr, but is just a way for you to
>customize your pod, i.e. adding extra volumes.
>
> The broad answer here is that the Solr Operator doesn't want to be in the
> business of managing internal Solr resources like a Config Set.
> We have already removed Collection and Alias support from the operator
> because it wasn't a good fit for the project. (I can expand on this if you
> would like)
>
> Our stance currently is to encourage users to use the ConfigSet API
>  for any configSet
> management they want to do, much like we encourage
> users to use the Collections API for managing Collections & Aliases.
>
> - Houston
>
> On Thu, Jul 15, 2021 at 9:58 AM Joel Bernstein  wrote:
>
> > I've been digging into the ConfigMap support in the Solr operator. Based
> on
> > the documentation only the solr.xml and log4j2.xml seem to be supported
> by
> > the Solr operator. A couple questions about this:
> >
> > 1) Am I correct in my reading of the documentation or is there broader
> > support for config files?
> >
> > 2) I also see this in the SolrCloud CRD:
> >
> >
> > *"If unspecified, each key-value pair in the Data field of the referenced
> > ConfigMap will be projected into the volume as a file whose name is the
> key
> > and content is the value. If specified, the listed keys will be projected
> > into the specified paths, and unlisted keys will not be present. If a key
> > is specified which is not present in the ConfigMap, the volume setup will
> > error unless it is marked optional. Paths must be relative and may not
> > contain the '..' path or start with '..'."*
> >
> > This seems to imply broader support, but the ConfigMap docs reference a
> > different part of the CRD under "configMapOptions".
> >
> > Thanks!
> >
> > Joel Bernstein
> > http://joelsolr.blogspot.com/
> >
>

MultipleAdditiveTreeModel

[Roopa Rao]

Hi All,

In LTR for MultipleAdditiveTreeModel what is the purpose of adding
NODE_SPLIT_SLACK
to the threshold?

Reference: org.apache.solr.ltr.model.MultipleAdditiveTreesModel

private static final float NODE_SPLIT_SLACK = 1E-6f;


public void setThreshold(float threshold) { this.threshold = threshold +
NODE_SPLIT_SLACK; }

We have a feature which can return 0.0 or 1.0

And model with this tree:

is_xyz_feature,threshold=0.9994,left=0.0010180053,right=-0.0057609854

However when Solr actually scores it it is taking it as follows
is_xyz_feature:1.0<= 1.01, Go Left

So all the time it goes to left which is incorrect.

Thanks,
Roopa

MultipleAdditiveTreeModel

[Roopa ML]

Hi All,
> 
> In LTR for MultipleAdditiveTreeModel what is the purpose of adding 
> NODE_SPLIT_SLACK to the threshold?
> 
> Reference: org.apache.solr.ltr.model.MultipleAdditiveTreesModel
> 
> private static final float NODE_SPLIT_SLACK = 1E-6f;
> 
> public void setThreshold(float threshold) {
>   this.threshold = threshold + NODE_SPLIT_SLACK;
> }
> 
> We have a feature which can return 0.0 or 1.0
> 
> And model with this tree:
> 
> is_xyz_feature,threshold=0.9994,left=0.0010180053,right=-0.0057609854
> 
> However when Solr actually scores it it is taking it as follows
> is_xyz_feature:1.0<= 1.01, Go Left
> 
> So all the time it goes to left which is incorrect.
> 
> Thanks,
> Roopa
> 
>