Re: Authentication problems

[MEXANIK]

Jan, Hi!!

I have such a problem that out of 5 collections, 2 of them can be read
using the john_sl user with admin_x accesses. But if the "admin_rwx"
accesses are taken away from the "admin" user, then the john_sl user will
not be able to read 2 collections previously available for reading ... How
is this understand? any ideas?

пн, 3 апр. 2023 г., 23:34 Jan Høydahl :

> Hi,
>
> I recommend reading the docs thoroughly and then clean up your config
> somewhat:
> https://solr.apache.org/guide/solr/latest/deployment-guide/rule-based-authorization-plugin.html
>
> Solr's authz works differently than most other RBAC products. You may be
> confused by order of evaluation, which is a bit upside down.
>
> Solr does NOT start with the user's role and then evaluate what
> paths/permissions he can do.
> On the contrary, Solr starts with the request path, e.g.
> /select, then hunts through your permissions:[] array
> top-to-bottom to find ONE SINGLE permission that matches the path (and
> optionally collection name), and once it finds that permission, it will
> check that the user has one of the roles listed in that permission (or
> "all").
>
> Also, if you do not list every single path or predefined permission, then
> any path not listed will be allowed by default, which is scary. It is
> common practice to have an "all" permission at the very end, and have that
> one require some kind of admin role. Looks like you have that.
>
> In your security.json you list a "read" permission several times, and also
> other permissions several times. Solr will only consider the FIRST which
> satisfies the request. So make sure to place the more specific and
> restrictive permissions on top, and then you can have "fall-through"
> permissions near the end.
>
> I also see that you use a custom Auth plugin, but I assume that one works.
> If you face continued problems I recommend creating a reproduction case
> with BasicAuth and as few roles/permissions as possible to reproduce your
> issue. Then others can try out your config and help you find flaws.
>
> Jan
>
> > 3. apr. 2023 kl. 12:24 skrev MEXANIK :
> >
> > Jan, Hi, Thank you
> >
> > Need more information so you can help?
> >
> >
> >
> > пт, 31 мар. 2023 г., 12:16 MEXANIK :
> >
> >> Sorry, test1Collection*
> >>
> >> чт, 30 мар. 2023 г., 17:14 MEXANIK :
> >>
> >>> When you send a read request for the test2Collection collection, the
> logs
> >>> do not display as a collection, and I get 10 entries with admin_x
> rights
> >>>
> >>>
> >>> attached log with description
> >>>
> >>>
> >>>
> >>> ср, 29 мар. 2023 г., 16:39 Jan Høydahl :
> >>>
>  Permissions are evaluated in order from top to bottom.
>  The first "read" permission found requires roles "admin_ro",
>  "admin_rwx", "solr-internal-traffic", so that should be selected.
> 
>  Do you have any logs that can shed light over what happens?
> 
>  Jan
> 
> > 29. mar. 2023 kl. 14:27 skrev MEXANIK :
> >
> > I logged in using the john_sl user in Solr UI, and sent a request to
>  read
> > the collection, but I can read some of the collections, but I don't
>  want
> > them to be read.
> >
> > If I replace the admin user's rights from admin_rwx to admin_x, then
>  the
> > john_sl user can't read either
> >
> > How do I make it so that john_sl cannot read collections with admin_x
> > rights, but at the same time so that the admin user has admin_rwx
>  rights??
> >
> > Help!!
> >
> > Example security.json http://replit.com/@irkuev666/Test#data.json
> 
> 
>
>

Re: Authentication problems

[Jan Høydahl]

I think the way forward here is to create a minimal re-production example for 
others to try.
Ideally using a setup with basic auth, you could help us reproduce with a 
script like this:

Copy my-security.json into current folder
bin/solr start -c (or using docker)
solr create -c coll1
solr create -c coll2
solr create -c coll3
# Enable security with custom config
bin/solr zk cp my-security.json zk:/security.json

Browse to http://localhost:8983 and login with user 'foo'

Verify that user 'foo' can search collection coll1:
curl "http://localhost:8983/solr/coll1/select?q=*:*";

Verify that user 'foo cannot search collection coll2:
curl "http://localhost:8983/solr/coll2/select?q=*:*";

Jan

> 4. apr. 2023 kl. 12:27 skrev MEXANIK :
> 
> Jan, Hi!!
> 
> I have such a problem that out of 5 collections, 2 of them can be read
> using the john_sl user with admin_x accesses. But if the "admin_rwx"
> accesses are taken away from the "admin" user, then the john_sl user will
> not be able to read 2 collections previously available for reading ... How
> is this understand? any ideas?
> 
> пн, 3 апр. 2023 г., 23:34 Jan Høydahl :
> 
>> Hi,
>> 
>> I recommend reading the docs thoroughly and then clean up your config
>> somewhat:
>> https://solr.apache.org/guide/solr/latest/deployment-guide/rule-based-authorization-plugin.html
>> 
>> Solr's authz works differently than most other RBAC products. You may be
>> confused by order of evaluation, which is a bit upside down.
>> 
>> Solr does NOT start with the user's role and then evaluate what
>> paths/permissions he can do.
>> On the contrary, Solr starts with the request path, e.g.
>> /select, then hunts through your permissions:[] array
>> top-to-bottom to find ONE SINGLE permission that matches the path (and
>> optionally collection name), and once it finds that permission, it will
>> check that the user has one of the roles listed in that permission (or
>> "all").
>> 
>> Also, if you do not list every single path or predefined permission, then
>> any path not listed will be allowed by default, which is scary. It is
>> common practice to have an "all" permission at the very end, and have that
>> one require some kind of admin role. Looks like you have that.
>> 
>> In your security.json you list a "read" permission several times, and also
>> other permissions several times. Solr will only consider the FIRST which
>> satisfies the request. So make sure to place the more specific and
>> restrictive permissions on top, and then you can have "fall-through"
>> permissions near the end.
>> 
>> I also see that you use a custom Auth plugin, but I assume that one works.
>> If you face continued problems I recommend creating a reproduction case
>> with BasicAuth and as few roles/permissions as possible to reproduce your
>> issue. Then others can try out your config and help you find flaws.
>> 
>> Jan
>> 
>>> 3. apr. 2023 kl. 12:24 skrev MEXANIK :
>>> 
>>> Jan, Hi, Thank you
>>> 
>>> Need more information so you can help?
>>> 
>>> 
>>> 
>>> пт, 31 мар. 2023 г., 12:16 MEXANIK :
>>> 
 Sorry, test1Collection*
 
 чт, 30 мар. 2023 г., 17:14 MEXANIK :
 
> When you send a read request for the test2Collection collection, the
>> logs
> do not display as a collection, and I get 10 entries with admin_x
>> rights
> 
> 
> attached log with description
> 
> 
> 
> ср, 29 мар. 2023 г., 16:39 Jan Høydahl :
> 
>> Permissions are evaluated in order from top to bottom.
>> The first "read" permission found requires roles "admin_ro",
>> "admin_rwx", "solr-internal-traffic", so that should be selected.
>> 
>> Do you have any logs that can shed light over what happens?
>> 
>> Jan
>> 
>>> 29. mar. 2023 kl. 14:27 skrev MEXANIK :
>>> 
>>> I logged in using the john_sl user in Solr UI, and sent a request to
>> read
>>> the collection, but I can read some of the collections, but I don't
>> want
>>> them to be read.
>>> 
>>> If I replace the admin user's rights from admin_rwx to admin_x, then
>> the
>>> john_sl user can't read either
>>> 
>>> How do I make it so that john_sl cannot read collections with admin_x
>>> rights, but at the same time so that the admin user has admin_rwx
>> rights??
>>> 
>>> Help!!
>>> 
>>> Example security.json http://replit.com/@irkuev666/Test#data.json
>> 
>> 
>> 
>> 

Date Range Facet sort order

[Thomas Redman]

I am using the SolrJ Json facet API to retrieve a facet on a range of date.

DateFacet df = (DateFacet)f;
RangeFacetMap rfm = new RangeFacetMap(df.getField(), 
df.getStartDate(), df.getEndDate(),
df.getSpan().getKeyword())
.setMinCount(1);

I see no way to change the sort order to index (I want the output in date 
order), I get results sorted on the count. I have tried adding the sort field 
like so:

rfm.put("sort","count asc");

but that had no effect. How does one change the sort order of the date range 
facets using the Json facet API?

Solr post, update "quiet" mode (warnings, errors only)?

[Victoria Stuart (VictoriasJourney.com)]

When I post to / update my Solr core is there a quiet (errors, warnings) only 
mode?

I use the CLI / terminal, e.g. (illustration, here)

  ... solr-core-8.11.1.jar  -Dauto=yes  -Dbasicauth=***:***  \
  -Durl=MYCORE/update  -Dc=MYCORE  \
  -Ddata=files org.apache.solr.util.SimplePostTool  MYDOCS

I want to hide thousands of these successful posts

  POSTing file MYFILE (text/html) to [base]/extract

but show / log errors, warnings such as these

  SimplePostTool: WARNING: Solr returned an error #400 (Bad Request) for url: 
...

==

Query on Solr and ZK ports

[HariBabu kuruva]

Hi All,

I could see the solr process is Listening on 7981 port along with the
normal solr port(8981). It is shown as DSTOP PORT , when I grep solr
process. Could you please give more details on this port, Can we disable
this ?

With regards to Zookeeper I could see port 8080 as a ZK admin port, How can
I use this, can i disable it if I don't want it ?
Also I could see ZK is listening on some random port (43801) along with the
other ports. Please throw some light on this.

-- 

Thanks and Regards,
 Hari
Mobile:9790756568

Re: Solr post, update "quiet" mode (warnings, errors only)?

[Victoria Stuart (VictoriasJourney.com)]

... never mind; I'll just "tee" the output to a file and grep for errors, 
warnings.

==
- Original Message(s): -
Subject: Solr post, update "quiet" mode (warnings, errors only)?

When I post to / update my Solr core is there a quiet (errors, warnings) only 
mode?

I use the CLI / terminal, e.g. (illustration, here)

  ... solr-core-8.11.1.jar  -Dauto=yes  -Dbasicauth=***:***  \
  -Durl=MYCORE/update  -Dc=MYCORE  \
  -Ddata=files org.apache.solr.util.SimplePostTool  MYDOCS

I want to hide thousands of these successful posts

  POSTing file MYFILE (text/html) to [base]/extract

but show / log errors, warnings such as these

  SimplePostTool: WARNING: Solr returned an error #400 (Bad Request) for url: 
...

==

Re: SOLR 9.2.0 cluster basic authentication with http2 solr client

[Shawn Heisey]

On 4/4/23 00:11, Andrei Minin wrote:

  protected SolrClient getSolrClient(String baseUrl) {
 return new 
Http2SolrClient.Builder(baseUrl).withHttpClient(httpClient).build();
   }


That's not good.  This patch would fix it:

https://paste.elyograg.org/view/456026f5

A similar patch would be needed for HttpClusterStateProvider.

Until we can get this done, can you try building the client using the ZK 
cluster info instead of URLs?  I suspect that might work, and if it 
doesn't, we'll need to fix that too.


For your other issue you mentioned ... if you were trying to use 
Http2SolrClient to talk to a Solr URL using https and getting an 
exception about a missing SSL context, that problem has been fixed in 
SolrJ 9.2.0.  There is a workaround for earlier 9.x releases.


Thanks,
Shawn

Re: SOLR 9.2.0 cluster basic authentication with http2 solr client

[Tomás Fernández Löbbe]

> Looking at the code in CloudHttp2SolrClient on main, branch_9x, and
> branch_9_2, the effective result of these two different options appear
> to be identical, the only difference being that the clientIsInternal
> boolean is false for the first option and true for the second.  I would
> not expect the described change to help.  I do hope I am wrong!
>
>
They aren't the same, the whole point is that, by creating the "myClient"
with the externally provided builder, the CloudHttp2SolrClient can use
whatever configuration was set in the builder, including the basic
authentication configuration, instead of using an all-default client.

Re: SOLR 9.2.0 cluster basic authentication with http2 solr client

[Tomás Fernández Löbbe]

Yes, good catch Andrei!
Mind filing a Jira issue?

On Tue, Apr 4, 2023 at 10:46 AM Shawn Heisey  wrote:

> On 4/4/23 00:11, Andrei Minin wrote:
> >   protected SolrClient getSolrClient(String baseUrl) {
> >  return new
> Http2SolrClient.Builder(baseUrl).withHttpClient(httpClient).build();
> >}
>
> That's not good.  This patch would fix it:
>
> https://paste.elyograg.org/view/456026f5
>
> A similar patch would be needed for HttpClusterStateProvider.
>
> Until we can get this done, can you try building the client using the ZK
> cluster info instead of URLs?  I suspect that might work, and if it
> doesn't, we'll need to fix that too.
>
> For your other issue you mentioned ... if you were trying to use
> Http2SolrClient to talk to a Solr URL using https and getting an
> exception about a missing SSL context, that problem has been fixed in
> SolrJ 9.2.0.  There is a workaround for earlier 9.x releases.
>
> Thanks,
> Shawn
>

Re: Date Range Facet sort order

[Chris Hostetter]

: I see no way to change the sort order to index (I want the output in 
: date order), I get results sorted on the count. I have tried adding the 
: sort field like so:

This ... doesn't make sense.  

AFAIK "index" order (or more specifically: The "natural order" of the 
bucket values) is the only possible sorting returned by JSON Range Facet 

ie: what you are asking for should be what you are getting, what you say 
you are getting shouldn't be possible.
(see examples below)


What does your solrj code that consumes the QueryResponse look like?


$ curl 'http://localhost:8983/solr/techproducts/query?rows=0&omitHeader=true' 
-d '
{
  "query": "*:*",
  "facet": {
"prices": {
  "type": "range",
  "field": "price",
  "start": 0,
  "end": 100,
  "gap": 20
}
  }
}'
{
  "response":{"numFound":32,"start":0,"numFoundExact":true,"docs":[]
  },
  "facets":{
"count":32,
"prices":{
  "buckets":[{
  "val":0.0,
  "count":5},
{
  "val":20.0,
  "count":0},
{
  "val":40.0,
  "count":0},
{
  "val":60.0,
  "count":1},
{
  "val":80.0,
  "count":1}]}}}


$ curl 
'http://localhost:8983/solr/techproducts/query?rows=0&omitHeader=true' -d'
{
  "query": "*:*",
  "facet": {
"prices": {
  "type": "range",
  "field": "manufacturedate_dt",
  "start": "1900-01-01T00:00:00Z",
  "end": "2200-01-01T00:00:00Z",
  "gap": "+100YEARS"
}
  }
}'
{
  "response":{"numFound":32,"start":0,"numFoundExact":true,"docs":[]
  },
  "facets":{
"count":32,
"prices":{
  "buckets":[{
  "val":"1900-01-01T00:00:00Z",
  "count":0},
{
  "val":"2000-01-01T00:00:00Z",
  "count":11},
{
  "val":"2100-01-01T00:00:00Z",
  "count":0}]}}}

Here's a test that uses SollrJ to assert that the execpted buckets come 
back in the expected order (by bucket value, not by bucket count) ...

https://github.com/apache/solr/blob/releases/solr/9.0.0/solr/solrj/src/test/org/apache/solr/client/ref_guide_examples/JsonRequestApiTest.java#L573-L596






-Hoss
http://www.lucidworks.com/

Re: SOLR 9.2.0 cluster basic authentication with http2 solr client

[Andrei Minin]

Ticket created: https://issues.apache.org/jira/browse/SOLR-16737

Thank you for supporting,


Andrei Minin | Principal Developer

Intellective

ami...@intellective.com

www.intellective.com

Follow us on LinkedIn | 
Twitter | 
Facebook


From: Tomás Fernández Löbbe 
Sent: Tuesday, April 4, 2023 22:22
To: users@solr.apache.org 
Subject: Re: SOLR 9.2.0 cluster basic authentication with http2 solr client

Yes, good catch Andrei!
Mind filing a Jira issue?

On Tue, Apr 4, 2023 at 10:46 AM Shawn Heisey  wrote:

> On 4/4/23 00:11, Andrei Minin wrote:
> >   protected SolrClient getSolrClient(String baseUrl) {
> >  return new
> Http2SolrClient.Builder(baseUrl).withHttpClient(httpClient).build();
> >}
>
> That's not good.  This patch would fix it:
>
> https://paste.elyograg.org/view/456026f5
>
> A similar patch would be needed for HttpClusterStateProvider.
>
> Until we can get this done, can you try building the client using the ZK
> cluster info instead of URLs?  I suspect that might work, and if it
> doesn't, we'll need to fix that too.
>
> For your other issue you mentioned ... if you were trying to use
> Http2SolrClient to talk to a Solr URL using https and getting an
> exception about a missing SSL context, that problem has been fixed in
> SolrJ 9.2.0.  There is a workaround for earlier 9.x releases.
>
> Thanks,
> Shawn
>