Re: PlacementPlugin to restrict two different shards of the same collection on the same node

[Jan Høydahl]

Hi,

I do not see this being a factor in the Ref Guide either 
https://solr.apache.org/guide/solr/latest/configuration-guide/replica-placement-plugins.html#affinityplacementfactory

Can you give an example of how the distribution? Were there other candidate 
nodes that had fewer or same number of cores that would be a more natural fit 
for selection?

Jan

> 26. jul. 2023 kl. 19:23 skrev Jing Tie :
> 
> Hello,
> 
> We just migrated to Solr 9.2 and started using
> AffinityPlacementPlugin. In a sharded environment, we made sure there
> is enough storage on each node and enough nodes in each AZ, but it is
> interesting that shard1 and shard2 of the same collection were
> distributed on the same node. In Solr 8, it is by default not allowing
> 2 shards of the same collection on the same node.
> 
> We looked into the PlacementPlugins code, and didn't see any
> restrictions against that. Does anyone have the same issue? Do you
> have any suggestions?
> 
> Thank you very much,
> Jing

Re: what is SolrAuthV2 and why does it break replication

[Jan Høydahl]

> nor does it work to set up security.json to allow replication without a 
> password

Exactly how did you configure security.json for this? It could be that you 
succeeded disabling security for the collection but not for the core URL?

Have you tried to hit 
http://[REDACTED]/solr/sequence2_shard1_replica_n1/replication URL directly 
using cURL without basicAuth?

Jan

> 27. jul. 2023 kl. 18:17 skrev Oakley, Craig (NIH/NLM/NCBI) [C] 
> :
> 
> I have still not received any suggestions or clarifications, so I am 
> resending this with a different subject.
> 
> I found that if I completely eliminate security.json, Leader/Follower 
> replication succeeds; but for obvious reasons, we do want security.json to be 
> there.
> 
> Setting -Dsolr.pki.sendVersion=v1 -Dsolr.pki.acceptVersions=v1,v2 does not 
> help; nor does it work to set up security.json to allow replication without a 
> password and to remove httpBasicAuthUser and httpBasicAuthPassword from 
> solrconfig.xml on the Follower side
> 
> Does anybody have any suggestions?
> 
> -Original Message-
> From: Oakley, Craig (NIH/NLM/NCBI) [C]  
> Sent: Tuesday, July 18, 2023 3:12 PM
> To: users@solr.apache.org
> Subject: RE: authentication for Leader/Follower replication
> 
> I am wondering whether anyone yet has any suggestions how to proceed
> 
> -Original Message-
> From: Oakley, Craig (NIH/NLM/NCBI) [C]  
> Sent: Thursday, July 6, 2023 4:00 PM
> To: users@solr.apache.org
> Subject: authentication for Leader/Follower replication
> 
> We are having problems transitioning Leader/Follower replication to Solr9.2.1
> 
> In Solr8.5 and below, what was then called Master/Slave replication had the 
> annoying problem that, even though we specified httpBasicAuthUser and 
> httpBasicAuthPassword, it would always attempt to connect first without a 
> password before retrying with a password. This made solr.log noisy with lots 
> of unnecessary login failures: but at least it worked.
> 
> When we transitioned to Solr8.11 (with the nomenclature changed to be less 
> oppressive) we found that this version of Leader/Follower replication refused 
> to retry (and refused to do anything with the values specified 
> httpBasicAuthUser and httpBasicAuthPassword). We needed to open up 
> replication in security.json to be available without password.
> 
> Now when we are preparing to upgrade to Solr9.2.1, we are having issues with 
> the following:
> 2023-07-06 15:46:53.315 INFO  (indexFetcher-39-thread-1) [   ] 
> o.a.s.h.IndexFetcher Last replication failed, so I'll force replication
> 2023-07-06 15:46:53.320 WARN  (indexFetcher-39-thread-1) [   ] 
> o.a.s.h.IndexFetcher Leader at: 
> http://[REDACTED]/solr/sequence2_shard1_replica_n1 is not available. Index 
> fetch failed by exception: 
> org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: 
> Error from server at http://[REDACTED]/solr/sequence2_shard1_replica_n1: 
> Expected mime type in [application/octet-stream, 
> application/vnd.apache.solr.javabin] but got text/html. 
> 
> 
> Error 401 Could not load principal from SolrAuthV2 header.
> 
> HTTP ERROR 401 Could not load principal from SolrAuthV2 header.
> 
> URI:/solr/sequence2_shard1_replica_n1/replication
> STATUS:401
> MESSAGE:Could not load principal from SolrAuthV2 
> header.
> SERVLET:default
> 
> 
> 
> 
> 
> I have added "blockUnknown":false to security.json and have confirmed that 
> the replication?command=indexversion command can be run without a password, 
> and that it can be run with the login and password specified in 
> httpBasicAuthUser and httpBasicAuthPassword
> 
> I have tried tweaking security.json with forwardCredentials values, but that 
> has not helped
> 
> Any suggestions?
> 
> 
> 

Re: authentication for Leader/Follower replication

[Shawn Heisey]

On 7/6/23 14:00, Oakley, Craig (NIH/NLM/NCBI) [C] wrote:

We are having problems transitioning Leader/Follower replication to Solr9.2.1

In Solr8.5 and below, what was then called Master/Slave replication had the 
annoying problem that, even though we specified httpBasicAuthUser and 
httpBasicAuthPassword, it would always attempt to connect first without a 
password before retrying with a password. This made solr.log noisy with lots of 
unnecessary login failures: but at least it worked.


In general, this is how basic auth via http works.  The client first 
attempts the request without any credentials, and receives a 401 response.


At this point, a browser would see the 401 response and display a popup 
asking for username/password.


Then on future requests to that server in the current session, the 
browser sends the supplied credentials on every request to that server.


If you are supplying credentials in the replication config, it should 
NOT follow that pattern ... the credentials should be always used.



Now when we are preparing to upgrade to Solr9.2.1, we are having issues with 
the following:
2023-07-06 15:46:53.315 INFO  (indexFetcher-39-thread-1) [   ] 
o.a.s.h.IndexFetcher Last replication failed, so I'll force replication
2023-07-06 15:46:53.320 WARN  (indexFetcher-39-thread-1) [   ] 
o.a.s.h.IndexFetcher Leader at: 
http://[REDACTED]/solr/sequence2_shard1_replica_n1 is not available.


The info above is valid for Solr running in standalone mode.  But those 
core names indicate that you are running in SolrCloud mode.


In cloud mode, Solr handles all replication.  Don't attempt to actually 
configure the replication handler in cloud mode ... Solr will handle it 
all for you, and will even automatically take care of authenticating the 
requests.  You don't need to configure the replication handler at all.


If you are using the replication handler as a "back door" to copy 
indexes to a separate Solr install, a better option is to use the 
backup/restore functionality in the Collections API.


Thanks,
Shawn