Solr configured for SSL always using plain text when talking to Zookeeper secure port
Hello, I have a question regarding Solr and Zookeeper configured for SSL. Simply, does Solr default to non-SSL when connecting with the Zookeeper secure port on localhost? Solr 8.9, Zookeeper 3.6.2 One Zookeeper instance configured for SSL One Solr instance configured for SSL, pointing at ZK secure port, started in Cloud mode. I have created self-signed certificates and keys for both Zookeeper and Solr. The truststore for both contains all the necessary certificates. I can configure Zookeeper for SSL and connect zkCli.sh to the secure port successfully. I can configure Solr for SSL and run it connecting to the standard ZK port (2181); the admin UI loads with https, etc. I have set the clusterprops.json urlScheme = https Whenever I configure Solr to connect on the secure ZK port, however, I see this in the Solr.log: o.a.z.ClientCnxn Opening socket connection to server localhost/127.0.0.1:2281 o.a.z.ClientCnxn SASL config status: Will not attempt to authenticate using SASL (unknown error) o.a.z.ClientCnxn Socket connection established, initiating session, client: /127.0.0.1:58062, server: localhost/127.0.0.1:2281 o.a.z.ClientCnxn Session 0x0 for server localhost/127.0.0.1:2281, Closing socket connection. Attempting reconnect except it is a SessionExpiredException => EndOfStreamException: Unable to read additional data from server sessionid 0x0, likely server has closed socket (These lines are separated by a few ms, no more) And the Zookeeper log has an error for each call: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record This is making me think Solr defaults to plain text on localhost regardless of setup. Regards Rob
Re: Solr configured for SSL always using plain text when talking to Zookeeper secure port
Hi, What happens when you point Solr to ZK secure port is that Solr tries to talk non-SSL TCP with the secure (HTTP) port and it fails. See more at https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide. You should be able to configure a set of system props on Solr side which instructs the Zookeeper Java client to talk SSL with Zookeeper, but you may need to add some additional jar files to Solr for it to work. See also https://issues.apache.org/jira/browse/SOLR-8342 and https://issues.apache.org/jira/browse/SOLR-7893 for an attempt to start documenting this in RefGuide and making some tool support for configuring. Contributions welcome! Jan > 16. sep. 2021 kl. 14:56 skrev Robert Pearce : > > Hello, > > I have a question regarding Solr and Zookeeper configured for SSL. Simply, > does Solr default to non-SSL when connecting with the Zookeeper secure port > on localhost? > > Solr 8.9, Zookeeper 3.6.2 > One Zookeeper instance configured for SSL > One Solr instance configured for SSL, pointing at ZK secure port, started in > Cloud mode. > > I have created self-signed certificates and keys for both Zookeeper and Solr. > The truststore for both contains all the necessary certificates. > I can configure Zookeeper for SSL and connect zkCli.sh to the secure port > successfully. > I can configure Solr for SSL and run it connecting to the standard ZK port > (2181); the admin UI loads with https, etc. > I have set the clusterprops.json urlScheme = https > > Whenever I configure Solr to connect on the secure ZK port, however, I see > this in the Solr.log: > > o.a.z.ClientCnxn Opening socket connection to server localhost/127.0.0.1:2281 > o.a.z.ClientCnxn SASL config status: Will not attempt to authenticate using > SASL (unknown error) > o.a.z.ClientCnxn Socket connection established, initiating session, client: > /127.0.0.1:58062, server: localhost/127.0.0.1:2281 > o.a.z.ClientCnxn Session 0x0 for server localhost/127.0.0.1:2281, Closing > socket connection. Attempting reconnect except it is a > SessionExpiredException => EndOfStreamException: Unable to read additional > data from server sessionid 0x0, likely server has closed socket > > (These lines are separated by a few ms, no more) > > And the Zookeeper log has an error for each call: > io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record > > This is making me think Solr defaults to plain text on localhost regardless > of setup. > > Regards > > Rob
Re: Index dependent groups of data
As Shawn explained, when a TLOG replica is not the leader, it does not index the documents directly but pulls index segments from the leader. However, this operation is generally rather fast - within a second or two - since it copies the changed segments, not the full index (and 70 million docs isn’t usually all that big anyway). I didn’t see where you said how soon you attempt to query the replicas, that might be helpful to know to understand if you really do have a problem. It’s important to note that a leader does not notify its replicas that it has new segments. This is controlled by the commit configuration - non-leader replicas poll their leader for possible changes at half the interval set for autoCommits, and if that is not set, then half the autoSoftCommit time. Whether you commit or not on the leader has nothing to do with what the replica does - it’s what has been configured for those params. Other factors may make the replication take longer, particularly a slow network or if you’re merging segments down to a single very large segment (then each replica has to pull the entire index every time you make an update). You can see when replication happens by looking at the logs for one of the replicas - if you have the default logging levels enabled, you’ll clearly see messages about polling for new segments, and when replication starts and finishes. If you make an index update and don’t see a replica poll within a reasonable amount of time, you may want to change those commit settings I mentioned. If you see it start but take a long time, it’s more likely the network or you’ve merged down to a big segment that takes a while over the wire. It seems inefficient to me to check who is a leader and only query the leader - that feels very much a workaround for an otherwise misconfigured cluster. The main reason for having replicas is to distribute the query load, and if you direct all your queries to leaders the replicas are basically doing nothing but waiting in case the leader goes down (which, however, is fine if all you care about is disaster recovery, but there are probably lighter weight approaches if that’s your only need). On Sep 8, 2021, 10:02 AM -0500, lstusr 5u93n4 , wrote: > > Info you might already know: TLOG (and PULL) replicas do not index, > > unless a TLOG replica is the leader, in which case it behaves exactly > > like NRT. A PULL replica can never become leader. > > > > When you have TLOG or PULL replicas, Solr is only going to do indexing > > on the shard leaders. When a commit finishes, it should be done on all > > cores that participate in indexing. > > > > Replication of the completed index segments to TLOG and PULL replicas > > will happen AFTER the commit is done, not concurrently. I don't think > > there's a reliable way of asking Solr to tell you when all replications > > are complete. > > Thanks Shawn, it's good to have this all spelled out. Validates what we're > seeing. > > > Does your "query only the leaders" code check clusterstate in ZK to > > figure out which replicas are leader? Leaders can change in response to > > problems. > > Yeah, exactly. Working implementation is to check > `/collections//state.json` in ZK to determine the leaders, and put a > watch on that node to react if the cluster state changes. > > I see what you're saying about determining if the replications are > complete. However, querying the leaders post-commit is good enough for our > particular use case, so we'll opt to keep the indexing speed as high as > possible and not wait on the replication before proceeding to the next > group of data. > > Thanks for all your help! > > Kyle > > > On Tue, 7 Sept 2021 at 17:13, Shawn Heisey wrote: > > > On 9/7/2021 3:08 PM, Shawn Heisey wrote: > > > I don't think there's a reliable way of asking Solr to tell you when > > > all replications are complete. > > > > > > You could use the replication handler (/solr/corename/replication) to > > gather this info and compare info from the leader index with info from > > the follower index(es). For this to be reliable, you would need to > > check clusterstate in ZK so you're absolutely sure which cores are > > leaders. I do not know off the top of my head what parameters need to > > be sent to the replication handler to gather that info. > > > > Thanks, > > Shawn > > > >
SOLR not indexing empty fields
Dear Team, I working on a project with Sitecore CMS and we use SOLR for indexing. Recently we had to upgrade the SOLR version to 8.4 and after that SOLR stopped indexing string fields when the return value is an empty string. SOLR was indexing the empty strings as "". I have tried out many configuration changes by adding properties like nullValue="NULLVALUE" emptyString="EMPTYVALUE" default="" returnType="string" With the above property, the value is set to "EMPTYVALUE" In case of empty string which was set to "" in SOLR 7.2.1 version. Current Sitecore Version: 10.1.1 Current SOLR version: 8.4 (standalone with 1 master & 2 slaves) Could you please help in sorting this situation? Is this behavior expected? Please let me know if any additional details are required from my end. Thanks and regards Vishnu VG
Re: SOLR not indexing empty fields
Neither 'nullValue' nor 'emptyString' nor 'returnType' are field or fieldtype settings in solr -- which makes me suspect that they must be SiteCore settings, and that the problem you are encountering has something to do with behavior in SiteCore -- not behavior in solr. Solr's StrField in 8.x preserves blank field values by default -- but the 'RemoveBlankFieldUpdateProcessorFactory' update processor may remove them first if it's enabled in solrconfig.xml (which, depending on your solrconfig.xml may be enabled by default, or based on the value of the 'update.autoCreateFields' system property. If you *ONLY* updated Solr, but did not upgrade SiteCore, then I'm going to guess what happened is that you also changed your solr configs (solrconfig.xml, schema, etc...) from the ones you were using with Solr 7.2 to the "defaults" that were included in Solr 8.4 Don't do that -- the new "default" configs will have new, differnet behavior. If you want the same behavior -- use the same configs you were using before. : Subject: SOLR not indexing empty fields : : Dear Team, : : I working on a project with Sitecore CMS and we use SOLR for indexing. : Recently we had to upgrade the SOLR version to 8.4 and after that SOLR : stopped indexing string fields when the return value is an empty string. : SOLR was indexing the empty strings as "". : : I have tried out many configuration changes by adding properties like : nullValue="NULLVALUE" emptyString="EMPTYVALUE" default="" : returnType="string" : : With the above property, the value is set to "EMPTYVALUE" In case of empty : string which was set to "" in SOLR 7.2.1 version. : : Current Sitecore Version: 10.1.1 : Current SOLR version: 8.4 (standalone with 1 master & 2 slaves) : : Could you please help in sorting this situation? Is this behavior expected? : Please let me know if any additional : details are required from my end. : : Thanks and regards : : Vishnu VG : -Hoss http://www.lucidworks.com/
