Backup from old server and Restore to new server

2023-09-25 Thread Jim Morgan
Hi, I inherited a solr installation from a previous sysadmin, and we'd like to see if we can move it from the current server to a new one. I don't have much knowledge of solr, but I know my way around Linux. Apologies in advance for any dumb questions, and please feel free to educate me. Old insta

Re: Backup from old server and Restore to new server

2023-09-25 Thread Vincenzo D'Amore
Hi Jim, how many cores (indexes) do you have in this Solr server? On Mon, Sep 25, 2023 at 9:41 AM Jim Morgan wrote: > Hi, > I inherited a solr installation from a previous sysadmin, and we'd like to > see if we can move it from the current server to a new one. I don't have > much knowledge of s

Re: Backup from old server and Restore to new server

2023-09-25 Thread Jan Høydahl
I highly recommend doing a full re-index from scratch to the new cluster. Then, test it in your dev/test environment. And finally, switch your search app to the new server. Jan > 25. sep. 2023 kl. 09:40 skrev Jim Morgan : > > Hi, > I inherited a solr installation from a previous sysadmin, and we

Re: Backup from old server and Restore to new server

2023-09-25 Thread Jim Morgan
Quick response! There are two cores. But I think only one of them is actively in use. I think the other one doesn't need to be migrated. But will know for sure when I manage it and can test the app with the new server! Core in use seems to be about 12Gb. On Mon, Sep 25, 2023 at 3:48 PM Vincenzo D'

Re: Backup from old server and Restore to new server

2023-09-25 Thread Jim Morgan
Thanks Jan. I've read that around the internet too. Are you saying that I should not use the backup/restore method, but INSTEAD, I should be trying to create the core on the new server and get the data into it by running a re-index, rather than copying it across? So if that's what you're saying, wh

Re: Backup from old server and Restore to new server

2023-09-25 Thread Jan Høydahl
Hi, Absolutely. And if your company lacks documentation and routines for installing and configuring this piece of software, it is an excellet opportuninty to fill that gap by doing it step by step. While you're at it, you can also read the upgrade notes https://solr.apache.org/guide/solr/latest

Re: Backup from old server and Restore to new server

2023-09-25 Thread Jim Morgan
That was my feeling as well. I could have just cloned the server, but doing it this way, I can document the procedure (and maybe even automate it in the future), so it never becomes an obstacle again. And also we get the latest version of software on the latest OS. So one further question, if you

Re: Backup from old server and Restore to new server

2023-09-25 Thread Vincenzo D'Amore
:) looking at solr.log you can see the requests and, even if you cannot be completely sure, which core is currently receiving requests. Before moving to a new solr version I should understand how is your server average load, how many requests per second your server is handling. Honestly I'm not sur

Re: Backup from old server and Restore to new server

2023-09-25 Thread Jim Morgan
Thanks. Good tip for the logs. Although they didn't prove very useful in determining what the problem with the import was: absolutely nothing was logged! Server load: I have that under control in terms of RAM and hard disk space and that won't be an issue. Is there a command to find out how many r

Re: Backup from old server and Restore to new server

2023-09-25 Thread Vincenzo D'Amore
In the past, when upgrading to a major version, well... sometimes it happened, you can even have a loss of performance. So before moving I would be sure if I can and what I gain from it. For example in the Solr 9.x the Data Import Handler has been moved out to an external component, still open sour

Re: Backup from old server and Restore to new server

2023-09-25 Thread Jan Høydahl
This is hard to tell whithout knowing what data you index to Solr and how. If you use Data Import Handler, that is no longer part of 9.0 so you definitely need to plan for how to proceed. You'll find tons of email threads on it in the archives. If you have an external indexing process, you simply

A general question about update ordering

2023-09-25 Thread Shawn Heisey
I have a general question about update ordering. Situation: Two indexing requests for the same uniqueKey value sent. One update "creates" the document, the other is an "update" with more fields populated. The "update" is sent by a different system than the "create" ... and it turns out that

Re: A general question about update ordering

2023-09-25 Thread Dmitri Maziuk
On 9/25/23 08:24, Shawn Heisey wrote: ... Solr 4.7 handles this as the customer wants, but Solr 9.1 doesn't. This sounds like a race condition, are you sure it's Solr doing the ordering and not e.g. the network stack? What I found is that when the "add" request comes first, 8.11.2 (at least

Re: A general question about update ordering

2023-09-25 Thread Walter Underwood
Updates and indexing used to be single-threaded. A while ago (maybe in Solr 6.x?), a bunch of locking was removed so that indexing could use multiple CPUs. That is probably what you are seeing. I don’t think Solr has ever guaranteed ordering for updates, even if it happened to work. Maybe use

Re: A general question about update ordering

2023-09-25 Thread Shawn Heisey
On 9/25/23 08:53, Dmitri Maziuk wrote: On 9/25/23 08:24, Shawn Heisey wrote: ... Solr 4.7 handles this as the customer wants, but Solr 9.1 doesn't. This sounds like a race condition, are you sure it's Solr doing the ordering and not e.g. the network stack? I have seen the requests in the lo

Re: A general question about update ordering

2023-09-25 Thread Dmitri Maziuk
On 9/25/23 10:49, Shawn Heisey wrote: On 9/25/23 08:53, Dmitri Maziuk wrote: ... What I found is that when the "add" request comes first, 8.11.2 (at least) will create a new -- useless -- document for it. I wish it failed instead. You might be able to achieve the behavior you want with the o

Re: A general question about update ordering

2023-09-25 Thread Shawn Heisey
On 9/25/23 09:44, Walter Underwood wrote: Updates and indexing used to be single-threaded. A while ago (maybe in Solr 6.x?), a bunch of locking was removed so that indexing could use multiple CPUs. That is probably what you are seeing. I don’t think Solr has ever guaranteed ordering for update

Re: A general question about update ordering

2023-09-25 Thread Shawn Heisey
On 9/25/23 10:24, Dmitri Maziuk wrote: (From your description, relying on the 4.x behaviour was a bad idea in the first place: what if "create" request e.g. hit a bad sector on a non-TLER drive and was stuck on i/o for 100x longer?) I agree that the way they are operating now is completely wro

Re: A general question about update ordering

2023-09-25 Thread Dmitri Maziuk
On 9/25/23 11:35, Shawn Heisey wrote: I agree that the way they are operating now is completely wrong, and I am surprised that it even worked.  They got VERY lucky. http://www.catb.org/jargon/html/S/schroedinbug.html cheers, Dima