Backup from old server and Restore to new server
Hi, I inherited a solr installation from a previous sysadmin, and we'd like to see if we can move it from the current server to a new one. I don't have much knowledge of solr, but I know my way around Linux. Apologies in advance for any dumb questions, and please feel free to educate me. Old install is Solr 8.5 running on Ubuntu 18. We're thinking that while we are switching servers, we might as well try to upgrade to Ubuntu 22 and Solr 9.3 (with corresponding JVM increase to 11). First question, is that a crazy idea? Or should we be moving to the same version of Solr on the new server, and then upgrade in place. We'd prefer to do it with minimal downtime, so setting up parallel servers and migrating the data across seems like the least disruptive. So I have tried the parallel server approach. No issues installing Solr and JVM. From reading around the web, starting with https://solr.apache.org/guide/solr/latest/deployment-guide/backup-restore.html And several other pages, which all seem to repeat the advice, the backup restore process is simple. This is a standalone install so I should use http://localhost:8983/solr/collname/replication?command=backup&location=/snapshots to backup, which works fine: the backup appears in /snapshots/ as snapshot.2023xxx. All good. I take the backup, and copy it to the new server, and put it in the same location, /snapshots/2023xxx Then apparenly I only need to run http://localhost:8983/solr/collname/replication?command=restore&name=snapshot.2023xxx&location=/snapshots But this doesn't work. I've tried a lot of different variations, combining snapshot name and location, with or without the collection name, but all I get is the zen like error Searching for Solr? You must type the correct path. Solr will respond. So that's where I'm at. I'm wondering if I've missed a step that's obvious to Solr veterans, but which is missing from the backup/restore page. Do I need to create the core first, for example. Or create the core and copy config files over to the new server.
Re: Backup from old server and Restore to new server
Hi Jim, how many cores (indexes) do you have in this Solr server? On Mon, Sep 25, 2023 at 9:41 AM Jim Morgan wrote: > Hi, > I inherited a solr installation from a previous sysadmin, and we'd like to > see if we can move it from the current server to a new one. I don't have > much knowledge of solr, but I know my way around Linux. Apologies in > advance for any dumb questions, and please feel free to educate me. > > Old install is Solr 8.5 running on Ubuntu 18. We're thinking that while we > are switching servers, we might as well try to upgrade to Ubuntu 22 and > Solr 9.3 (with corresponding JVM increase to 11). > First question, is that a crazy idea? Or should we be moving to the same > version of Solr on the new server, and then upgrade in place. We'd prefer > to do it with minimal downtime, so setting up parallel servers and > migrating the data across seems like the least disruptive. > > So I have tried the parallel server approach. No issues installing Solr and > JVM. From reading around the web, starting with > > https://solr.apache.org/guide/solr/latest/deployment-guide/backup-restore.html > And several other pages, which all seem to repeat the advice, the backup > restore process is simple. This is a standalone install so I should use > > > http://localhost:8983/solr/collname/replication?command=backup&location=/snapshots > > to backup, which works fine: the backup appears in /snapshots/ as > snapshot.2023xxx. All good. > > I take the backup, and copy it to the new server, and put it in the same > location, /snapshots/2023xxx > Then apparenly I only need to run > > > http://localhost:8983/solr/collname/replication?command=restore&name=snapshot.2023xxx&location=/snapshots > > But this doesn't work. I've tried a lot of different variations, combining > snapshot name and location, with or without the collection name, but all I > get is the zen like error > Searching for Solr? > You must type the correct path. > Solr will respond. > > So that's where I'm at. I'm wondering if I've missed a step that's obvious > to Solr veterans, but which is missing from the backup/restore page. > Do I need to create the core first, for example. Or create the core and > copy config files over to the new server. > -- Vincenzo D'Amore
Re: Backup from old server and Restore to new server
I highly recommend doing a full re-index from scratch to the new cluster. Then, test it in your dev/test environment. And finally, switch your search app to the new server. Jan > 25. sep. 2023 kl. 09:40 skrev Jim Morgan : > > Hi, > I inherited a solr installation from a previous sysadmin, and we'd like to > see if we can move it from the current server to a new one. I don't have > much knowledge of solr, but I know my way around Linux. Apologies in > advance for any dumb questions, and please feel free to educate me. > > Old install is Solr 8.5 running on Ubuntu 18. We're thinking that while we > are switching servers, we might as well try to upgrade to Ubuntu 22 and > Solr 9.3 (with corresponding JVM increase to 11). > First question, is that a crazy idea? Or should we be moving to the same > version of Solr on the new server, and then upgrade in place. We'd prefer > to do it with minimal downtime, so setting up parallel servers and > migrating the data across seems like the least disruptive. > > So I have tried the parallel server approach. No issues installing Solr and > JVM. From reading around the web, starting with > https://solr.apache.org/guide/solr/latest/deployment-guide/backup-restore.html > And several other pages, which all seem to repeat the advice, the backup > restore process is simple. This is a standalone install so I should use > > http://localhost:8983/solr/collname/replication?command=backup&location=/snapshots > > to backup, which works fine: the backup appears in /snapshots/ as > snapshot.2023xxx. All good. > > I take the backup, and copy it to the new server, and put it in the same > location, /snapshots/2023xxx > Then apparenly I only need to run > > http://localhost:8983/solr/collname/replication?command=restore&name=snapshot.2023xxx&location=/snapshots > > But this doesn't work. I've tried a lot of different variations, combining > snapshot name and location, with or without the collection name, but all I > get is the zen like error >Searching for Solr? >You must type the correct path. >Solr will respond. > > So that's where I'm at. I'm wondering if I've missed a step that's obvious > to Solr veterans, but which is missing from the backup/restore page. > Do I need to create the core first, for example. Or create the core and > copy config files over to the new server.
Re: Backup from old server and Restore to new server
Quick response! There are two cores. But I think only one of them is actively in use. I think the other one doesn't need to be migrated. But will know for sure when I manage it and can test the app with the new server! Core in use seems to be about 12Gb. On Mon, Sep 25, 2023 at 3:48 PM Vincenzo D'Amore wrote: > Hi Jim, > > how many cores (indexes) do you have in this Solr server? > > On Mon, Sep 25, 2023 at 9:41 AM Jim Morgan > wrote: > > > Hi, > > I inherited a solr installation from a previous sysadmin, and we'd like > to > > see if we can move it from the current server to a new one. I don't have > > much knowledge of solr, but I know my way around Linux. Apologies in > > advance for any dumb questions, and please feel free to educate me. > > > > Old install is Solr 8.5 running on Ubuntu 18. We're thinking that while > we > > are switching servers, we might as well try to upgrade to Ubuntu 22 and > > Solr 9.3 (with corresponding JVM increase to 11). > > First question, is that a crazy idea? Or should we be moving to the same > > version of Solr on the new server, and then upgrade in place. We'd prefer > > to do it with minimal downtime, so setting up parallel servers and > > migrating the data across seems like the least disruptive. > > > > So I have tried the parallel server approach. No issues installing Solr > and > > JVM. From reading around the web, starting with > > > > > https://solr.apache.org/guide/solr/latest/deployment-guide/backup-restore.html > > And several other pages, which all seem to repeat the advice, the backup > > restore process is simple. This is a standalone install so I should use > > > > > > > http://localhost:8983/solr/collname/replication?command=backup&location=/snapshots > > > > to backup, which works fine: the backup appears in /snapshots/ as > > snapshot.2023xxx. All good. > > > > I take the backup, and copy it to the new server, and put it in the same > > location, /snapshots/2023xxx > > Then apparenly I only need to run > > > > > > > http://localhost:8983/solr/collname/replication?command=restore&name=snapshot.2023xxx&location=/snapshots > > > > But this doesn't work. I've tried a lot of different variations, > combining > > snapshot name and location, with or without the collection name, but all > I > > get is the zen like error > > Searching for Solr? > > You must type the correct path. > > Solr will respond. > > > > So that's where I'm at. I'm wondering if I've missed a step that's > obvious > > to Solr veterans, but which is missing from the backup/restore page. > > Do I need to create the core first, for example. Or create the core and > > copy config files over to the new server. > > > > > -- > Vincenzo D'Amore >
Re: Backup from old server and Restore to new server
Thanks Jan. I've read that around the internet too. Are you saying that I should not use the backup/restore method, but INSTEAD, I should be trying to create the core on the new server and get the data into it by running a re-index, rather than copying it across? So if that's what you're saying, what would be the steps I'd need to take? I'm guessing create core, copy config files, On Mon, Sep 25, 2023 at 3:55 PM Jan Høydahl wrote: > I highly recommend doing a full re-index from scratch to the new cluster. > Then, test it in your dev/test environment. > And finally, switch your search app to the new server. > > Jan > > > 25. sep. 2023 kl. 09:40 skrev Jim Morgan >: > > > > Hi, > > I inherited a solr installation from a previous sysadmin, and we'd like > to > > see if we can move it from the current server to a new one. I don't have > > much knowledge of solr, but I know my way around Linux. Apologies in > > advance for any dumb questions, and please feel free to educate me. > > > > Old install is Solr 8.5 running on Ubuntu 18. We're thinking that while > we > > are switching servers, we might as well try to upgrade to Ubuntu 22 and > > Solr 9.3 (with corresponding JVM increase to 11). > > First question, is that a crazy idea? Or should we be moving to the same > > version of Solr on the new server, and then upgrade in place. We'd prefer > > to do it with minimal downtime, so setting up parallel servers and > > migrating the data across seems like the least disruptive. > > > > So I have tried the parallel server approach. No issues installing Solr > and > > JVM. From reading around the web, starting with > > > https://solr.apache.org/guide/solr/latest/deployment-guide/backup-restore.html > > And several other pages, which all seem to repeat the advice, the backup > > restore process is simple. This is a standalone install so I should use > > > > > http://localhost:8983/solr/collname/replication?command=backup&location=/snapshots > > > > to backup, which works fine: the backup appears in /snapshots/ as > > snapshot.2023xxx. All good. > > > > I take the backup, and copy it to the new server, and put it in the same > > location, /snapshots/2023xxx > > Then apparenly I only need to run > > > > > http://localhost:8983/solr/collname/replication?command=restore&name=snapshot.2023xxx&location=/snapshots > > > > But this doesn't work. I've tried a lot of different variations, > combining > > snapshot name and location, with or without the collection name, but all > I > > get is the zen like error > >Searching for Solr? > >You must type the correct path. > >Solr will respond. > > > > So that's where I'm at. I'm wondering if I've missed a step that's > obvious > > to Solr veterans, but which is missing from the backup/restore page. > > Do I need to create the core first, for example. Or create the core and > > copy config files over to the new server. > >
Re: Backup from old server and Restore to new server
Hi, Absolutely. And if your company lacks documentation and routines for installing and configuring this piece of software, it is an excellet opportuninty to fill that gap by doing it step by step. While you're at it, you can also read the upgrade notes https://solr.apache.org/guide/solr/latest/upgrade-notes/major-changes-in-solr-9.html and see if you are affected by any of the breaking changes. If you're running legacy setup (not Cloud) then you simply place your folder in the new Solr's SOLR_HOME folder and you don't need any API calls to create the core. Jan > 25. sep. 2023 kl. 10:12 skrev Jim Morgan : > > Thanks Jan. I've read that around the internet too. Are you saying that I > should not use the backup/restore method, but INSTEAD, I should be trying > to create the core on the new server and get the data into it by running a > re-index, rather than copying it across? > So if that's what you're saying, what would be the steps I'd need to take? > I'm guessing create core, copy config files, > > On Mon, Sep 25, 2023 at 3:55 PM Jan Høydahl wrote: > >> I highly recommend doing a full re-index from scratch to the new cluster. >> Then, test it in your dev/test environment. >> And finally, switch your search app to the new server. >> >> Jan >> >>> 25. sep. 2023 kl. 09:40 skrev Jim Morgan >> : >>> >>> Hi, >>> I inherited a solr installation from a previous sysadmin, and we'd like >> to >>> see if we can move it from the current server to a new one. I don't have >>> much knowledge of solr, but I know my way around Linux. Apologies in >>> advance for any dumb questions, and please feel free to educate me. >>> >>> Old install is Solr 8.5 running on Ubuntu 18. We're thinking that while >> we >>> are switching servers, we might as well try to upgrade to Ubuntu 22 and >>> Solr 9.3 (with corresponding JVM increase to 11). >>> First question, is that a crazy idea? Or should we be moving to the same >>> version of Solr on the new server, and then upgrade in place. We'd prefer >>> to do it with minimal downtime, so setting up parallel servers and >>> migrating the data across seems like the least disruptive. >>> >>> So I have tried the parallel server approach. No issues installing Solr >> and >>> JVM. From reading around the web, starting with >>> >> https://solr.apache.org/guide/solr/latest/deployment-guide/backup-restore.html >>> And several other pages, which all seem to repeat the advice, the backup >>> restore process is simple. This is a standalone install so I should use >>> >>> >> http://localhost:8983/solr/collname/replication?command=backup&location=/snapshots >>> >>> to backup, which works fine: the backup appears in /snapshots/ as >>> snapshot.2023xxx. All good. >>> >>> I take the backup, and copy it to the new server, and put it in the same >>> location, /snapshots/2023xxx >>> Then apparenly I only need to run >>> >>> >> http://localhost:8983/solr/collname/replication?command=restore&name=snapshot.2023xxx&location=/snapshots >>> >>> But this doesn't work. I've tried a lot of different variations, >> combining >>> snapshot name and location, with or without the collection name, but all >> I >>> get is the zen like error >>> Searching for Solr? >>> You must type the correct path. >>> Solr will respond. >>> >>> So that's where I'm at. I'm wondering if I've missed a step that's >> obvious >>> to Solr veterans, but which is missing from the backup/restore page. >>> Do I need to create the core first, for example. Or create the core and >>> copy config files over to the new server. >> >>
Re: Backup from old server and Restore to new server
That was my feeling as well. I could have just cloned the server, but doing it this way, I can document the procedure (and maybe even automate it in the future), so it never becomes an obstacle again. And also we get the latest version of software on the latest OS. So one further question, if you don't mind. If I copy the config directory over to the new server, will that trigger the re-indexing automatically? And (oh, no, two questions) when the re-indexing starts, will there be a flood of traffic on whatever its indexing from? I'm keen to avoid a sudden surge which takes down our webservers. On Mon, Sep 25, 2023 at 4:39 PM Jan Høydahl wrote: > Hi, > > Absolutely. And if your company lacks documentation and routines for > installing and configuring > this piece of software, it is an excellet opportuninty to fill that gap by > doing it step by step. > While you're at it, you can also read the upgrade notes > https://solr.apache.org/guide/solr/latest/upgrade-notes/major-changes-in-solr-9.html > and see if you are affected by any of the breaking changes. > > If you're running legacy setup (not Cloud) then you simply place your > folder in the new Solr's SOLR_HOME folder and you don't need > any API calls to create the core. > > Jan > > > 25. sep. 2023 kl. 10:12 skrev Jim Morgan >: > > > > Thanks Jan. I've read that around the internet too. Are you saying that I > > should not use the backup/restore method, but INSTEAD, I should be trying > > to create the core on the new server and get the data into it by running > a > > re-index, rather than copying it across? > > So if that's what you're saying, what would be the steps I'd need to > take? > > I'm guessing create core, copy config files, > > > > On Mon, Sep 25, 2023 at 3:55 PM Jan Høydahl > wrote: > > > >> I highly recommend doing a full re-index from scratch to the new > cluster. > >> Then, test it in your dev/test environment. > >> And finally, switch your search app to the new server. > >> > >> Jan > >> > >>> 25. sep. 2023 kl. 09:40 skrev Jim Morgan >>> : > >>> > >>> Hi, > >>> I inherited a solr installation from a previous sysadmin, and we'd like > >> to > >>> see if we can move it from the current server to a new one. I don't > have > >>> much knowledge of solr, but I know my way around Linux. Apologies in > >>> advance for any dumb questions, and please feel free to educate me. > >>> > >>> Old install is Solr 8.5 running on Ubuntu 18. We're thinking that while > >> we > >>> are switching servers, we might as well try to upgrade to Ubuntu 22 and > >>> Solr 9.3 (with corresponding JVM increase to 11). > >>> First question, is that a crazy idea? Or should we be moving to the > same > >>> version of Solr on the new server, and then upgrade in place. We'd > prefer > >>> to do it with minimal downtime, so setting up parallel servers and > >>> migrating the data across seems like the least disruptive. > >>> > >>> So I have tried the parallel server approach. No issues installing Solr > >> and > >>> JVM. From reading around the web, starting with > >>> > >> > https://solr.apache.org/guide/solr/latest/deployment-guide/backup-restore.html > >>> And several other pages, which all seem to repeat the advice, the > backup > >>> restore process is simple. This is a standalone install so I should use > >>> > >>> > >> > http://localhost:8983/solr/collname/replication?command=backup&location=/snapshots > >>> > >>> to backup, which works fine: the backup appears in /snapshots/ as > >>> snapshot.2023xxx. All good. > >>> > >>> I take the backup, and copy it to the new server, and put it in the > same > >>> location, /snapshots/2023xxx > >>> Then apparenly I only need to run > >>> > >>> > >> > http://localhost:8983/solr/collname/replication?command=restore&name=snapshot.2023xxx&location=/snapshots > >>> > >>> But this doesn't work. I've tried a lot of different variations, > >> combining > >>> snapshot name and location, with or without the collection name, but > all > >> I > >>> get is the zen like error > >>> Searching for Solr? > >>> You must type the correct path. > >>> Solr will respond. > >>> > >>> So that's where I'm at. I'm wondering if I've missed a step that's > >> obvious > >>> to Solr veterans, but which is missing from the backup/restore page. > >>> Do I need to create the core first, for example. Or create the core and > >>> copy config files over to the new server. > >> > >> > >
Re: Backup from old server and Restore to new server
:) looking at solr.log you can see the requests and, even if you cannot be completely sure, which core is currently receiving requests. Before moving to a new solr version I should understand how is your server average load, how many requests per second your server is handling. Honestly I'm not sure that moving to the latest solr version is a good idea, what do you think to gain moving immediately to the latest version? Maybe better move to the latest solr 8 version On Mon, Sep 25, 2023 at 10:10 AM Jim Morgan wrote: > Quick response! There are two cores. But I think only one of them is > actively in use. I think the other one doesn't need to be migrated. But > will know for sure when I manage it and can test the app with the new > server! > Core in use seems to be about 12Gb. > > On Mon, Sep 25, 2023 at 3:48 PM Vincenzo D'Amore > wrote: > > > Hi Jim, > > > > how many cores (indexes) do you have in this Solr server? > > > > On Mon, Sep 25, 2023 at 9:41 AM Jim Morgan > > > wrote: > > > > > Hi, > > > I inherited a solr installation from a previous sysadmin, and we'd like > > to > > > see if we can move it from the current server to a new one. I don't > have > > > much knowledge of solr, but I know my way around Linux. Apologies in > > > advance for any dumb questions, and please feel free to educate me. > > > > > > Old install is Solr 8.5 running on Ubuntu 18. We're thinking that while > > we > > > are switching servers, we might as well try to upgrade to Ubuntu 22 and > > > Solr 9.3 (with corresponding JVM increase to 11). > > > First question, is that a crazy idea? Or should we be moving to the > same > > > version of Solr on the new server, and then upgrade in place. We'd > prefer > > > to do it with minimal downtime, so setting up parallel servers and > > > migrating the data across seems like the least disruptive. > > > > > > So I have tried the parallel server approach. No issues installing Solr > > and > > > JVM. From reading around the web, starting with > > > > > > > > > https://solr.apache.org/guide/solr/latest/deployment-guide/backup-restore.html > > > And several other pages, which all seem to repeat the advice, the > backup > > > restore process is simple. This is a standalone install so I should use > > > > > > > > > > > > http://localhost:8983/solr/collname/replication?command=backup&location=/snapshots > > > > > > to backup, which works fine: the backup appears in /snapshots/ as > > > snapshot.2023xxx. All good. > > > > > > I take the backup, and copy it to the new server, and put it in the > same > > > location, /snapshots/2023xxx > > > Then apparenly I only need to run > > > > > > > > > > > > http://localhost:8983/solr/collname/replication?command=restore&name=snapshot.2023xxx&location=/snapshots > > > > > > But this doesn't work. I've tried a lot of different variations, > > combining > > > snapshot name and location, with or without the collection name, but > all > > I > > > get is the zen like error > > > Searching for Solr? > > > You must type the correct path. > > > Solr will respond. > > > > > > So that's where I'm at. I'm wondering if I've missed a step that's > > obvious > > > to Solr veterans, but which is missing from the backup/restore page. > > > Do I need to create the core first, for example. Or create the core and > > > copy config files over to the new server. > > > > > > > > > -- > > Vincenzo D'Amore > > > -- Vincenzo D'Amore
Re: Backup from old server and Restore to new server
Thanks. Good tip for the logs. Although they didn't prove very useful in determining what the problem with the import was: absolutely nothing was logged! Server load: I have that under control in terms of RAM and hard disk space and that won't be an issue. Is there a command to find out how many requests are being processed? Or any useful stats/status summary. Or do you mean use grep and wc on logs ... also can do, but as I'm learning about Solr I'm keen to learn the tricks. Moving to the latest version was basically me trying to get as much done in one disruption as possible. We have to do the migration, so I figured we should try to move to the latest solr version as well. If we can't do it, then I'll take the latest 8.x version, sure, but I thought I'd try. On Mon, Sep 25, 2023 at 4:59 PM Vincenzo D'Amore wrote: > :) > looking at solr.log you can see the requests and, even if you cannot be > completely sure, which core is currently receiving requests. > Before moving to a new solr version I should understand how is your server > average load, how many requests per second your server is handling. > Honestly I'm not sure that moving to the latest solr version is a good > idea, what do you think to gain moving immediately to the latest version? > Maybe better move to the latest solr 8 version > > On Mon, Sep 25, 2023 at 10:10 AM Jim Morgan > > wrote: > > > Quick response! There are two cores. But I think only one of them is > > actively in use. I think the other one doesn't need to be migrated. But > > will know for sure when I manage it and can test the app with the new > > server! > > Core in use seems to be about 12Gb. > > > > On Mon, Sep 25, 2023 at 3:48 PM Vincenzo D'Amore > > wrote: > > > > > Hi Jim, > > > > > > how many cores (indexes) do you have in this Solr server? > > > > > > On Mon, Sep 25, 2023 at 9:41 AM Jim Morgan > > > > > > wrote: > > > > > > > Hi, > > > > I inherited a solr installation from a previous sysadmin, and we'd > like > > > to > > > > see if we can move it from the current server to a new one. I don't > > have > > > > much knowledge of solr, but I know my way around Linux. Apologies in > > > > advance for any dumb questions, and please feel free to educate me. > > > > > > > > Old install is Solr 8.5 running on Ubuntu 18. We're thinking that > while > > > we > > > > are switching servers, we might as well try to upgrade to Ubuntu 22 > and > > > > Solr 9.3 (with corresponding JVM increase to 11). > > > > First question, is that a crazy idea? Or should we be moving to the > > same > > > > version of Solr on the new server, and then upgrade in place. We'd > > prefer > > > > to do it with minimal downtime, so setting up parallel servers and > > > > migrating the data across seems like the least disruptive. > > > > > > > > So I have tried the parallel server approach. No issues installing > Solr > > > and > > > > JVM. From reading around the web, starting with > > > > > > > > > > > > > > https://solr.apache.org/guide/solr/latest/deployment-guide/backup-restore.html > > > > And several other pages, which all seem to repeat the advice, the > > backup > > > > restore process is simple. This is a standalone install so I should > use > > > > > > > > > > > > > > > > > > http://localhost:8983/solr/collname/replication?command=backup&location=/snapshots > > > > > > > > to backup, which works fine: the backup appears in /snapshots/ as > > > > snapshot.2023xxx. All good. > > > > > > > > I take the backup, and copy it to the new server, and put it in the > > same > > > > location, /snapshots/2023xxx > > > > Then apparenly I only need to run > > > > > > > > > > > > > > > > > > http://localhost:8983/solr/collname/replication?command=restore&name=snapshot.2023xxx&location=/snapshots > > > > > > > > But this doesn't work. I've tried a lot of different variations, > > > combining > > > > snapshot name and location, with or without the collection name, but > > all > > > I > > > > get is the zen like error > > > > Searching for Solr? > > > > You must type the correct path. > > > > Solr will respond. > > > > > > > > So that's where I'm at. I'm wondering if I've missed a step that's > > > obvious > > > > to Solr veterans, but which is missing from the backup/restore page. > > > > Do I need to create the core first, for example. Or create the core > and > > > > copy config files over to the new server. > > > > > > > > > > > > > -- > > > Vincenzo D'Amore > > > > > > > > -- > Vincenzo D'Amore >
Re: Backup from old server and Restore to new server
In the past, when upgrading to a major version, well... sometimes it happened, you can even have a loss of performance. So before moving I would be sure if I can and what I gain from it. For example in the Solr 9.x the Data Import Handler has been moved out to an external component, still open source but it is external. On Mon, Sep 25, 2023 at 11:13 AM Jim Morgan wrote: > Thanks. Good tip for the logs. Although they didn't prove very useful in > determining what the problem with the import was: absolutely nothing was > logged! > > Server load: I have that under control in terms of RAM and hard disk space > and that won't be an issue. Is there a command to find out how many > requests are being processed? Or any useful stats/status summary. Or do you > mean use grep and wc on logs ... also can do, but as I'm learning about > Solr I'm keen to learn the tricks. > > Moving to the latest version was basically me trying to get as much done in > one disruption as possible. We have to do the migration, so I figured we > should try to move to the latest solr version as well. If we can't do it, > then I'll take the latest 8.x version, sure, but I thought I'd try. > > On Mon, Sep 25, 2023 at 4:59 PM Vincenzo D'Amore > wrote: > > > :) > > looking at solr.log you can see the requests and, even if you cannot be > > completely sure, which core is currently receiving requests. > > Before moving to a new solr version I should understand how is your > server > > average load, how many requests per second your server is handling. > > Honestly I'm not sure that moving to the latest solr version is a good > > idea, what do you think to gain moving immediately to the latest version? > > Maybe better move to the latest solr 8 version > > > > On Mon, Sep 25, 2023 at 10:10 AM Jim Morgan > > > > > wrote: > > > > > Quick response! There are two cores. But I think only one of them is > > > actively in use. I think the other one doesn't need to be migrated. But > > > will know for sure when I manage it and can test the app with the new > > > server! > > > Core in use seems to be about 12Gb. > > > > > > On Mon, Sep 25, 2023 at 3:48 PM Vincenzo D'Amore > > > wrote: > > > > > > > Hi Jim, > > > > > > > > how many cores (indexes) do you have in this Solr server? > > > > > > > > On Mon, Sep 25, 2023 at 9:41 AM Jim Morgan > > > > > > > > > wrote: > > > > > > > > > Hi, > > > > > I inherited a solr installation from a previous sysadmin, and we'd > > like > > > > to > > > > > see if we can move it from the current server to a new one. I don't > > > have > > > > > much knowledge of solr, but I know my way around Linux. Apologies > in > > > > > advance for any dumb questions, and please feel free to educate me. > > > > > > > > > > Old install is Solr 8.5 running on Ubuntu 18. We're thinking that > > while > > > > we > > > > > are switching servers, we might as well try to upgrade to Ubuntu 22 > > and > > > > > Solr 9.3 (with corresponding JVM increase to 11). > > > > > First question, is that a crazy idea? Or should we be moving to the > > > same > > > > > version of Solr on the new server, and then upgrade in place. We'd > > > prefer > > > > > to do it with minimal downtime, so setting up parallel servers and > > > > > migrating the data across seems like the least disruptive. > > > > > > > > > > So I have tried the parallel server approach. No issues installing > > Solr > > > > and > > > > > JVM. From reading around the web, starting with > > > > > > > > > > > > > > > > > > > > https://solr.apache.org/guide/solr/latest/deployment-guide/backup-restore.html > > > > > And several other pages, which all seem to repeat the advice, the > > > backup > > > > > restore process is simple. This is a standalone install so I should > > use > > > > > > > > > > > > > > > > > > > > > > > > > http://localhost:8983/solr/collname/replication?command=backup&location=/snapshots > > > > > > > > > > to backup, which works fine: the backup appears in /snapshots/ as > > > > > snapshot.2023xxx. All good. > > > > > > > > > > I take the backup, and copy it to the new server, and put it in the > > > same > > > > > location, /snapshots/2023xxx > > > > > Then apparenly I only need to run > > > > > > > > > > > > > > > > > > > > > > > > > http://localhost:8983/solr/collname/replication?command=restore&name=snapshot.2023xxx&location=/snapshots > > > > > > > > > > But this doesn't work. I've tried a lot of different variations, > > > > combining > > > > > snapshot name and location, with or without the collection name, > but > > > all > > > > I > > > > > get is the zen like error > > > > > Searching for Solr? > > > > > You must type the correct path. > > > > > Solr will respond. > > > > > > > > > > So that's where I'm at. I'm wondering if I've missed a step that's > > > > obvious > > > > > to Solr veterans, but which is missing from the backup/restore > page. > > > > > Do I need to create the core first, for example. Or create the core > > and > > > > > copy c
Re: Backup from old server and Restore to new server
This is hard to tell whithout knowing what data you index to Solr and how. If you use Data Import Handler, that is no longer part of 9.0 so you definitely need to plan for how to proceed. You'll find tons of email threads on it in the archives. If you have an external indexing process, you simply trigger that process.. Jan > 25. sep. 2023 kl. 10:54 skrev Jim Morgan : > > That was my feeling as well. I could have just cloned the server, but doing > it this way, I can document the procedure (and maybe even automate it in > the future), so it never becomes an obstacle again. And also we get the > latest version of software on the latest OS. > > So one further question, if you don't mind. If I copy the config directory > over to the new server, will that trigger the re-indexing automatically? > And (oh, no, two questions) when the re-indexing starts, will there be a > flood of traffic on whatever its indexing from? I'm keen to avoid a sudden > surge which takes down our webservers. > > On Mon, Sep 25, 2023 at 4:39 PM Jan Høydahl wrote: > >> Hi, >> >> Absolutely. And if your company lacks documentation and routines for >> installing and configuring >> this piece of software, it is an excellet opportuninty to fill that gap by >> doing it step by step. >> While you're at it, you can also read the upgrade notes >> https://solr.apache.org/guide/solr/latest/upgrade-notes/major-changes-in-solr-9.html >> and see if you are affected by any of the breaking changes. >> >> If you're running legacy setup (not Cloud) then you simply place your >> folder in the new Solr's SOLR_HOME folder and you don't need >> any API calls to create the core. >> >> Jan >> >>> 25. sep. 2023 kl. 10:12 skrev Jim Morgan >> : >>> >>> Thanks Jan. I've read that around the internet too. Are you saying that I >>> should not use the backup/restore method, but INSTEAD, I should be trying >>> to create the core on the new server and get the data into it by running >> a >>> re-index, rather than copying it across? >>> So if that's what you're saying, what would be the steps I'd need to >> take? >>> I'm guessing create core, copy config files, >>> >>> On Mon, Sep 25, 2023 at 3:55 PM Jan Høydahl >> wrote: >>> I highly recommend doing a full re-index from scratch to the new >> cluster. Then, test it in your dev/test environment. And finally, switch your search app to the new server. Jan > 25. sep. 2023 kl. 09:40 skrev Jim Morgan : > > Hi, > I inherited a solr installation from a previous sysadmin, and we'd like to > see if we can move it from the current server to a new one. I don't >> have > much knowledge of solr, but I know my way around Linux. Apologies in > advance for any dumb questions, and please feel free to educate me. > > Old install is Solr 8.5 running on Ubuntu 18. We're thinking that while we > are switching servers, we might as well try to upgrade to Ubuntu 22 and > Solr 9.3 (with corresponding JVM increase to 11). > First question, is that a crazy idea? Or should we be moving to the >> same > version of Solr on the new server, and then upgrade in place. We'd >> prefer > to do it with minimal downtime, so setting up parallel servers and > migrating the data across seems like the least disruptive. > > So I have tried the parallel server approach. No issues installing Solr and > JVM. From reading around the web, starting with > >> https://solr.apache.org/guide/solr/latest/deployment-guide/backup-restore.html > And several other pages, which all seem to repeat the advice, the >> backup > restore process is simple. This is a standalone install so I should use > > >> http://localhost:8983/solr/collname/replication?command=backup&location=/snapshots > > to backup, which works fine: the backup appears in /snapshots/ as > snapshot.2023xxx. All good. > > I take the backup, and copy it to the new server, and put it in the >> same > location, /snapshots/2023xxx > Then apparenly I only need to run > > >> http://localhost:8983/solr/collname/replication?command=restore&name=snapshot.2023xxx&location=/snapshots > > But this doesn't work. I've tried a lot of different variations, combining > snapshot name and location, with or without the collection name, but >> all I > get is the zen like error > Searching for Solr? > You must type the correct path. > Solr will respond. > > So that's where I'm at. I'm wondering if I've missed a step that's obvious > to Solr veterans, but which is missing from the backup/restore page. > Do I need to create the core first, for example. Or create the core and > copy config files over to the new server. >> >>
A general question about update ordering
I have a general question about update ordering. Situation: Two indexing requests for the same uniqueKey value sent. One update "creates" the document, the other is an "update" with more fields populated. The "update" is sent by a different system than the "create" ... and it turns out that they are sent in the wrong order, with the update coming one millisecond BEFORE the create. Solr 4.7 handles this as the customer wants, but Solr 9.1 doesn't. I suspect that 4.7 is ignoring the order in which the updates were received, so the indexing operation that takes longer is actually applied after the one that happens faster ... while 9.1 is applying the updates in the order they were received. Is my idea correct? Is Lucene 9 or Solr 9 smarter than 4 about order of operations? Thanks, Shawn
Re: A general question about update ordering
On 9/25/23 08:24, Shawn Heisey wrote: ... Solr 4.7 handles this as the customer wants, but Solr 9.1 doesn't. This sounds like a race condition, are you sure it's Solr doing the ordering and not e.g. the network stack? What I found is that when the "add" request comes first, 8.11.2 (at least) will create a new -- useless -- document for it. I wish it failed instead. Dima
Re: A general question about update ordering
Updates and indexing used to be single-threaded. A while ago (maybe in Solr 6.x?), a bunch of locking was removed so that indexing could use multiple CPUs. That is probably what you are seeing. I don’t think Solr has ever guaranteed ordering for updates, even if it happened to work. Maybe use a real-time get before the update to check whether the document is really there? That should be pretty fast. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Sep 25, 2023, at 9:53 AM, Dmitri Maziuk wrote: > > On 9/25/23 08:24, Shawn Heisey wrote: > ... >> Solr 4.7 handles this as the customer wants, but Solr 9.1 doesn't. > > This sounds like a race condition, are you sure it's Solr doing the ordering > and not e.g. the network stack? > > What I found is that when the "add" request comes first, 8.11.2 (at least) > will create a new -- useless -- document for it. I wish it failed instead. > > Dima >
Re: A general question about update ordering
On 9/25/23 08:53, Dmitri Maziuk wrote: On 9/25/23 08:24, Shawn Heisey wrote: ... Solr 4.7 handles this as the customer wants, but Solr 9.1 doesn't. This sounds like a race condition, are you sure it's Solr doing the ordering and not e.g. the network stack? I have seen the requests in the logs on the two systems that send the indexing requests and the matching entries in solr.log on 9.x. The "update" request is sent one millisecond before the "create" request. The "update" request has quite a bit more field data than the "create" request, so I am sure it takes longer. When these requests are sent to a 4.7 cluster, the end result is that the updated record is kept. I believe this is happening due to how long the two updates take, and the order of receipt is ignored. When exactly the same thing is sent to a 9.1.1 cluster, the "create" record is kept instead of the "update" record. The customer has said that they think this is a bug in 9.1.1 ... but I think Solr 9 is actually doing it right, honoring the order the updates were received, and Solr 4 just happened to meet their expectations due to what I would call a bug. What I found is that when the "add" request comes first, 8.11.2 (at least) will create a new -- useless -- document for it. I wish it failed instead. You might be able to achieve the behavior you want with the optimistic concurrency feature: https://solr.apache.org/guide/solr/latest/indexing-guide/partial-document-updates.html#optimistic-concurrency Maybe this feature can also help in my customer's setup. Thanks, Shawn
Re: A general question about update ordering
On 9/25/23 10:49, Shawn Heisey wrote: On 9/25/23 08:53, Dmitri Maziuk wrote: ... What I found is that when the "add" request comes first, 8.11.2 (at least) will create a new -- useless -- document for it. I wish it failed instead. You might be able to achieve the behavior you want with the optimistic concurrency feature: Not in my case, but thanks for the tip: one more thing to file for future reference. Anyway, what Walter said: you're racing on threads. (From your description, relying on the 4.x behaviour was a bad idea in the first place: what if "create" request e.g. hit a bad sector on a non-TLER drive and was stuck on i/o for 100x longer?) Dima
Re: A general question about update ordering
On 9/25/23 09:44, Walter Underwood wrote: Updates and indexing used to be single-threaded. A while ago (maybe in Solr 6.x?), a bunch of locking was removed so that indexing could use multiple CPUs. That is probably what you are seeing. I don’t think Solr has ever guaranteed ordering for updates, even if it happened to work. Maybe use a real-time get before the update to check whether the document is really there? That should be pretty fast. Because the two requests are sent by different services and coordinating those requests woud likely be difficult, I think the simplest change that would fix the problem is to use optimistic concurrency, setting the field to 0 on the "create" request so the request fails if the document already exists. And of course the resulting error would need to be explicitly detected so it can be ignored, while still correctly handling other errors. I have not seen the SolrJ code that actually makes the indexing requests. In this, I am assuming, hopefully correctly, that optimistic concurrency checks uncommitted docs like the /get handler, as it is very unlikely that the earlier update has actually been committed when the second update comes in. Thanks, Shawn
Re: A general question about update ordering
On 9/25/23 10:24, Dmitri Maziuk wrote: (From your description, relying on the 4.x behaviour was a bad idea in the first place: what if "create" request e.g. hit a bad sector on a non-TLER drive and was stuck on i/o for 100x longer?) I agree that the way they are operating now is completely wrong, and I am surprised that it even worked. They got VERY lucky. Thanks, Shawn
Re: A general question about update ordering
On 9/25/23 11:35, Shawn Heisey wrote: I agree that the way they are operating now is completely wrong, and I am surprised that it even worked. They got VERY lucky. http://www.catb.org/jargon/html/S/schroedinbug.html cheers, Dima
