Hi Anders,
We just completed a migration from 2.0.16 to 2.2.10 and we ended up using a file system copy with NFS. (two servers, one with 1500 users and 54 GB of mail, the other with 5,500 users and 15GB of mail) This is on RHEL 3.0 using Simon's RPM's. (Simon, we can't thank you enough!) Old version was RedHat 7.0 with cyrus from source.
Basic Sequence is as follows:
PreConversion
1. startup NFS and mount old spool partitions on new box and old /var/lib/imap to new box
2. Increase quotas on old server to match new server (and get rid of any mail in the queue to over quota users, flush the queue.) This was OK since we were increasing quotas anyway.
3. Stop mail services on new box
4. Clean up new server (delete mailboxes, annotations, deliver, tls_sessions db's and supporting files. Purge db, proc, quota,sieve user and log directories in /var/lib/imap on the new server. Purge all /var/spool/imap partitions/delete all mail.
5. reconfigure MTA on new box to block receipt of new mail until the process is completed.
Conversion
1. Stop mail services on old box after confirming queue is empty
2. Copy mail spool from old server to new server. ( we ended up doing 3 partitions at a time since this seemed to give the quickest copy. Your mileage will vary depending on the disk config you are using.) our 54GB copy took about 6 hours, but we had pretty fast IO hardware on the receiving side.
3. fix ownership/permissions of all files in new spool. (assuming uid's are not the same)
4. Export mailboxes db to text on old server using ctl_mboxlist, copy file to new server
5. Import mailboxes db on new server from text file with ctl_mboxlist
6. Copy sieve scripts via nfs, change ownership/permissions and run massievec to compile
7. Copy seen and sub files over from /var/lib/imap/user (if you don't use the same hashing scheme, you will have to rehash, we kept it the same)
8. fix ownership and then convert seen to new format (we went from flat to skiplist) via cvt_cyrusdb
9. start cyrus and check logs
10. rerun quota setting script and run recursive quota fix on all quota roots. This seemed easier than bringing the quotas over since we were using a standard size.
11. Shutdown mail again on new server.
12. We also did a IMP to squirrelmail conversion, so we had a step in here to export and convert the address books.
13. unmount nfs mounts
14. shutdown nfs permanently, disconnect old box from network
15. change ip address and DNS name of new box to match old box. ( we could have used a CNAME, but this was cleaner)
16. Reboot new box to be safe
17. Mail should start automatically, then Test, Test, Test
18. Reconfigure MTA to allow mail to be delivered. Test
This process was clean and painless, although time consuming. Mail services for our larger server were down from 5:30PM Saturday night until aproximately 3:30AM Sunday morning. Obviously, depending on what version you are coming from or going to, things may vary as far as hashing, DB format conversions required etc. The documentation was actually extremely helpful in deciding which steps were appropriate for us. Also, since we use external LDAP for authentication, we did not have to move any SASL related data.
The nice thing about this process is that we were able to test the conversion 3 times. (we just nfs copied the mail spool live and lived with the inconsistancies for the tests.) We could then test the new server with a full set of users and mail and throw it to select end users to confirm that things look good. We also had a backout strategy. At any point in the process if things fell apart (until we astarted accepting mail on the new box), we could just shut off the new box and plug the old one back in.
Hope this helps, I can give more detail on specific steps if you have any questions. Our documentation on this is extensive but very specific to us and includes all sorts of special things that are not useful for a general audience. (like our ezmlm mailing list conversion) The project leader on this for us (John Widera) did a great job.
One other thing to note, if you are using RHEL 3, be sure to get the latest kernel. There was a bug in the file system caching code in the update 3 that kills any heavily used system with large amounts of memory (like our server) and lots of small files (like cyrus) This showed up in our conversion testing and we had to backrev the kernel until the new one was released.
Good luck, John Wade
Rob Tanner wrote:
Anders,
I went through a similar problem several months ago, updating from an
ESYS server (commercial clone of a 1.x.x Cyrus server). I had all
kinds of incompatibility problems on some test mailboxes I moved over
manually and also manually built the mailboxes file. Some folders
would be okay and other would be reported as invalid and reconstruct
wouldn't clean it up. I finally bit the bullet and used the perl
program, imapsync, to move the mailboxes over. With imapsync, since it
actually talks to the old and new server via the imap protocol, the
move was error free. The down side is that because it uses the imap
protocol it's bloody slow. I had some 5000 accounts with many
gigabytes of mail store and the total transfer took some three and a
half days to complete. But then all I had to do afterwards was move
the alias that points to the imap server and users were bust reading
mail. The other downside is that the SEEN flag is not a property of
the mailbox and doesn't get transfered, so all mail showed up as
unseen. Since I was already aware of it, we announced it ahead of
time.
-- Rob
--On Friday, December 24, 2004 12:37:55 PM +0100 Anders Norrbring <[EMAIL PROTECTED]> wrote:
Hiya!
I'm about to upgrade my "old" server to a newer platform, both hardware, O/S and Cyrus itself are going to be new versions.
So, how can I ensure that all mail is moved correctly from the old platform to the new? I have around 3000 accounts with lots of e-mails stored in the IMAP folders.
What I need is basically a HOW-TO with step-by-step instructions on how to solve the move.
Merry Christmas to you all!
Anders Norrbring Norrbring Consulting
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
--- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
