+------------------------------------------------------------------------------ | On 2011-01-13 07:36:12, Jaques Cochet wrote: | | I'm working on a mail system design for an ISP that includes hosting | of multiple virtual domains managed by this ISP (300.000 mailbox). HA | and performance are both important concerns for the client, so I have | at least 2 of every server (webmail, pop3, imap, relay and smtp | (postfix)) for which i'm using either L4 or MX record load | balancing/HA. | I hate the idea of distributing mailboxes among servers and I'm trying | to go for a single mailstore that is accessible by POP3/IMAP servers | and delivery SMTP servers and I'm planning to use a SAN for this. The | basic idea is to share the mailstore between SMTP servers (clustered | storage using GFS maybe) and make the same mailstore available to | POP/IMAP server using NFS. Am I on the right track here?
There's no reason to share the mailstore filesystem to the front-end SMTP boxes, and quite a few good reasons not to. What kind of locking issues you might run into with GFS, I couldn't say. Depending on the IMAP/POP3 daemon you're using, NFS may not be a good choice to export the mailstore from the fileserver to the IMAP system. iSCSI may be a better choice. Depends on what "SAN" means in this context. We just run our mailstore services on the fileservers. Our setup is simple and monolithic, but has worked pretty well for the last decade or so (platform, filesystems, and hardware has changed drastically, but the base architecture remains): The mailstore runs the IMAP daemon, and an MTA. In front of the IMAP daemon we run nginx in IMAP proxy mode to mangle authentication and determine the backend IMAP server the client talks to (e.g., in the event of failover). If you do decide to need to shard the mailstore, nginx's IMAP proxy will help you a lot. 300,000 accounts is a lot. At the very least, sharding will help your customer support staff when you need to do maintenance (n% of the angry calls. :) If you're running Linux, you'll probably want to use DRBD to replicate the mailstore to the failover, or if your SAN supports replication, whatever it uses. (We use ZFS for replication.) The front-end MXes (we have 8) filter mail and either discard to a quarantine system or relay ham to the MTA running on the mailstore. The mailstore MTA delivers into, unsurprisingly, the mailstore (via lmtp/deliver). You'll also want to configure fallbacks for the mailstore MTA with different policy settings than the front-end MXes (you don't want mail sitting on the front-end boxes, choking up their queues). Our webmail instances run imapproxy locally and speak IMAP to the mailstore. Again, they are behind nginx (in HTTP mode) for load-balancing/failover. Cheers. -- bdha cyberpunk is dead. long live cyberpunk.
