hi Mark,
The thing i can tell you about scalability is this:
- you need a clusterable database (For 2.0 you should be able to write
a oracle driver in about 4 hours).
(real clustered databases partition automatically).
- you'll need the LMTP daemon, not dbmail-smtp (increase in speed and
handling is about 1000%).
- Dbmail 2.0 will have a lot of imap optimalization which will increase
performance a lot.
- use a native webmail like webDBmail.
We currently have a db2 licence laying around. We might write a driver
for that and test for speed.
You could also hire us to tune a monsterous dbmail system for you.
We're already doing some very
big dbmail setups. ;)
I once started dbmail with the thought of a multi-million based e-mail
system with super ease of use and
maintenance.
Best regards,
Eelco
On 3-dec-03, at 19:45, Mark Mackay - Orcon wrote:
From a recent post I made for the trackrecord list:
We're currently running dbmail with:
- Dual Xeon 2.4G processors / SCSI Disks
- MySQL/InnoDB
- roughly 90k mailboxes
- 100G of mail data
- alias selection offloaded to smtp servers (we don't use aliases
table we
deliver straight to userid)
- Load average between 0.4 and 2.5 on a dual processor machine,
averaging
at around the .8 mark (so say 50% load)
We have 4 machines handling SMTP/Virus/Spam filtering and then
handing it
to dbmail-smtp for delviery, and 3 machines
doing POP/IMAP -- all of which are light-moderate loaded.
---------------------
I'd be *really keen* to hear from people with larger installations
(even if
privately and off list), who have ideas on how dbmail can scale. I
need to
work out what the next evolution for my cluster will be for the
question
below, and also just for general organic growth, etc.
I've been posed the question by management lately -- how would dbmail
scale if we wanted to offer say 3 Million
mailboxes, rather than just the current limit I see of 100-200k
mailboxes
using my current setup?
Ideas that spring to mind are:
Clustered databases
-------------------
Oracle or some database with multiple write masters and
cluser-wide locking for writes springs to mind. I understand
there is problems with MySQL one-way replication and
directing all writes to a master server, but reads from
multiple slaves - due to write-locking not being supported
thus the slave getting out of sync with the master, etc. I
scanned the list again recently but couldn't find any obvious
posts with solutions for this.
Last minute addition >> I notice SAP DB is now MySQL MaxDB
and is available. Anyone looked at this as a viable backend
for dbmail? Can't for the life of me find the details on
replication support/etc on the site...
Partitioned databases
---------------------
The old customea A-E on database server 1, F-L on database
server 2, etc. This would seem to do the trick, although with
dbmail it may be user_idnr 1-50,000 on DB1, 50,001-100,000 on
DB2 I guess, with some sort of central mailbox -> store index
table. I haven't looked at the code for 2.0 recently, but
from memory 1.x maintained a persistent connection with the
mailstore, thus this could be quite bad if there were say 100
database servers. I'm not sure what the overhead would be for
connection setup/teardown if doing dynamic connections to
different servers.
Any other ideas? (even if not likely to be implemented
officially by dbmail -- may code our own variation if needed).
ADDITIONAL QUESTIONS:
- Has anyone ported dbmail to Oracle or something that has two-way
clustering?
-
Cheers,
Mark.
_______________________________________________
Dbmail mailing list
Dbmail@dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail
_________________________
E.J.A. van Beek
ICT Manager
IC&S
T: +31 30 6355730
F: +31 30 6355731
PGP-key:
www.ic-s.nl/keys/eelco.txt
