Can I make a very weak suggestion to look at that ZLIB compression
extension I think you mentioned in the past?
It would have to be done by proxying in imap-login similar to how SSL
connections are handled. But aren't you using SSL already, and why
not? Using that would give compression for free. Although I haven't
really looked at if it's already automatically enabled or if I or
clients should do something special..
I don't think that SSL in general has compression enabled? Could be
wrong, but I believe it's a option, but badly supported? I'm not an
expert though so I don't know that for sure... I would be interested if
someone had a recipe for enabling compression on TLS?
Also if you use SSL then you can no longer do after the fact
compression. By definition, encryption done well produces an output
which cannot be compressed. So it's even more important to precompress
before encryption
Anyway, just a thought - I'm assuming that the probable
implementation is going to be fairly simple. I would think that zlib
and/or lzo would be good compressors if there is a choice of
implementations? Certainly LZO would be a good choice for faster
100mbit connections
http://www.ietf.org/rfc/rfc4978.txt specifies DEFLATE format that can
be implemented using zlib.
I think this is probably what you referenced before.
My own experience is using a very powerful (cpu hungry) compressor where
it doesn't seem to matter all that much if stuff is base64 encoded or
not. Long shot is that whilst all that reflushing sounds really nice I
think it's just icing compared with just doing blind compression of
everything...
My guess is that with the replication stuff you are going to see a
5x-10x speedup on exchanging long lists of guids to compare folders,
etc. Compression on the actual mailbodies may be much less. In my case
even with an incompressible jpg file which is base64 encoded we still
knock off the expected 1/3 in file size due to the base64 encoding so
it's a nice benefit
(My customers are on dialup connections of just 2,400 baud... ie 20KB
per *minute* http://www.mailasail.com )
For replication I would have thought support of an optional non RFC LZO
compressor would be beneficial on anything under gigabit links..?
Ed W
