> On Oct 17, 2020, at 3:09 AM, Demi M. Obenour <demioben...@gmail.com> wrote:
>
>> The practical limit to the deferred queue size is therefore ~2 days of
>> throughput, and depends heavily on the per-delivery latency. If
>> delivery failures are slow (tarpitting or otherwise slow destinations)
>> the impact is greater.
>
> Can the latency problems be worked around by increasing concurrency?
Yes, so unsurprisingly, Postfix amortises latency via carefully managed
concurrency.
> My understanding is that Postfix might have problems at very high
> concurrency due to using one process per connection, whereas some
> other servers are event-driven and can handle thousands of connections
> without using too much memory.
Postfix reuses processes for multiple deliveries, so process creation
is effectively amortised. SMTP delivery being a rather expensive operation
(DNS lookups, connection setup, TLS handshakes, ...) the fractional (becase
shared across multiple deliveries) cost of process creation is dwarfed by
the actual SMTP transaction costs.
On modern hardware (anything built in the last ~2 decades), you can run
thousands of concurrent SMTP delivery agents, without any difficulty,
their executables are loaded only once, and per connection memory utilisation
is modest. You run out of remote sites' willingness to receive your email
long before you run out of local capacity to send it.
The event driven design mostly just makes those other servers more complex,
and more prone to security bugs. Postfix 3.4 and later grudgingly do some
event-driven work because TLS connection reuse with OpenSSL is not possible
out-of-process. So the tlsproxy(8) process context switches between multiple
TLS connections, but the rest of the SMTP delivery agent is one connection
per process and performs just fine. The architecture is however more robust
and secure.
Postfix is not an HTTP server handling tens to hundreds of thousands of requests
per second, and does not benefit from the optimisations needed for those kinds
of workloads. Premature optimisations that sacrifice robustness and security
for little gain are not part of the design.
--
Viktor.
