Those possible (and likely) explanations for a few of the rejections are actually really insightful. It sounds like we agree that there are cases where 4xx should not be retried. That's progress! The question around the downside of queuing them up for a while is essentially 2-part.
First, we send a lot of email; 9 billion messages on black friday alone. Each message generates an average of 8 events (deferrals, opens, clicks, spam reports, unsubscribes etc). Storage cost is a small part of the reason for minimizing MTA chatter. It's not a big enough factor for us to make irresponsible decisions in this area to save money, but it's a factor. Second factor (and the main factor) is our customers. We send 32 deferral events over the course of 72 hours to our customers' data warehouses for each message that reaches max queue time. That's 32 events they are consuming and storing as well. The customers, not being RFC-reading (and RFC-writing) mail operators like the group here, ask us why we keep retrying for so long on messages we know will never deliver. To the customer these deferral events are just a nuisance. Making matters worse, they incorrectly believe that *our* MTA is to blame. That we should be handling this better. So do we read the RFC to our customers and tell them to pound sand? Or do we take some time and see if we can help make their experience a little better *and* save us some money at the same time? Luke On Mon, Feb 27, 2023 at 8:24 PM Michael Orlitzky via mailop < mailop@mailop.org> wrote: > On Mon, 2023-02-27 at 18:53 -0700, Luke via mailop wrote: > > For better or worse, *not *retrying *some *4xx is even easier to > > justify. > > Here are a few massive examples. > > > > 421 4.7.1 Message permanently deferred: > > > > 452 Sender Rejected: > > Keeping in mind that the RFC says not to use the text in your decision, > these two sound the most reasonable. Skipping for brevity. > > > > 450 4.1.1 Recipient Address Rejected: this one is tough because with > > some MXs retries result in delivery, in others, it's a dead end. > > Dynamic rule handling per receiving MX would be awesome, but it would > > require machine learning to accomplish at scale. > > This could postfix's unverified_recipient_reject_code. When (for > example) postfix sits in front of an Exchange server and when a full > list of valid recipients is not available, postfix can launch a > background probe to see if the recipient is valid in Exchange before it > will accept a message destined for it (otherwise, you get backscatter). > If postfix is up but Exchange is down, it'll give you this rejection. > > > > 451 Relay Not Permitted: this one can mean a lot of things and comes > > in a few flavors, but when the data shows retrying never results in a > > delivery, we permfail it. > > This happens when somebody's domain expires and the MX record gets > pointed somewhere it shouldn't. This is the right response because when > they finally find the guy who knows the guy who knows the guy who > receives the email that resets the password at the registrar three days > later, it's nice to have all their mail start flowing as if nothing > ever happened. > > > > Unlike the rare 5xx we decide to retry, these 4xx that shouldn't be > > retried are far more numerous and come from large-ish reputable > > receivers. Believe me, simply retrying 4xx and failing 5xx would make > > our jobs a lot easier. Unfortunately the real world isn't so simple > > Not pounding on servers that have told you to go away with a 5xx has > obvious benefits for both sides though. What's the downside to leaving > the 4xx in the queue for a while? No one is going to fault you for > retrying a message they told you to retry, and even 1% of legitimate > mail is a lot to toss out without a good reason. > > Finally, while it might be 1% of *your* mail it's not 1% of *our* mail. > Michael Scott said Wayne Gretzky said that you lose 100% of the > messages you don't retry to recipients who would have accepted them. > > _______________________________________________ > mailop mailing list > mailop@mailop.org > https://list.mailop.org/listinfo/mailop >
_______________________________________________ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
