Oh, and the way we narrowed it down was somewhat oblique. Because their logs said a TLS connection was established we had a hard time convincing them it wasn't. They were convinced it was us who was broke.
We had to send them a PCAP and then they ran one and got the same results. We were communicating via their IronPort "secure email" system and I noticed that the Cisco copyright notice on their messages was from 2012. That put me on the path to look at the Cisco release notes. Once I pointed out that they seemed to be a bit behind and there were fixes in later versions, the conversation went in a different direction. :-) > From: sixsigm...@hotmail.com > To: bl...@ispn.net; nanog@nanog.org > Subject: RE: Verizon FiOS outbound mail TLS problem - Superpages people here? > Date: Sat, 6 Jun 2015 19:13:38 -0400 > > We had a similar issue around November last year where an upgrade on our > PostFix MTA to a current version of OpenSSL, which has Mandatory TLS > enabled for certain recipient domains, suddenly started generating the > same errors with just one recipient domain. > > We eventually figured > out that the problem was they were running an outdated version of the > AsyncOS on their Cisco IronPorts. Firmware versions prior to 8.02 had > several problems with TLS and one of them was an inability to > interoperate with senders who used a newer version of OpenSSL. Their > IronPort logs in fact showed a TLS connection was established when it > wasn't. (We had switched them to Opportunistic TLS to be able to send > emails but their logs still showed TLS while a PCAP showed clear text > SMTP.) > > As soon as that company updated their IronPorts to a v8.5 > variant the problem went away. They would not tell us what version they > used to run but did confirm it was prior to v8.02. > > Interestingly, www.checktls.com > said they were OK. The admins at Check TLS confirmed that, at that time > (the end of 2014), they were running a version of OpenSSL on their > website that was still compatible with the older AsyncOS version. > > FWIW, > > Ray > > Date: Thu, 4 Jun 2015 11:46:35 -0500 > > From: bl...@ispn.net > > To: nanog@nanog.org > > Subject: Re: Verizon FiOS outbound mail TLS problem - Superpages people > > here? > > > > I have no relation, but as a mail server operator I can say that I > > wouldn't be surprised if this is actually a TLS version mismatch or > > intolerance problem. I would suggest ensuring that both ends support TLS > > 1.0, 1.1, and 1.2 and use version tolerant TLS implementations. Next on > > the short list would be not having compatible cyphers between the two > > servers. > > > > Either way, since the error was a 403 error, the expected behavior would > > be to queue and retry in plain text; Sounds like a broken MTA > > implementation or misconfiguration if the sending servers do not revert > > to plain text. > > > > --Blake > > > > Jay Ashworth wrote on 6/4/2015 11:15 AM: > > > Anyone on the list who does outbound delivery for Verizon (which I think > > > is actually Superpages)? A client has smart-hosted outbounds to *one* > > > of his customers bouncing suddenly with > > > > > > Deferred: 403 4.7.0 TLS handshake failed. > > > > > > *My* inclination is to think that a cert expired somewhere, but his > > > non-tech > > > contact there tells him that the tech people think things are ok. > > > > > > I'm trying to get a mailer log fragment from them. > > > > > > Cheers, > > > -- jra > > > > > >