I am developing an application similar to a VPN which uses OpenSSL for DTLS and am trying to sort out path MTU discovery. The first issue I am having is that the man pages for SSL_write and SSL_read provide the following warning:
> When an SSL_write() [or SSL_read()] operation has to be repeated because of SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE, it must be repeated with the same arguments. I would expect the reason for this is that the operation may have been partially completed but can't be fully completed until the socket is ready for read or write. So this should not apply to DTLS because it should only be sending complete datagrams, is that correct? Otherwise there seems to be a rather serious problem with UDP sockets that set the don't fragment bit: A call to SSL_write() may return WANT_READ or WANT_WRITE (e.g. because of a renegotiation), and then when the renegotiation completes and SSL_write() is called again the data could be too big for the MTU and the OS could return EMSGSIZE. Any subsequent call with the same arguments would then have the same result so calling SSL_write with those arguments will never succeed, but calling with different arguments is prohibited by the documentation. And clearing the don't fragment bit is no solution because DF may be required, e.g. because the payload itself is a IP packet with DF set, or the peer device can't reassemble large datagrams, or the firewall is known to drop fragmented packets. So is it safe to assume that the warning applies only to TLS and not DTLS? The second issue is that I'm trying to figure out how to determine the DTLS protocol overhead in order to calculate the path MTU of the connection. I can determine the path MTU of the underlying transport and only need to add to it the DTLS overhead. The trouble is that this depends on the ciphersuite negotiated because that determines the size of the HMAC, whether the data must be padded to a block boundary, etc. So I need some way to get OpenSSL to tell me this information, but I don't see anything like “int SSL_get_dtls_overhead(SSL* ssl)” unless I'm missing it somewhere. The ideal would be something like “int SSL_get_connection_mtu(SSL* ssl, int transport_mtu)” which would provide the MTU of the TLS connection given the MTU of the socket, that way OpenSSL could provide the exact number instead of assuming the worst-case amount of block padding. How do I get this information out of OpenSSL?
