PR #22387 opened by Nariman-Sayed
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/22387
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/22387.patch

Some WebRTC servers such as Pion send STUN packets concurrently during
the DTLS handshake. Unlike OpenSSL and GnuTLS which filter non-DTLS
packets internally, mbedtls passes all received UDP packets directly to
its DTLS state machine, causing the handshake to fail.

Fix this by wrapping ffurl_read() in a do-while loop that discards any
packet whose first byte falls outside the DTLS content-type range
[20, 63], as specified by RFC 5764 Section 5.1.2.

Signed-off-by: Nariman-Sayed <[email protected]>


>From 070a8a8543e84979e05c110941a68424abf1842a Mon Sep 17 00:00:00 2001
From: Nariman-Sayed <[email protected]>
Date: Thu, 5 Mar 2026 08:05:58 +0200
Subject: [PATCH] avformat/tls_mbedtls: fix DTLS handshake failure when
 receiving non-DTLS packets

Some WebRTC servers such as Pion send STUN packets concurrently during
the DTLS handshake. Unlike OpenSSL and GnuTLS which filter non-DTLS
packets internally, mbedtls passes all received UDP packets directly to
its DTLS state machine, causing the handshake to fail.

Fix this by wrapping ffurl_read() in a do-while loop that discards any
packet whose first byte falls outside the DTLS content-type range
[20, 63], as specified by RFC 5764 Section 5.1.2.

Signed-off-by: Nariman-Sayed <[email protected]>
---
 libavformat/tls_mbedtls.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/libavformat/tls_mbedtls.c b/libavformat/tls_mbedtls.c
index f13833b6ed..deb2d16d9d 100644
--- a/libavformat/tls_mbedtls.c
+++ b/libavformat/tls_mbedtls.c
@@ -381,8 +381,12 @@ static int mbedtls_recv(void *ctx, unsigned char *buf, 
size_t len)
     TLSContext *tls_ctx = (TLSContext*) ctx;
     TLSShared *shr = &tls_ctx->tls_shared;
     URLContext *h = shr->is_dtls ? shr->udp : shr->tcp;
-    int ret = ffurl_read(h, buf, len);
-    if (ret >= 0) {
+    int ret;
+
+    do {
+        ret = ffurl_read(h, buf, len);
+        if (ret <= 0)
+            break;
         if (shr->is_dtls && shr->listen && !tls_ctx->dest_addr_len) {
             int err_ret;
 
@@ -394,8 +398,11 @@ static int mbedtls_recv(void *ctx, unsigned char *buf, 
size_t len)
             }
             av_log(tls_ctx, AV_LOG_TRACE, "Set UDP remote addr on UDP socket, 
now 'connected'\n");
         }
+        /* Skip non-DTLS packets such as STUN to avoid handshake failures. */
+    } while (shr->is_dtls && (buf[0] < 20 || buf[0] > 63));
+
+    if (ret >= 0)
         return ret;
-    }
     if (h->max_packet_size && len > h->max_packet_size)
         return MBEDTLS_ERR_SSL_BUFFER_TOO_SMALL;
 
-- 
2.52.0

_______________________________________________
ffmpeg-devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]

Reply via email to