Hi, I am trying to implement/test ktls as per the blog article
https://www.nginx.com/blog/improving-nginx-performance-with-kernel-tls/#tls-protocol ########################### This is done on CentOS8 VM # uname -r 4.18.0-348.7.1.el8_5.x86_64 ########################### # openssl-3.0.1/.openssl/bin/openssl ciphers TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES256-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:RSA-PSK-AES256-GCM-SHA384:DHE-PSK-AES256-GCM-SHA384:RSA-PSK-CHACHA20-POLY1305:DHE-PSK-CHACHA20-POLY1305:ECDHE-PSK-CHACHA20-POLY1305:AES256-GCM-SHA384:PSK-AES256-GCM-SHA384:PSK-CHACHA20-POLY1305:RSA-PSK-AES128-GCM-SHA256:DHE-PSK-AES128-GCM-SHA256:AES128-GCM-SHA256:PSK-AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:ECDHE-PSK-AES256-CBC-SHA384:ECDHE-PSK-AES256-CBC-SHA:SRP-RSA-AES-256-CBC-SHA:SRP-AES-256-CBC-SHA:RSA-PSK-AES256-CBC-SHA384:DHE-PSK-AES256-CBC-SHA384:RSA-PSK-AES256-CBC-SHA:DHE-PSK-AES256-CBC-SHA:AES256-SHA:PSK-AES256-CBC-SHA384:PSK-AES256-CBC-SHA:ECDHE-PSK-AES128-CBC-SHA256:ECDHE-PSK-AES128-CBC-SHA:SRP-RSA-AES-128-CBC-SHA:SRP-AES-128-CBC-SHA:RSA-PSK-AES128-CBC-SHA256:DHE-PSK-AES128-CBC-SHA256:RSA-PSK-AES128-CBC-SHA:DHE-PSK-AES128-CBC-SHA:AES128-SHA:PSK-AES128-CBC-SHA256:PSK-AES128-CBC-SHA ########################### # /usr/sbin/nginx-debug -V nginx version: nginx/1.21.6 built by gcc 8.5.0 20210514 (Red Hat 8.5.0-4) (GCC) built with OpenSSL 3.0.1 14 Dec 2021 TLS SNI support enabled configure arguments: --with-debug --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/etc/nginx/modules --with-pcre=./pcre2-10.39 --with-pcre-jit --with-zlib=./zlib-1.2.11 --with-openssl=./openssl-3.0.1 --with-openssl-opt=enable-ktls --with-openssl-opt=enable-tls1_3 --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error_log ############################ The debug log does not show any signs of ktls in use (snippet from the log provided below on download of a 1G file) 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame 0000000002077A08 was sent 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent: 0000000002077A08 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame 0000000002077D30 was sent 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent: 0000000002077D30 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame 0000000002075E58 was sent 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent: 0000000002075E58 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame 0000000002075F60 was sent 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent: 0000000002075F60 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame 0000000002077BA8 was sent 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent: 0000000002077BA8 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame 0000000002077AA0 was sent 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent: 0000000002077AA0 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame 0000000002077890 was sent 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent: 0000000002077890 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame 0000000002075BC8 was sent 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent: 0000000002075BC8 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http write filter 0000000000000000 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 read: 15, 0000000002791FC0, 32768, 21168128 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 read: 15, 0000000002791FC0, 32768, 21168128 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 read: 15, 0000000002799FD0, 32768, 21200896 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http postpone filter "/1G?" 0000000002075DD8 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 write new buf t:1 f:1 0000000002791FC0, pos 0000000002791FC0, size: 32768 file: 21168128, size: 32768 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 write new buf t:1 f:1 0000000002799FD0, pos 0000000002799FD0, size: 32768 file: 21200896, size: 32768 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http write filter: l:0 f:1 s:65536 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http write filter limit 2097152 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 send chain: 0000000002075DF8 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 windows: conn:10297344 stream:868352 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame 0000000002075BC8: len:8192 flags:0 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame 0000000002077890: len:8192 flags:0 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame 0000000002077AA0: len:8192 flags:0 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame 0000000002077BA8: len:8192 flags:0 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame 0000000002075F60: len:8192 flags:0 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame 0000000002075E58: len:8192 flags:0 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame 0000000002077D30: len:8192 flags:0 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame 0000000002077A08: len:8192 flags:0 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out: 0000000002077A08 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out: 0000000002077D30 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out: 0000000002075E58 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out: 0000000002075F60 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out: 0000000002077BA8 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out: 0000000002077AA0 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out: 0000000002077890 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out: 0000000002075BC8 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 9 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 9 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 8174 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL to write: 16384 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL_write: 16384 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 18 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 9 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 9 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 8156 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL to write: 16384 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL_write: 16384 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 36 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 9 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 9 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 8138 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL to write: 16384 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL_write: 16384 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 54 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 9 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 9 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 8120 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL to write: 16384 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL_write: 16384 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL buf copy: 72 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL to write: 72 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 SSL_write: 72 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame 0000000002075BC8 was sent 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent: 0000000002075BC8 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame 0000000002077890 was sent 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent: 0000000002075BC8 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame 0000000002077890 was sent 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent: 0000000002077890 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame 0000000002077AA0 was sent 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent: 0000000002077AA0 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame 0000000002077BA8 was sent 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent: 0000000002077BA8 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame 0000000002075F60 was sent 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent: 0000000002075F60 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame 0000000002075E58 was sent 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent: 0000000002075E58 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame 0000000002077D30 was sent 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent: 0000000002077D30 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 DATA frame 0000000002077A08 was sent 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame sent: 0000000002077A08 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http write filter 0000000000000000 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 read: 15, 0000000002799FD0, 32768, 21233664 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 read: 15, 0000000002791FC0, 32768, 21266432 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http postpone filter "/1G?" 0000000002075DE8 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 write new buf t:1 f:1 0000000002799FD0, pos 0000000002799FD0, size: 32768 file: 21233664, size: 32768 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 write new buf t:1 f:1 0000000002791FC0, pos 0000000002791FC0, size: 32768 file: 21266432, size: 32768 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http write filter: l:0 f:1 s:65536 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 send chain: 0000000002077C50 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 windows: conn:10231808 stream:802816 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame 0000000002077A08: len:8192 flags:0 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame 0000000002077D30: len:8192 flags:0 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame 0000000002075E58: len:8192 flags:0 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame 0000000002075F60: len:8192 flags:0 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame 0000000002077BA8: len:8192 flags:0 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame 0000000002077AA0: len:8192 flags:0 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame 0000000002077890: len:8192 flags:0 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2:1 create DATA frame 0000000002075BC8: len:8192 flags:0 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out: 0000000002075BC8 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out: 0000000002077890 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out: 0000000002077AA0 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out: 0000000002077BA8 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out: 0000000002075F60 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out: 0000000002075E58 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out: 0000000002077D30 sid:1 bl:0 len:8192 2022/01/27 13:41:33 [debug] 1843564#1843564: *140 http2 frame out: 0000000002077A08 sid:1 bl:0 len:8192 ############################################# [root@65-108-156-104 nginx-1.21.6]# grep SSL_sendfile /var/log/nginx/error_log [root@65-108-156-104 nginx-1.21.6]# grep BIO /var/log/nginx/error_log [root@65-108-156-104 nginx-1.21.6]# There is no SSL_sendfile in the log ############################################## # TLS Settings ssl_protocols TLSv1.3; ssl_session_cache shared:SSL:32m; ssl_dhparam /etc/nginx/ssl/dhparam.pem; ssl_session_timeout 1d; ssl_session_tickets off; ssl_ocsp_cache shared:ocspcache:10m; server{ ... ssl_conf_command Options KTLS; .. } ################################################# What am I doing wrong? Thanks in advance, -- *Anoop P Alias*
_______________________________________________ nginx mailing list -- nginx@nginx.org To unsubscribe send an email to nginx-le...@nginx.org
