On 1/26/20 8:08 PM, Rick Macklem wrote:
John Baldwin wrote:
[stuff snipped]
Hmmm, this might be a fair bit of work indeed.
Right now KTLS only works for transmit (though I have some WIP for receive).
KTLS does assumes that the initial handshake and key negotiation is handled by
OpenSSL. OpenSSL uses custom setockopt() calls to tell the kernel which
session keys to use.
I think what you would want to do is use something like OpenSSL_connect() in
userspace, and then check to see if KTLS "worked". If it did, you can tell
the kernel it can write to the socket directly, otherwise you will have to
bounce data back out to userspace to run it through SSL_write() and have
userspace do SSL_read() and then feed data into the kernel.
The pseudo-code might look something like:
SSL *s;
s = SSL_new(...);
/* fd is the existing TCP socket */
SSL_set_fd(s, fd);
OpenSSL_connect(s);
if (BIO_get_ktls_send(SSL_get_wbio(s)) {
/* Can use KTLS for transmit. */
}
if (BIO_get_ktls_recv(SSL_get_rbio(s)) {
/* Can use KTLS for receive. */
}
So, I've been making some progress. The first stab at the daemons that do the
handshake are now on svn in base/projects/nfs-over-tls/usr.sbin/rpctlscd and
rpctlssd.
A couple of questions...
1 - I haven't found BIO_get_ktls_send() or BIO_get_ktls_recv(). Are they in some
different library?
They only existing currently in OpenSSL master (which will be OpenSSL 3.0.0
when it
is released). I have some not-yet-tested WIP changes to backport those changes
into
the base OpenSSL, but it will also add overhead to future OpenSSL imports
perhaps,
so it is something I need to work with secteam@ on to decide if it's viable
once I
have a tested PoC.
I will try to at least provide a patch to the security/openssl port to add a
KTLS
option "soon" that you could use for testing.
2 - After a successful SSL_connect(), the receive queue for the socket has
478bytes
of stuff in it. SSL_read() seems to know how to skip over it, but I
haven't
figured out a good way to do this. (I currently just do a recv(..478,0)
on the
socket.)
Any idea what to do with this? (Or will the receive side of the ktls
figure out
how to skip over it?)
I don't know yet. :-/ With the TOE-based TLS I had been testing with, this
doesn't
happen because the NIC blocks the data until it gets the key and then it's
always
available via KTLS. With software-based KTLS for RX (which I'm going to start
working on soon), this won't be the case and you will potentially have some data
already ready by OpenSSL that needs to be drained from OpenSSL before you can
depend on KTLS. It's probably only the first few messsages, but I will need to
figure
out a way that you can tell how much pending data in userland you need to read
via
SSL_read() and then pass back into the kernel before relying on KTLS (it would
just
be a single chunk of data after SSL_connect you would have to do this for).
I'm currently testing with a kernel that doesn't have options KERN_TLS and
(so long as I get rid of the 478 bytes), it then just does unencrypted RPCs.
So, I guess the big question is.... can I get access to your WIP code for KTLS
receive? (I have no idea if I can make progress on it, but I can't do a lot more
before I have that.)
The WIP only works right now if you have a Chelsio T6 NIC as it uses the T6's
TCP
offload engine to do TLS. If you don't have that gear, ping me off-list. It
would also let you not worry about the SSL_read case for now for initial
testing.
--
John Baldwin
_______________________________________________
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
