Even though OpenSSL is much faster than the Java 8 TLS implementation (I haven't tested against Java 9, which is much faster than Java 8, but probably still slower than OpenSSL), all the tests were without zero copy in the sense that is being discussed here (i.e. sendfile). To benefit from sendfile with TLS, kernel-level changes/modules are required:
https://github.com/ktls/af_ktls http://www.phoronix.com/scan.php?page=news_item&px=FreeBSD-Faster-Sendfile Ismael On Mon, Mar 6, 2017 at 4:18 PM, Todd Palino <tpal...@gmail.com> wrote: > So that’s not quite true, Hans. First, as far as the performance hit being > not a big impact (25% is huge). Or that it’s to be expected. Part of the > problem is that the Java TLS implementation does not support zero copy. > OpenSSL does, and in fact there’s been a ticket open to allow Kafka to > support using OpenSSL for a while now: > > https://issues.apache.org/jira/browse/KAFKA-2561 > > > > > On Mon, Mar 6, 2017 at 6:30 AM, Hans Jespersen <h...@confluent.io> wrote: > > > > > Its not a single message at a time that is encrypted with TLS its the > > entire network byte stream so a Kafka broker can’t even see the Kafka > > Protocol tunneled inside TLS unless it’s terminated at the broker. > > It is true that losing the zero copy optimization impacts performance > > somewhat but it’s not what I would call a “big impact” because Kafka > does > > a lot of other things to get it’s performance (like using page cache and > > doing lots on sequential disk I/O). The difference should be something in > > the order of 25-30% slower with TLS enabled which is about what you would > > see with any other messaging protocol with TLS on vs off. > > > > If you wanted to encrypt each message independently before sending to > > Kafka then zero copy would still be in effect and all the consumers would > > get the same encrypted message (and have to understand how to decrypt > it). > > > > -hans > > > > > > > > > On Mar 6, 2017, at 5:38 AM, Nicolas Motte <lingusi...@gmail.com> > wrote: > > > > > > Hi everyone, > > > > > > I understand one of the reasons why Kafka is performant is by using > > > zero-copy. > > > > > > I often hear that when encryption is enabled, then Kafka has to copy > the > > > data in user space to decode the message, so it has a big impact on > > > performance. > > > > > > If it is true, I don t get why the message has to be decoded by Kafka. > I > > > would assume that whether the message is encrypted or not, Kafka simply > > > receives it, appends it to the file, and when a consumer wants to read > > it, > > > it simply reads at the right offset... > > > > > > Also I m wondering if it s the case if we don t use keys (pure queuing > > > system with key=null). > > > > > > Cheers > > > Nico > > > > > > > -- > *Todd Palino* > Staff Site Reliability Engineer > Data Infrastructure Streaming > > > > linkedin.com/in/toddpalino >
