[
https://issues.apache.org/jira/browse/KAFKA-2561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906669#comment-14906669
]
Ismael Juma commented on KAFKA-2561:
------------------------------------
[~becket_qin], there are two problems:
* The JDK SSLEngine implementation for AES-GCM doesn't use the relevant CPU
instructions yet and a highly optimised implementation of AES-GCM is faster
than a highly optimised implementation of AES-CBC. OpenSSL is twice as fast
when using AES-GCM instead of AES-CBC in a SSLEngine micro-benchmark. In the
same benchmark, JDK 8u60 with AES-CBC is four times faster than with AES-GCM.
This issue should improve in Java 9 (http://openjdk.java.net/jeps/8046943).
* The JDK SSLEngine implementation is not particularly well implemented and
generates a lot of garbage based on what Netty's Norman Maurer said in a
presentation.
I don't think we can do much about it apart from waiting for Java 9 and
potentially contributing improvements to the implementation. We are not the
first to run into this and that is why Tomcat, Finagle and Netty have the
OpenSSL implementation too.
Also note that the results I posted in the description are for a test with a
single broker and consumer running in the same machine. If you have many cores
and they are lightly loaded, you _may_ still be able to saturate the network in
the meantime.
> Optionally support OpenSSL for SSL/TLS
> ---------------------------------------
>
> Key: KAFKA-2561
> URL: https://issues.apache.org/jira/browse/KAFKA-2561
> Project: Kafka
> Issue Type: New Feature
> Components: security
> Affects Versions: 0.9.0.0
> Reporter: Ismael Juma
>
> JDK's `SSLEngine` is unfortunately a bit slow (KAFKA-2431 covers this in more
> detail). We should consider supporting OpenSSL for SSL/TLS. Initial
> experiments on my laptop show that it performs a lot better:
> {code}
> start.time, end.time, data.consumed.in.MB, MB.sec, data.consumed.in.nMsg,
> nMsg.sec, config
> 2015-09-21 14:41:58:245, 2015-09-21 14:47:02:583, 28610.2295, 94.0081,
> 30000000, 98574.6111, Java 8u60/server auth JDK
> SSLEngine/TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
> 2015-09-21 14:38:24:526, 2015-09-21 14:40:19:941, 28610.2295, 247.8900,
> 30000000, 259931.5514, Java 8u60/server auth
> OpenSslEngine/TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
> 2015-09-21 14:49:03:062, 2015-09-21 14:50:27:764, 28610.2295, 337.7751,
> 30000000, 354182.9000, Java 8u60/plaintext
> {code}
> Extracting the throughput figures:
> * JDK SSLEngine: 94 MB/s
> * OpenSSL SSLEngine: 247 MB/s
> * Plaintext: 337 MB/s (code from trunk, so no zero-copy due to KAFKA-2517)
> In order to get these figures, I used Netty's `OpenSslEngine` by hacking
> `SSLFactory` to use Netty's `SslContextBuilder` and made a few changes to
> `SSLTransportLayer` in order to workaround differences in behaviour between
> `OpenSslEngine` and JDK's SSLEngine (filed
> https://github.com/netty/netty/issues/4235 and
> https://github.com/netty/netty/issues/4238 upstream).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
