We are using 0.10.0.1 of Kafka and (Java) client libraries. We recently
decided to start using SSL for Kafka communication between broker and
clients. Right now, we have a pretty basic setup with just 1 broker with
SSL keystore setup and the Java client(s) communicate using the
Producer/Consumer APIs against this single broker. There's no client
auth (intentionally) right now. We also have plain text enabled for the
initial testing.
What we have noticed is that the consumer/producer performance when SSL
is enabled is noticeably poor when compared to plain text. I understand
that there are expected to be performance impacts when SSL is enabled
but the order of magnitude is too high and in fact it shows up in a very
noticeable fashion in our product. I do have the numbers, but I haven't
been able to narrow it down yet (because there's nothing that stands
out, except that it's slow). Our application code is exactly the same
between non-SSL and SSL usage.
Furthermore, I'm aware of this specific JIRA in Kafka
https://issues.apache.org/jira/browse/KAFKA-2561 which acknowledges a
similar issue. So what I would like to know is, in context of Kafka
0.10.x releases and Java 8 support, are there any timelines that the dev
team is looking for in terms of improving this performance issue (which
I believe requires usage of OpenSSL or other non-JDK implementations of
SSLEngine)? We would like to go for GA of our product in the next couple
of months and in order to do that, we do plan to have Kafka over SSL
working with reasonably good performance, but the current performance
isn't promising. Expecting this to be fixed in the next couple of months
and have it available in 0.10.x is probably too much to expect, but if
we know the plans around this, we should be able to come up with a plan
of our own for our product.
-Jaikiran
- Plans to improve SSL performance in Kafka, for 0.10.x? Jaikiran Pai
-
