Ben Stopford created KAFKA-2517:
-----------------------------------
Summary: Performance Regression post SSL implementation
Key: KAFKA-2517
URL: https://issues.apache.org/jira/browse/KAFKA-2517
Project: Kafka
Issue Type: Bug
Reporter: Ben Stopford
Assignee: Ben Stopford
Priority: Critical
Fix For: 0.8.3
It would appear that we incurred a performance regression on submission of the
SSL work affecting the performance of the new Kafka Consumer.
Running with 1KB messages. Macbook 2.3 GHz Intel Core i7, 8GB, APPLE SSD
SM256E. Single server instance. All local.
kafka-consumer-perf-test.sh ... --messages 3000000 --new-consumer
Pre-SSL changes (commit 503bd36647695e8cc91893ffb80346dd03eb0bc5)
Steady state throughputs = 234.8 MB/s
(2861.5913, 234.8261, 3000596, 246233.0543)
Post-SSL changes (commit 13c432f7952de27e9bf8cb4adb33a91ae3a4b738)
Steady state throughput = 178.1 MB/s
(2861.5913, 178.1480, 3000596, 186801.7182)
Implication is a 25% reduction in consumer throughput for these test
conditions.
This appears to be caused by the use of PlaintextTransportLayer rather than
SocketChannel in FileMessageSet.writeTo() meaning a zero copy transfer is not
invoked.
Switching to the use of a SocketChannel directly in FileMessageSet.writeTo()
yields the following result:
Steady state throughput = 281.8 MB/s
(2861.5913, 281.8191, 3000596, 295508.7650)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
