Hi Guys, We are trying to secure the Kafka-Cluster in order to enforce topic level security based on sentry roles. We are seeing a big performance impact after SSL_SASL is enabled. I read multiple blog posts describing the performance impact but that also said that the impact would be negligible, but I see a significant hit in my case (60-85%). Could someone please suggest if you have seen this kind of performance impact and what is done to overcome the same? We cannot afford to have an unsecure cluster.
In the below example, I'm trying to produce a record of size 1200 bytes and set the batch.size property to 100000. Before Securityis Enabled: kafka-producer-perf-test --topic myPerformanceTestTopic --num-records 1000000 --print-metrics --record-size 1200 --throughput 1000000 --producer-props acks=1 bootstrap.servers=<host_name>:9092 batch.size=100000 18/08/22 10:12:37 INFO utils.AppInfoParser: Kafka commitId : unknown 294801 records sent, 58960.2 records/sec (67.47 MB/sec), 55.0 ms avg latency, 265.0 max latency. 275261 records sent, 49632.3 records/sec (56.80 MB/sec), 251.1 ms avg latency, 1420.0 max latency. 293654 records sent, 58730.8 records/sec (67.21 MB/sec), 244.0 ms avg latency, 1485.0 max latency. 1000000 records sent, 57733.387218 records/sec (66.07 MB/sec), 162.61 ms avg latency, 1485.00 ms max latency, 73 ms 50th, 546 ms 95th, 1459 ms 99th, 1477 ms 99.9th. After Security is Enabled: kafka-producer-perf-test --topic myPerformanceTestTopic --num-records 1000000 --print-metrics --record-size 1200 --throughput 1000000 --producer-props acks=1 bootstrap.servers=<host_name>:9094 batch.size=100000 --producer.config <my_secure_path>/sasl-ssl-auth.properties 18/08/22 12:33:36 INFO utils.AppInfoParser: Kafka commitId : unknown 39610 records sent, 7917.2 records/sec (9.06 MB/sec), 1669.5 ms avg latency, 2608.0 max latency. 58320 records sent, 11659.3 records/sec (13.34 MB/sec), 2514.2 ms avg latency, 3242.0 max latency. 92016 records sent, 18399.5 records/sec (21.06 MB/sec), 1579.2 ms avg latency, 2119.0 max latency. 84645 records sent, 16925.6 records/sec (19.37 MB/sec), 1578.7 ms avg latency, 2111.0 max latency. 106515 records sent, 21286.0 records/sec (24.36 MB/sec), 1300.4 ms avg latency, 1662.0 max latency. l74520 records sent, 14895.1 records/sec (17.05 MB/sec), 1688.1 ms avg latency, 2350.0 max latency. 77841 records sent, 15562.0 records/sec (17.81 MB/sec), 1749.2 ms avg latency, 2030.0 max latency. 94851 records sent, 18970.2 records/sec (21.71 MB/sec), 1495.4 ms avg latency, 2111.0 max latency. 102870 records sent, 20569.9 records/sec (23.54 MB/sec), 1345.6 ms avg latency, 1559.0 max latency. 121095 records sent, 24219.0 records/sec (27.72 MB/sec), 1143.8 ms avg latency, 1738.0 max latency. 126036 records sent, 25202.2 records/sec (28.84 MB/sec), 1080.5 ms avg latency, 1384.0 max latency. 1000000 records sent, 17906.385417 records/sec (20.49 MB/sec), 1465.52 ms avg latency, 3242.00 ms max latency, 1355 ms 50th, 2339 ms 95th, 2914 ms 99th, 3211 ms 99.9th Thank you, Harsha
