[
https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15268242#comment-15268242
]
Jiangjie Qin commented on KAFKA-3565:
-------------------------------------
[~junrao] I updated the results in the Google sheet. The results are in
test_run9 and test_run10
The linger.ms was run with 0 and 10000 ms. The batch size is set to 1000000.
The modified test will flush when the batch is almost full (calculated by
BatchSize/RecordSize - 10). Each test scenario runs at least 30 seconds.
Trunk performs better than 0.9 in all the tested cases. (max.in.flight.requests
actually does not matter here, though.)
{noformat}
run 9:
1st: max.in.flight.requests.per.connection=1, valueBound=500, linger.ms=0,
messageSize=100, compression.type=gzip (2.19 > 1.83, 19%)
1st: max.in.flight.requests.per.connection=1, valueBound=500, linger.ms=0,
messageSize=100, compression.type=snappy (18.17 > 14.96, 21%)
1st: max.in.flight.requests.per.connection=1, valueBound=500, linger.ms=0,
messageSize=1000, compression.type=gzip (4.32 > 2.74, 57%)
1st: max.in.flight.requests.per.connection=1, valueBound=500, linger.ms=0,
messageSize=1000, compression.type=snappy (28.66 > 20.59, 39%)
1st: max.in.flight.requests.per.connection=1, valueBound=500,
linger.ms=100000, messageSize=100, compression.type=gzip (1.61 > 1.09,
47%)
1st: max.in.flight.requests.per.connection=1, valueBound=500,
linger.ms=100000, messageSize=100, compression.type=snappy (10.77 > 8.41,
28%)
1st: max.in.flight.requests.per.connection=1, valueBound=500,
linger.ms=100000, messageSize=1000, compression.type=gzip (2.56 > 1.51,
69%)
1st: max.in.flight.requests.per.connection=1, valueBound=500,
linger.ms=100000, messageSize=1000, compression.type=snappy (21.37 >
16.24, 31%)
1st: max.in.flight.requests.per.connection=1, valueBound=5000, linger.ms=0,
messageSize=100, compression.type=gzip (2.36 > 1.89, 24%)
1st: max.in.flight.requests.per.connection=1, valueBound=5000, linger.ms=0,
messageSize=100, compression.type=snappy (19.25 > 14.99, 28%)
1st: max.in.flight.requests.per.connection=1, valueBound=5000, linger.ms=0,
messageSize=1000, compression.type=gzip (5.13 > 2.89, 77%)
1st: max.in.flight.requests.per.connection=1, valueBound=5000, linger.ms=0,
messageSize=1000, compression.type=snappy (32.24 > 21.95, 46%)
1st: max.in.flight.requests.per.connection=1, valueBound=5000,
linger.ms=100000, messageSize=100, compression.type=gzip (1.69 > 1.13,
49%)
1st: max.in.flight.requests.per.connection=1, valueBound=5000,
linger.ms=100000, messageSize=100, compression.type=snappy (12.34 > 9.61,
28%)
1st: max.in.flight.requests.per.connection=1, valueBound=5000,
linger.ms=100000, messageSize=1000, compression.type=gzip (2.68 > 1.61,
66%)
1st: max.in.flight.requests.per.connection=1, valueBound=5000,
linger.ms=100000, messageSize=1000, compression.type=snappy (23.70 >
18.03, 31%)
1st: max.in.flight.requests.per.connection=1, valueBound=50000, linger.ms=0,
messageSize=100, compression.type=gzip (2.58 > 2.04, 26%)
1st: max.in.flight.requests.per.connection=1, valueBound=50000, linger.ms=0,
messageSize=100, compression.type=snappy (19.44 > 16.20, 20%)
1st: max.in.flight.requests.per.connection=1, valueBound=50000, linger.ms=0,
messageSize=1000, compression.type=gzip (5.65 > 2.97, 90%)
1st: max.in.flight.requests.per.connection=1, valueBound=50000, linger.ms=0,
messageSize=1000, compression.type=snappy (33.82 > 26.91, 25%)
1st: max.in.flight.requests.per.connection=1, valueBound=50000,
linger.ms=100000, messageSize=100, compression.type=gzip (1.74 > 1.17,
48%)
1st: max.in.flight.requests.per.connection=1, valueBound=50000,
linger.ms=100000, messageSize=100, compression.type=snappy (12.67 >
10.60, 19%)
1st: max.in.flight.requests.per.connection=1, valueBound=50000,
linger.ms=100000, messageSize=1000, compression.type=gzip (2.61 > 1.59,
64%)
1st: max.in.flight.requests.per.connection=1, valueBound=50000,
linger.ms=100000, messageSize=1000, compression.type=snappy (24.06 >
21.71, 10%)
1st: max.in.flight.requests.per.connection=5, valueBound=500, linger.ms=0,
messageSize=100, compression.type=gzip (2.06 > 1.88, 9%)
1st: max.in.flight.requests.per.connection=5, valueBound=500, linger.ms=0,
messageSize=100, compression.type=snappy (16.77 > 13.71, 22%)
1st: max.in.flight.requests.per.connection=5, valueBound=500, linger.ms=0,
messageSize=1000, compression.type=gzip (3.90 > 2.73, 42%)
1st: max.in.flight.requests.per.connection=5, valueBound=500, linger.ms=0,
messageSize=1000, compression.type=snappy (29.85 > 19.11, 56%)
1st: max.in.flight.requests.per.connection=5, valueBound=500,
linger.ms=100000, messageSize=100, compression.type=gzip (1.62 > 1.09,
48%)
1st: max.in.flight.requests.per.connection=5, valueBound=500,
linger.ms=100000, messageSize=100, compression.type=snappy (11.18 > 8.68,
28%)
1st: max.in.flight.requests.per.connection=5, valueBound=500,
linger.ms=100000, messageSize=1000, compression.type=gzip (2.53 > 1.51,
67%)
1st: max.in.flight.requests.per.connection=5, valueBound=500,
linger.ms=100000, messageSize=1000, compression.type=snappy (21.45 >
16.14, 32%)
1st: max.in.flight.requests.per.connection=5, valueBound=5000, linger.ms=0,
messageSize=100, compression.type=gzip (2.28 > 2.01, 13%)
1st: max.in.flight.requests.per.connection=5, valueBound=5000, linger.ms=0,
messageSize=100, compression.type=snappy (18.67 > 14.18, 31%)
1st: max.in.flight.requests.per.connection=5, valueBound=5000, linger.ms=0,
messageSize=1000, compression.type=gzip (5.28 > 2.95, 78%)
1st: max.in.flight.requests.per.connection=5, valueBound=5000, linger.ms=0,
messageSize=1000, compression.type=snappy (34.24 > 22.01, 55%)
1st: max.in.flight.requests.per.connection=5, valueBound=5000,
linger.ms=100000, messageSize=100, compression.type=gzip (1.61 > 1.14,
41%)
1st: max.in.flight.requests.per.connection=5, valueBound=5000,
linger.ms=100000, messageSize=100, compression.type=snappy (12.95 > 9.82,
31%)
1st: max.in.flight.requests.per.connection=5, valueBound=5000,
linger.ms=100000, messageSize=1000, compression.type=gzip (2.68 > 1.63,
64%)
1st: max.in.flight.requests.per.connection=5, valueBound=5000,
linger.ms=100000, messageSize=1000, compression.type=snappy (24.54 >
18.31, 34%)
1st: max.in.flight.requests.per.connection=5, valueBound=50000, linger.ms=0,
messageSize=100, compression.type=gzip (2.50 > 2.17, 15%)
1st: max.in.flight.requests.per.connection=5, valueBound=50000, linger.ms=0,
messageSize=100, compression.type=snappy (18.78 > 15.79, 18%)
1st: max.in.flight.requests.per.connection=5, valueBound=50000, linger.ms=0,
messageSize=1000, compression.type=gzip (5.51 > 3.14, 75%)
1st: max.in.flight.requests.per.connection=5, valueBound=50000, linger.ms=0,
messageSize=1000, compression.type=snappy (35.34 > 28.92, 22%)
1st: max.in.flight.requests.per.connection=5, valueBound=50000,
linger.ms=100000, messageSize=100, compression.type=gzip (1.75 > 1.15,
52%)
1st: max.in.flight.requests.per.connection=5, valueBound=50000,
linger.ms=100000, messageSize=100, compression.type=snappy (13.41 >
11.02, 21%)
1st: max.in.flight.requests.per.connection=5, valueBound=50000,
linger.ms=100000, messageSize=1000, compression.type=gzip (2.62 > 1.58,
65%)
1st: max.in.flight.requests.per.connection=5, valueBound=50000,
linger.ms=100000, messageSize=1000, compression.type=snappy (25.40 >
22.47, 13%)
run 10:
1st: max.in.flight.requests.per.connection=1, valueBound=500, linger.ms=0,
messageSize=100, compression.type=gzip (2.15 > 1.81, 18%)
1st: max.in.flight.requests.per.connection=1, valueBound=500, linger.ms=0,
messageSize=100, compression.type=snappy (18.72 > 14.84, 26%)
1st: max.in.flight.requests.per.connection=1, valueBound=500, linger.ms=0,
messageSize=1000, compression.type=gzip (3.94 > 2.74, 43%)
1st: max.in.flight.requests.per.connection=1, valueBound=500, linger.ms=0,
messageSize=1000, compression.type=snappy (29.69 > 20.33, 46%)
1st: max.in.flight.requests.per.connection=1, valueBound=500,
linger.ms=100000, messageSize=100, compression.type=gzip (1.64 > 1.09,
50%)
1st: max.in.flight.requests.per.connection=1, valueBound=500,
linger.ms=100000, messageSize=100, compression.type=snappy (10.93 > 8.41,
29%)
1st: max.in.flight.requests.per.connection=1, valueBound=500,
linger.ms=100000, messageSize=1000, compression.type=gzip (2.50 > 1.51,
65%)
1st: max.in.flight.requests.per.connection=1, valueBound=500,
linger.ms=100000, messageSize=1000, compression.type=snappy (21.25 >
16.19, 31%)
1st: max.in.flight.requests.per.connection=1, valueBound=5000, linger.ms=0,
messageSize=100, compression.type=gzip (2.41 > 1.88, 28%)
1st: max.in.flight.requests.per.connection=1, valueBound=5000, linger.ms=0,
messageSize=100, compression.type=snappy (18.96 > 14.97, 26%)
1st: max.in.flight.requests.per.connection=1, valueBound=5000, linger.ms=0,
messageSize=1000, compression.type=gzip (5.22 > 2.89, 80%)
1st: max.in.flight.requests.per.connection=1, valueBound=5000, linger.ms=0,
messageSize=1000, compression.type=snappy (32.47 > 22.00, 47%)
1st: max.in.flight.requests.per.connection=1, valueBound=5000,
linger.ms=100000, messageSize=100, compression.type=gzip (1.74 > 1.12,
55%)
1st: max.in.flight.requests.per.connection=1, valueBound=5000,
linger.ms=100000, messageSize=100, compression.type=snappy (11.86 > 9.60,
23%)
1st: max.in.flight.requests.per.connection=1, valueBound=5000,
linger.ms=100000, messageSize=1000, compression.type=gzip (2.77 > 1.63,
69%)
1st: max.in.flight.requests.per.connection=1, valueBound=5000,
linger.ms=100000, messageSize=1000, compression.type=snappy (24.10 >
17.71, 36%)
1st: max.in.flight.requests.per.connection=1, valueBound=50000, linger.ms=0,
messageSize=100, compression.type=gzip (2.57 > 1.99, 29%)
1st: max.in.flight.requests.per.connection=1, valueBound=50000, linger.ms=0,
messageSize=100, compression.type=snappy (19.95 > 16.77, 18%)
1st: max.in.flight.requests.per.connection=1, valueBound=50000, linger.ms=0,
messageSize=1000, compression.type=gzip (5.63 > 2.98, 88%)
1st: max.in.flight.requests.per.connection=1, valueBound=50000, linger.ms=0,
messageSize=1000, compression.type=snappy (32.94 > 27.33, 20%)
1st: max.in.flight.requests.per.connection=1, valueBound=50000,
linger.ms=100000, messageSize=100, compression.type=gzip (1.73 > 1.14,
51%)
1st: max.in.flight.requests.per.connection=1, valueBound=50000,
linger.ms=100000, messageSize=100, compression.type=snappy (12.82 >
10.85, 18%)
1st: max.in.flight.requests.per.connection=1, valueBound=50000,
linger.ms=100000, messageSize=1000, compression.type=gzip (2.60 > 1.58,
64%)
1st: max.in.flight.requests.per.connection=1, valueBound=50000,
linger.ms=100000, messageSize=1000, compression.type=snappy (24.29 >
21.69, 11%)
1st: max.in.flight.requests.per.connection=5, valueBound=500, linger.ms=0,
messageSize=100, compression.type=gzip (2.01 > 1.90, 5%)
1st: max.in.flight.requests.per.connection=5, valueBound=500, linger.ms=0,
messageSize=100, compression.type=snappy (17.93 > 13.49, 32%)
1st: max.in.flight.requests.per.connection=5, valueBound=500, linger.ms=0,
messageSize=1000, compression.type=gzip (3.92 > 2.74, 43%)
1st: max.in.flight.requests.per.connection=5, valueBound=500, linger.ms=0,
messageSize=1000, compression.type=snappy (28.23 > 19.01, 48%)
1st: max.in.flight.requests.per.connection=5, valueBound=500,
linger.ms=100000, messageSize=100, compression.type=gzip (1.67 > 1.07,
56%)
1st: max.in.flight.requests.per.connection=5, valueBound=500,
linger.ms=100000, messageSize=100, compression.type=snappy (11.10 > 8.66,
28%)
1st: max.in.flight.requests.per.connection=5, valueBound=500,
linger.ms=100000, messageSize=1000, compression.type=gzip (2.48 > 1.52,
63%)
1st: max.in.flight.requests.per.connection=5, valueBound=500,
linger.ms=100000, messageSize=1000, compression.type=snappy (21.07 >
16.24, 29%)
1st: max.in.flight.requests.per.connection=5, valueBound=5000, linger.ms=0,
messageSize=100, compression.type=gzip (2.30 > 2.00, 15%)
1st: max.in.flight.requests.per.connection=5, valueBound=5000, linger.ms=0,
messageSize=100, compression.type=snappy (18.85 > 14.39, 30%)
1st: max.in.flight.requests.per.connection=5, valueBound=5000, linger.ms=0,
messageSize=1000, compression.type=gzip (5.20 > 2.90, 79%)
1st: max.in.flight.requests.per.connection=5, valueBound=5000, linger.ms=0,
messageSize=1000, compression.type=snappy (32.94 > 21.86, 50%)
1st: max.in.flight.requests.per.connection=5, valueBound=5000,
linger.ms=100000, messageSize=100, compression.type=gzip (1.54 > 1.14,
35%)
1st: max.in.flight.requests.per.connection=5, valueBound=5000,
linger.ms=100000, messageSize=100, compression.type=snappy (13.09 > 9.65,
35%)
1st: max.in.flight.requests.per.connection=5, valueBound=5000,
linger.ms=100000, messageSize=1000, compression.type=gzip (2.68 > 1.63,
64%)
1st: max.in.flight.requests.per.connection=5, valueBound=5000,
linger.ms=100000, messageSize=1000, compression.type=snappy (24.80 >
18.33, 35%)
1st: max.in.flight.requests.per.connection=5, valueBound=50000, linger.ms=0,
messageSize=100, compression.type=gzip (2.49 > 2.15, 15%)
1st: max.in.flight.requests.per.connection=5, valueBound=50000, linger.ms=0,
messageSize=100, compression.type=snappy (17.77 > 15.84, 12%)
1st: max.in.flight.requests.per.connection=5, valueBound=50000, linger.ms=0,
messageSize=1000, compression.type=gzip (5.72 > 3.00, 90%)
1st: max.in.flight.requests.per.connection=5, valueBound=50000, linger.ms=0,
messageSize=1000, compression.type=snappy (33.26 > 28.44, 16%)
1st: max.in.flight.requests.per.connection=5, valueBound=50000,
linger.ms=100000, messageSize=100, compression.type=gzip (1.73 > 1.15,
50%)
1st: max.in.flight.requests.per.connection=5, valueBound=50000,
linger.ms=100000, messageSize=100, compression.type=snappy (13.14 >
11.11, 18%)
1st: max.in.flight.requests.per.connection=5, valueBound=50000,
linger.ms=100000, messageSize=1000, compression.type=gzip (2.61 > 1.57,
66%)
1st: max.in.flight.requests.per.connection=5, valueBound=50000,
linger.ms=100000, messageSize=1000, compression.type=snappy (25.34 >
22.63, 11%)
{noformat}
> Producer's throughput lower with compressed data after KIP-31/32
> ----------------------------------------------------------------
>
> Key: KAFKA-3565
> URL: https://issues.apache.org/jira/browse/KAFKA-3565
> Project: Kafka
> Issue Type: Bug
> Reporter: Ismael Juma
> Priority: Critical
> Fix For: 0.10.0.0
>
>
> Relative offsets were introduced by KIP-31 so that the broker does not have
> to recompress data (this was previously required after offsets were
> assigned). The implicit assumption is that reducing CPU usage required by
> recompression would mean that producer throughput for compressed data would
> increase.
> However, this doesn't seem to be the case:
> {code}
> Commit: eee95228fabe1643baa016a2d49fb0a9fe2c66bd (one before KIP-31/32)
> test_id:
> 2016-04-15--012.kafkatest.tests.benchmark_test.Benchmark.test_producer_throughput.topic=topic-replication-factor-three.security_protocol=PLAINTEXT.acks=1.message_size=100.compression_type=snappy
> status: PASS
> run time: 59.030 seconds
> {"records_per_sec": 519418.343653, "mb_per_sec": 49.54}
> {code}
> Full results: https://gist.github.com/ijuma/0afada4ff51ad6a5ac2125714d748292
> {code}
> Commit: fa594c811e4e329b6e7b897bce910c6772c46c0f (KIP-31/32)
> test_id:
> 2016-04-15--013.kafkatest.tests.benchmark_test.Benchmark.test_producer_throughput.topic=topic-replication-factor-three.security_protocol=PLAINTEXT.acks=1.message_size=100.compression_type=snappy
> status: PASS
> run time: 1 minute 0.243 seconds
> {"records_per_sec": 427308.818848, "mb_per_sec": 40.75}
> {code}
> Full results: https://gist.github.com/ijuma/e49430f0548c4de5691ad47696f5c87d
> The difference for the uncompressed case is smaller (and within what one
> would expect given the additional size overhead caused by the timestamp
> field):
> {code}
> Commit: eee95228fabe1643baa016a2d49fb0a9fe2c66bd (one before KIP-31/32)
> test_id:
> 2016-04-15--010.kafkatest.tests.benchmark_test.Benchmark.test_producer_throughput.topic=topic-replication-factor-three.security_protocol=PLAINTEXT.acks=1.message_size=100
> status: PASS
> run time: 1 minute 4.176 seconds
> {"records_per_sec": 321018.17747, "mb_per_sec": 30.61}
> {code}
> Full results: https://gist.github.com/ijuma/5fec369d686751a2d84debae8f324d4f
> {code}
> Commit: fa594c811e4e329b6e7b897bce910c6772c46c0f (KIP-31/32)
> test_id:
> 2016-04-15--014.kafkatest.tests.benchmark_test.Benchmark.test_producer_throughput.topic=topic-replication-factor-three.security_protocol=PLAINTEXT.acks=1.message_size=100
> status: PASS
> run time: 1 minute 5.079 seconds
> {"records_per_sec": 291777.608696, "mb_per_sec": 27.83}
> {code}
> Full results: https://gist.github.com/ijuma/1d35bd831ff9931448b0294bd9b787ed
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
