Donal Evans created GEODE-10091:
-----------------------------------
Summary: Benchmark instability in RedisZaddAndZremBenchmark
Key: GEODE-10091
URL: https://issues.apache.org/jira/browse/GEODE-10091
Project: Geode
Issue Type: Bug
Components: benchmarks, redis
Affects Versions: 1.15.0
Reporter: Donal Evans
{noformat}
This is ITERATION 1 of benchmarking against baseline.
RedisGetBenchmark avg ops/sec
Baseline: 428358.55 Test: 446241.49 Difference: +4.2%
avg latency
Baseline: 1678809.55 Test: 1611045.34 Difference: -4.0%
RedisHgetBenchmark avg ops/sec
Baseline: 438535.25 Test: 441078.97 Difference: +0.6%
avg latency
Baseline: 1638968.92 Test: 1630999.16 Difference: -0.5%
RedisHsetAndHgetBenchmark avg ops/sec
Baseline: 191331.54 Test: 186806.37 Difference: -2.4%
avg latency
Baseline: 3759694.09 Test: 3850396.04 Difference: +2.4%
RedisHsetBenchmark avg ops/sec
Baseline: 308079.41 Test: 303715.78 Difference: -1.4%
avg latency
Baseline: 2332102.91 Test: 2367639.93 Difference: +1.5%
RedisSetBenchmark avg ops/sec
Baseline: 318147.74 Test: 326996.22 Difference: +2.8%
avg latency
Baseline: 2263687.57 Test: 2198181.68 Difference: -2.9%
RedisWeightedHsetAndHgetBenchmark avg ops/sec
Baseline: 390383.79 Test: 385565.48 Difference: -1.2%
avg latency
Baseline: 1842561.96 Test: 1864716.09 Difference: +1.2%
RedisWeightedZaddAndZrangeBenchmark avg ops/sec
Baseline: 375078.82 Test: 350886.03 Difference: -6.5%
avg latency
Baseline: 1917476.15 Test: 2049452.87 Difference: +6.9%
RedisZaddAndZremBenchmark avg ops/sec
Baseline: 124763.60 Test: 118600.72 Difference: -4.9%
avg latency
Baseline: 5766048.34 Test: 6065491.13 Difference: +5.2%
RedisZaddBenchmark avg ops/sec
Baseline: 432494.29 Test: 438594.53 Difference: +1.4%
avg latency
Baseline: 1662639.09 Test: 1639906.18 Difference: -1.4%
RedisZrangeBenchmark avg ops/sec
Baseline: 295523.77 Test: 340381.87 Difference: +15.2%
avg latency
Baseline: 2433250.54 Test: 2112961.66 Difference: -13.2%
RedisZrangeByScoreBenchmark avg ops/sec
Baseline: 285836.33 Test: 356142.91 Difference: +24.6%
avg latency
Baseline: 2515708.23 Test: 2019249.68 Difference: -19.7%
This is ITERATION 2 of benchmarking against baseline.
RedisWeightedZaddAndZrangeBenchmark avg ops/sec
Baseline: 358570.33 Test: 348012.02 Difference: -2.9%
avg latency
Baseline: 2005804.05 Test: 2066535.24 Difference: +3.0%
RedisZaddAndZremBenchmark avg ops/sec
Baseline: 122900.38 Test: 115226.22 Difference: -6.2%
avg latency
Baseline: 5858357.13 Test: 6243018.38 Difference: +6.6%
This is ITERATION 3 of benchmarking against baseline.
RedisZaddAndZremBenchmark avg ops/sec
Baseline: 125677.36 Test: 118359.94 Difference: -5.8%
avg latency
Baseline: 5721859.26 Test: 6082944.28 Difference: +6.3%
This is ITERATION 4 of benchmarking against baseline.
RedisZaddAndZremBenchmark avg ops/sec
Baseline: 122099.00 Test: 115864.88 Difference: -5.1%
avg latency
Baseline: 5893264.03 Test: 6203439.89 Difference: +5.3%
This is ITERATION 5 of benchmarking against baseline.
RedisZaddAndZremBenchmark avg ops/sec
Baseline: 122234.53 Test: 116038.15 Difference: -5.1%
avg latency
Baseline: 5883499.99 Test: 6202493.46 Difference: +5.4%{noformat}
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
This benchmark run is the result of comparing
b7297023c309210e70f3792980e75d10c54e7421 with baseline
geode-for-redis-benchmark-baseline
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Commit Message
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
commit b7297023c309210e70f3792980e75d10c54e7421
Author: Jens Deppe <[email protected]>
Date: Fri Feb 25 13:41:18 2022 -0800
GEODE-10083: Fix RedisProxy to correctly process MOVED response (#7394)
- When a MOVED response is received, the already queued processor needs
to be discarded otherwise subsequent requests may possibly be
corrupted.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Benchmark Results URI
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
http://files.apachegeode-ci.info/benchmarks/apache-develop-main/b7297023c309210e70f3792980e75d10c54e7421/benchmarks-apache-develop-main-benchmark-radish-171-31258406-radish.tgz
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
The above benchmark failed on all 5 iterations of this run, but passed after
one repeat when the concourse job was rerun against the same commit:
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
This benchmark run is the result of comparing
b7297023c309210e70f3792980e75d10c54e7421 with baseline
geode-for-redis-benchmark-baseline
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Commit Message
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
commit b7297023c309210e70f3792980e75d10c54e7421
Author: Jens Deppe <[email protected]>
Date: Fri Feb 25 13:41:18 2022 -0800
GEODE-10083: Fix RedisProxy to correctly process MOVED response (#7394)
- When a MOVED response is received, the already queued processor needs
to be discarded otherwise subsequent requests may possibly be
corrupted.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Benchmark Results URI
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
http://files.apachegeode-ci.info/benchmarks/apache-develop-main/b7297023c309210e70f3792980e75d10c54e7421/benchmarks-apache-develop-main-benchmark-radish-172-31385041-radish.tgz
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
