[
https://issues.apache.org/jira/browse/GEODE-8950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293270#comment-17293270
]
Donal Evans commented on GEODE-8950:
------------------------------------
I ran the benchmark 10 times each against support/1.13 and support/1.14. While
there was definite instability in the results, after some crude analysis, it
appears that support/1.14 does show a statistically significant performance
degradation compared to support/1.13 of about 4.6% +/- 1%. Link to the
spreadsheet with data and charts:
https://docs.google.com/spreadsheets/d/1jTPiWMF1L1ItrQBMokXqhD4veMIryhiNSLxw36XegqQ/edit?usp=sharing
Increasing the stability of the test would make it easier to say with
confidence when the degradation was introduced, since fewer repeat runs would
be required to get reliable data.
> Benchmark failure - P2pPartitionedPutLongBenchmark
> --------------------------------------------------
>
> Key: GEODE-8950
> URL: https://issues.apache.org/jira/browse/GEODE-8950
> Project: Geode
> Issue Type: Bug
> Components: benchmarks
> Affects Versions: 1.15.0
> Reporter: Donal Evans
> Assignee: Hale Bales
> Priority: Major
>
> Multiple benchmark failures due to P2pPartitionedPutLongBenchmark have been
> seen recently.
> This run saw 3 out of the 5 repeats fail due to flagged degradations in
> P2pPartitionedPutLongBenchmark:
> [https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/Benchmark_base/builds/16|https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/Benchmark_base/builds/16#L601ed52d:5552]
> This run saw 1 out of the 5 repeats fail due to flagged degradations in
> P2pPartitionedPutLongBenchmark:
> [https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/Benchmark_base/builds/20]
> This run saw 4 out of the 5 repeats fail due to flagged degradations in
> P2pPartitionedPutLongBenchmark:
> [https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/Benchmark_base/builds/27]
> In all the above benchmarks, the other failed runs were due to EOFExceptions
> rather than flagged degradations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
