[
https://issues.apache.org/jira/browse/FLINK-27571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652744#comment-17652744
]
Yanfei Lei commented on FLINK-27571:
------------------------------------
Hi [~roman] , could you please help take a look
[pull/63|https://github.com/apache/flink-benchmarks/pull/63] in your free time?
thanks :)
Thanks to [pull/55|https://github.com/apache/flink-benchmarks/pull/55], the
information for future new benchmarks will be correctly stored.
But for the old existing benchmarks, their information cannot be changed only
through ‘http://url/result/add/json/’. Because *codespeed* stores the basic
information of benchmark when a result is added {*}for the first time{*}, and
will not update it later( related code is
[here|https://github.com/tobami/codespeed/blob/263860bc298fd970c8466b3161de386582e4f801/codespeed/results.py#L61]).
So I logged into the Jenkins master and modified the `codespeed_benchmark`
table in SQLite, currently "less is better" can be displayed normally on the
[timeline|http://codespeed.dak8s.net:8000/timeline/#/?ben=schedulingDownstreamTasks.BATCH&extr=on&quarts=on&equid=off&env=2&revs=200&exe=1,3,5,6,8,9].
> Recognize "less is better" benchmarks in regression detection script
> --------------------------------------------------------------------
>
> Key: FLINK-27571
> URL: https://issues.apache.org/jira/browse/FLINK-27571
> Project: Flink
> Issue Type: Bug
> Components: Benchmarks
> Affects Versions: 1.16.0
> Reporter: Roman Khachatryan
> Assignee: Yanfei Lei
> Priority: Major
> Labels: pull-request-available
> Attachments: Screenshot_2022-05-09_10-33-11.png,
> image-2022-12-29-14-39-59-976.png
>
>
> Example benchmark:
> [http://codespeed.dak8s.net:8000/timeline/#/?exe=5&ben=schedulingDownstreamTasks.BATCH&extr=on&quarts=on&equid=off&env=2&revs=200]
>
> [Proposed
> solution|https://issues.apache.org/jira/browse/FLINK-27555?focusedCommentId=17534423&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17534423]:
> {quote}
> I think #2 is the correct way.
> Maybe we can modify the save_jmh_result.py to correctly set the 'units' and
> the 'lessisbetter' fields of benchmark results. The 'units' is already
> contained in the jmh result and the 'lessisbetter' can be derived from the
> mode(false if it is 'thrpt' mode, otherwise true). An example of the jmh
> result format can be found at https://i.stack.imgur.com/vB3fV.png.
> This can fix the web UI as well as the REST result, and then the
> regression_report.py will be able to identify which benchmarks are "less is
> better" and treat them differently.
> {quote}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
