[
https://issues.apache.org/jira/browse/FLINK-26864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17519139#comment-17519139
]
Sebastian Mattheis edited comment on FLINK-26864 at 4/7/22 8:11 PM:
--------------------------------------------------------------------
I have created a PR with a fix to recover performance values as before in
[https://github.com/apache/flink/pull/19398]
*Performance evaluation:*
# The performance before regression was assessed in *build #148*
([http://codespeed.dak8s.net:8080/job/flink-master-benchmarks-java8/148/])
based on apache/flink master commit 010f622555. ({*}This is the baseline{*}.)
# The performanc regression was observed in *build #149*
([http://codespeed.dak8s.net:8080/job/flink-master-benchmarks-java8/149/])
based on apache/flink master commit ff3336266e.
# The fix is based on current apache/master in commit 39433bafd0 and has been
evaluated in *build #74*
[http://codespeed.dak8s.net:8080/job/flink-benchmark-request/74/]. (This is the
change of PR [https://github.com/apache/flink/pull/19398])
# The fix has also been cherry-picked on top of commit ff3336266e (regression)
to verify the fix which has been evaluted in *build #73*
([http://codespeed.dak8s.net:8080/job/flink-benchmark-request/73/]).
The following table shows the performance implications of the various changes
where for the delta (in percent) the basis is build # 148.
([^jmh-result-summary.ods])
!jmh-result-summary.png!
*Conclusion:*
Build #73 shows that the fix, if cherry-picked on the commit that first showed
a performance regression in build #149, recovers the performance values from
before as of build #148. Build #74, which is the fix based on the current
master, shows that there is no other significant performance except the
following:
* org.apache.flink.benchmark.CheckpointingTimeBenchmark.checkpointSingleInput
-> -18.74%
*
org.apache.flink.benchmark.SerializationFrameworkMiniBenchmarks.serializerHeavyString
-> -14%
However, both deviations seem to be within a normal range of varying
performance values and did not drop with the changes after March 25:
*
org.apache.flink.benchmark.CheckpointingTimeBenchmark.checkpointSingleInput.UNALIGNED_1
!checkpointSingleInput_UNALIGNED_1.png!
*
org.apache.flink.benchmark.SerializationFrameworkMiniBenchmarks.serializerHeavyString
!serializerHeavyString.png!
was (Author: JIRAUSER284806):
I have created a PR with a fix to recover performance values as before in
[https://github.com/apache/flink/pull/19398]
*Performance evaluation:*
# The performance before regression was assessed in *build #148*
([http://codespeed.dak8s.net:8080/job/flink-master-benchmarks-java8/148/])
based on apache/flink master commit 010f622555. ({*}This is the baseline{*}.)
# The performanc regression was observed in *build #149*
([http://codespeed.dak8s.net:8080/job/flink-master-benchmarks-java8/149/])
based on apache/flink master commit ff3336266e.
# The fix is based on current apache/master in commit 39433bafd0 and has been
evaluated in *build #74*
[http://codespeed.dak8s.net:8080/job/flink-benchmark-request/74/]. (This is the
change of PR [https://github.com/apache/flink/pull/19398])
# The fix has also been cherry-picked on top of commit ff3336266e (regression)
to verify the fix which has been evaluted in *build #73*
([http://codespeed.dak8s.net:8080/job/flink-benchmark-request/73/]).
The following table shows the performance implications of the various changes
where for the delta (in percent) the basis is build # 148.
([^jmh-result-summary.ods])
!jmh-result-summary.png!
*Conclusion:*
Build #73 shows that the fix, if cherry-picked on the commit that first showed
a performance regression in build #149, recovers the performance values from
before in build #48. Build #74, which is the fix applied to current master,
shows that there is no other significant performance regression in comparison
to before the regression in build #73 except the following
* org.apache.flink.benchmark.CheckpointingTimeBenchmark.checkpointSingleInput
-> -18.74%
*
org.apache.flink.benchmark.SerializationFrameworkMiniBenchmarks.serializerHeavyString
-> -14%
However, both deviations seem to be within a normal range of varying
performance values and did not drop with the changes after March 25:
*
org.apache.flink.benchmark.CheckpointingTimeBenchmark.checkpointSingleInput.UNALIGNED_1
!checkpointSingleInput_UNALIGNED_1.png!
*
org.apache.flink.benchmark.SerializationFrameworkMiniBenchmarks.serializerHeavyString
!serializerHeavyString.png!
> Performance regression on 25.03.2022
> ------------------------------------
>
> Key: FLINK-26864
> URL: https://issues.apache.org/jira/browse/FLINK-26864
> Project: Flink
> Issue Type: Bug
> Components: Benchmarks
> Affects Versions: 1.16.0
> Reporter: Piotr Nowojski
> Assignee: Sebastian Mattheis
> Priority: Blocker
> Labels: pull-request-available
> Attachments: checkpointSingleInput_UNALIGNED_1.png,
> jmh-result-summary.ods, jmh-result-summary.png, serializerHeavyString.png
>
>
> http://codespeed.dak8s.net:8000/timeline/#/?exe=1&ben=arrayKeyBy&extr=on&quarts=on&equid=off&env=2&revs=200
> http://codespeed.dak8s.net:8000/timeline/#/?exe=1&ben=remoteFilePartition&extr=on&quarts=on&equid=off&env=2&revs=200
> http://codespeed.dak8s.net:8000/timeline/#/?exe=1&ben=remoteSortPartition&extr=on&quarts=on&equid=off&env=2&revs=200
> http://codespeed.dak8s.net:8000/timeline/#/?exe=1&ben=tupleKeyBy&extr=on&quarts=on&equid=off&env=2&revs=200
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
