(rocketmq-test-tool) branch main updated: Integrate performance benchmarking tool into CI pipeline (#7)
This is an automated email from the ASF dual-hosted git repository. jinrongtong pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/rocketmq-test-tool.git The following commit(s) were added to refs/heads/main by this push: new ce372e5 Integrate performance benchmarking tool into CI pipeline (#7) ce372e5 is described below commit ce372e5f3906ca1891e4918b05be14608eae608e Author: 小陈 AuthorDate: Fri Sep 27 15:11:12 2024 +0800 Integrate performance benchmarking tool into CI pipeline (#7) * Wrap the performance benchmark process into an action * Merge consumer and producer scripts into a single script * Remove redundant parameters * Add logic to check if CI passes * Reduce code duplication * Improve CI pass condition logic for more accurate validation * Revert "Reduce code duplication" This reverts commit eadddf6fc7c9eff4a92d8634792dd0b18956f5d9. * Fix bug * Simplified the validation logic * Update * Move data processing scripts into the image * Add support for setting thresholds via parameters * Use the same K8s secret configuration as the main repository --- benchmark-runner/Dockerfile | 24 benchmark-runner/action.yml | 104 +++ benchmark-runner/entry.sh | 300 3 files changed, 428 insertions(+) diff --git a/benchmark-runner/Dockerfile b/benchmark-runner/Dockerfile new file mode 100644 index 000..070f1fc --- /dev/null +++ b/benchmark-runner/Dockerfile @@ -0,0 +1,24 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# Container image that runs your code +FROM registry.cn-guangzhou.aliyuncs.com/cc-aliyun/ubuntu-python:v1.2.0 + +RUN pip install numpy pandas matplotlib + +COPY entry.sh /entry.sh + +ENTRYPOINT ["/entry.sh"] \ No newline at end of file diff --git a/benchmark-runner/action.yml b/benchmark-runner/action.yml new file mode 100644 index 000..2b379a4 --- /dev/null +++ b/benchmark-runner/action.yml @@ -0,0 +1,104 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# action.yml +name: 'RocketMQ Benchmark Helper' +description: '' +inputs: + action: # id of input +description: 'action' +required: true +default: '' + test-version: # id of input +description: 'test version' +required: true +default: '' + ask-config: # id of input +description: 'ask config' +required: true +default: '' + docker-repo-username: # id of input +description: 'docker repo username' +required: false +default: '' + docker-repo-password: # id of input +description: 'docker repo password' +required: false +default: '' + chart-git: # id of input +description: 'chart git' +required: false +default: '' + chart-branch: # id of input +description: 'chart branch' +required: false +default: '' + chart-path: # id of input +description: 'chart path' +required: false +default: './' + job-id: # id of input +description: 'job id' +required: true +default: '' + helm-values: # id of input +description: 'helm values' +required: true +default: '' + test-time: +description: 'test time' +required: false +default: '1800' # default 30 minutes + min-send-tps-threshold: +description: 'min-send-tps-threshold' +required: false +default: '13000' # default threshold : only one broker as m
Re: [PR] Integrate performance benchmarking tool into CI pipeline [rocketmq-test-tool]
RongtongJin merged PR #7: URL: https://github.com/apache/rocketmq-test-tool/pull/7 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user 3424672656 (triggered by github-actions[bot]). Head commit for run: c7abdd38cbb12fa53700944e3ff04cbbcfed542c / wanghuaiyuan Optimize code style Report URL: https://github.com/apache/rocketmq/actions/runs/11066156271 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" is working again!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has succeeded. Run started by GitHub user chi3316 (triggered by chi3316). Head commit for run: 319426d2e16b57d7e422e93a1fce828f0e29ef53 / Jamie Update test tool Report URL: https://github.com/apache/rocketmq/actions/runs/11066698714 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Maven" failed!
The GitHub Actions job "Build and Run Tests by Maven" on rocketmq.git has failed. Run started by GitHub user 3424672656 (triggered by 3424672656). Head commit for run: c7abdd38cbb12fa53700944e3ff04cbbcfed542c / wanghuaiyuan Optimize code style Report URL: https://github.com/apache/rocketmq/actions/runs/11066156274 With regards, GitHub Actions via GitBox
Re: [PR] [ISUUE #8762] Fix dLedger lock granularity issue causing LMQ message loss [rocketmq]
codecov-commenter commented on PR #8763: URL: https://github.com/apache/rocketmq/pull/8763#issuecomment-2378733243 ## [Codecov](https://app.codecov.io/gh/apache/rocketmq/pull/8763?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report Attention: Patch coverage is `12.5%` with `35 lines` in your changes missing coverage. Please review. > Project coverage is 47.30%. Comparing base [(`daf3d1a`)](https://app.codecov.io/gh/apache/rocketmq/commit/daf3d1a666f3a9d1a9e314a124468e1d05f84ba2?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`dd99891`)](https://app.codecov.io/gh/apache/rocketmq/commit/dd99891affc73f1237f53b5e15992b4911f260ac?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache). | [Files with missing lines](https://app.codecov.io/gh/apache/rocketmq/pull/8763?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines | |---|---|---| | [...pache/rocketmq/store/queue/MultiDispatchUtils.java](https://app.codecov.io/gh/apache/rocketmq/pull/8763?src=pr&el=tree&filepath=store%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fstore%2Fqueue%2FMultiDispatchUtils.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL3F1ZXVlL011bHRpRGlzcGF0Y2hVdGlscy5qYXZh) | 3.57% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/8763?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | | [...pache/rocketmq/store/dledger/DLedgerCommitLog.java](https://app.codecov.io/gh/apache/rocketmq/pull/8763?src=pr&el=tree&filepath=store%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fstore%2Fdledger%2FDLedgerCommitLog.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL2RsZWRnZXIvRExlZGdlckNvbW1pdExvZy5qYXZh) | 33.33% | [6 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/8763?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop#8763 +/- ## = - Coverage 47.51% 47.30% -0.21% + Complexity 1158911548 -41 = Files 1282 1282 Lines 8984889888 +40 Branches 1155711568 +11 = - Hits 4268942522 -167 - Misses 4193242118 +186 - Partials5227 5248 +21 ``` | [Flag](https://app.codecov.io/gh/apache/rocketmq/pull/8763/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | |---|---|---| | [](https://app.codecov.io/gh/apache/rocketmq/pull/8763/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `47.30% <12.50%> (-0.21%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more. [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/rocketmq/pull/8763?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user 3424672656 (triggered by github-actions[bot]). Head commit for run: c7abdd38cbb12fa53700944e3ff04cbbcfed542c / wanghuaiyuan Optimize code style Report URL: https://github.com/apache/rocketmq/actions/runs/11066156271 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Maven" failed!
The GitHub Actions job "Build and Run Tests by Maven" on rocketmq.git has failed. Run started by GitHub user 3424672656 (triggered by github-actions[bot]). Head commit for run: c7abdd38cbb12fa53700944e3ff04cbbcfed542c / wanghuaiyuan Optimize code style Report URL: https://github.com/apache/rocketmq/actions/runs/11066156274 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user yuz10 (triggered by yuz10). Head commit for run: c7597fbe99684085a188af5b8461dae6a4ed546f / yuz10 <845238...@qq.com> [ISSUE #8765] fix low performance of delay message when enable rocksdb consume queue Report URL: https://github.com/apache/rocketmq/actions/runs/11067843354 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Maven" is working again!
The GitHub Actions job "Build and Run Tests by Maven" on rocketmq.git has succeeded. Run started by GitHub user bxfjb (triggered by bxfjb). Head commit for run: dd99891affc73f1237f53b5e15992b4911f260ac / zhaoyuhan fix unittest Report URL: https://github.com/apache/rocketmq/actions/runs/11066809789 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user yuz10 (triggered by yuz10). Head commit for run: 20faf8ef7736e392e3793f03faca29490718e7dc / yuz10 <845238...@qq.com> clean code Report URL: https://github.com/apache/rocketmq/actions/runs/11067871305 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user yuz10 (triggered by github-actions[bot]). Head commit for run: c7597fbe99684085a188af5b8461dae6a4ed546f / yuz10 <845238...@qq.com> [ISSUE #8765] fix low performance of delay message when enable rocksdb consume queue Report URL: https://github.com/apache/rocketmq/actions/runs/11067843354 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user yuz10 (triggered by github-actions[bot]). Head commit for run: c7597fbe99684085a188af5b8461dae6a4ed546f / yuz10 <845238...@qq.com> [ISSUE #8765] fix low performance of delay message when enable rocksdb consume queue Report URL: https://github.com/apache/rocketmq/actions/runs/11067843354 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Run Integration Tests" is working again!
The GitHub Actions job "Run Integration Tests" on rocketmq.git has succeeded. Run started by GitHub user yuz10 (triggered by yuz10). Head commit for run: c7597fbe99684085a188af5b8461dae6a4ed546f / yuz10 <845238...@qq.com> [ISSUE #8765] fix low performance of delay message when enable rocksdb consume queue Report URL: https://github.com/apache/rocketmq/actions/runs/11067843328 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user 3424672656 (triggered by 3424672656). Head commit for run: 56606d33227e1b44efbdb134b190c474e44c1f75 / wanghuaiyuan Optimize code style Report URL: https://github.com/apache/rocketmq/actions/runs/11068181221 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "E2E test for pull request" failed!
The GitHub Actions job "E2E test for pull request" on rocketmq.git has failed. Run started by GitHub user 3424672656 (triggered by 3424672656). Head commit for run: daf3d1a666f3a9d1a9e314a124468e1d05f84ba2 / lizhimins <707364...@qq.com> [maven-release-plugin] prepare for next development iteration (#8750) Report URL: https://github.com/apache/rocketmq/actions/runs/11066183183 With regards, GitHub Actions via GitBox
Re: [PR] Update performance benchmark test tool [rocketmq]
codecov-commenter commented on PR #8761: URL: https://github.com/apache/rocketmq/pull/8761#issuecomment-2378717354 ## [Codecov](https://app.codecov.io/gh/apache/rocketmq/pull/8761?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report All modified and coverable lines are covered by tests :white_check_mark: > Please [upload](https://docs.codecov.com/docs/codecov-uploader) report for BASE (`cicd-perf-benchmark@4ef6b68`). [Learn more](https://docs.codecov.io/docs/error-reference?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#section-missing-base-commit) about missing BASE report. Additional details and impacted files ```diff @@ Coverage Diff @@ ## cicd-perf-benchmark#8761 +/- ## == Coverage ? 47.32% Complexity ?11541 == Files ? 1282 Lines ?89848 Branches ?11557 == Hits ?42525 Misses ?42077 Partials ? 5246 ``` | [Flag](https://app.codecov.io/gh/apache/rocketmq/pull/8761/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | |---|---|---| | [](https://app.codecov.io/gh/apache/rocketmq/pull/8761/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `47.32% <ø> (?)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more. [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/rocketmq/pull/8761?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user bxfjb (triggered by github-actions[bot]). Head commit for run: dd99891affc73f1237f53b5e15992b4911f260ac / zhaoyuhan fix unittest Report URL: https://github.com/apache/rocketmq/actions/runs/11066809698 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user 3424672656 (triggered by 3424672656). Head commit for run: c7abdd38cbb12fa53700944e3ff04cbbcfed542c / wanghuaiyuan Optimize code style Report URL: https://github.com/apache/rocketmq/actions/runs/11066156271 With regards, GitHub Actions via GitBox
[PR] [ISSUE #8765] fix low performance of delay message when enable rocksdb consume queue [rocketmq]
yuz10 opened a new pull request, #8766: URL: https://github.com/apache/rocketmq/pull/8766 ### Which Issue(s) This PR Fixes Fixes #8765 ### Brief Description ### How Did You Test This Change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user bxfjb (triggered by github-actions[bot]). Head commit for run: dd99891affc73f1237f53b5e15992b4911f260ac / zhaoyuhan fix unittest Report URL: https://github.com/apache/rocketmq/actions/runs/11066809698 With regards, GitHub Actions via GitBox
Re: [PR] #8509 lmq offset sync master to slave [rocketmq]
domyway commented on PR #8709: URL: https://github.com/apache/rocketmq/pull/8709#issuecomment-2378645691 > CR need How is it going -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Bug] DLedger lock granularity issue causing LMQ message loss [rocketmq]
bxfjb opened a new issue, #8762: URL: https://github.com/apache/rocketmq/issues/8762 ### Before Creating the Bug Report - [X] I found a bug, not just asking a question, which should be created in [GitHub Discussions](https://github.com/apache/rocketmq/discussions). - [X] I have searched the [GitHub Issues](https://github.com/apache/rocketmq/issues) and [GitHub Discussions](https://github.com/apache/rocketmq/discussions) of this repository and believe that this is not a duplicate. - [X] I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ. ### Runtime platform environment any ### RocketMQ version develop ### JDK Version 1.8 ### Describe the Bug Issue #7511 reported the bug but #7525 didn't fix it in `DLedgerCommitlog`. ### Steps to Reproduce See #7511. ### What Did You Expect to See? See #7511. ### What Did You See Instead? See #7511. ### Additional Context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] grpc客户端不适配从代理主模式（slaveActingMaster）模式 [rocketmq]
jaylu163 commented on issue #8716: URL: https://github.com/apache/rocketmq/issues/8716#issuecomment-2378654177 你好，开启proxy后的集群可以正常发送消息不。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GH] (rocketmq): Workflow run "E2E test for pull request" failed!
The GitHub Actions job "E2E test for pull request" on rocketmq.git has failed. Run started by GitHub user bxfjb (triggered by bxfjb). Head commit for run: daf3d1a666f3a9d1a9e314a124468e1d05f84ba2 / lizhimins <707364...@qq.com> [maven-release-plugin] prepare for next development iteration (#8750) Report URL: https://github.com/apache/rocketmq/actions/runs/11066874677 With regards, GitHub Actions via GitBox
[I] [Bug] Bad performace of delay message in rocksdb consumequeue [rocketmq]
yuz10 opened a new issue, #8765:
URL: https://github.com/apache/rocketmq/issues/8765
### Before Creating the Bug Report
- [X] I found a bug, not just asking a question, which should be created in
[GitHub Discussions](https://github.com/apache/rocketmq/discussions).
- [X] I have searched the [GitHub
Issues](https://github.com/apache/rocketmq/issues) and [GitHub
Discussions](https://github.com/apache/rocketmq/discussions) of this
repository and believe that this is not a duplicate.
- [X] I have confirmed that this bug belongs to the current repository, not
other repositories of RocketMQ.
### Runtime platform environment
Ubuntu 24.04
### RocketMQ version
5.3.1
### JDK Version
1.8
### Describe the Bug
Bad performace of delay message in rocksdb consumequeue
### Steps to Reproduce
1. set `storeType=defaultRocksDB` in broker.conf
2. send delay messages with delaylever=2 using `cd benchmark;sh producer.sh
-n localhost:9876 -t testTopic -d true -e 2`
3. watch delay message count using `mqadmin topicstatus -n localhost:9876
-t SCHEDULE_TOPIC_`
4. watch delay message deliver count using `cat
${storeRootPath}/config/delayOffset.json`
### What Did You Expect to See?
delay message are deliverd quick.
### What Did You See Instead?
delay message are deliverd slow.
### Additional Context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
(rocketmq) branch cicd-perf-benchmark updated: Update test tool (#8761)
This is an automated email from the ASF dual-hosted git repository. jinrongtong pushed a commit to branch cicd-perf-benchmark in repository https://gitbox.apache.org/repos/asf/rocketmq.git The following commit(s) were added to refs/heads/cicd-perf-benchmark by this push: new 8633b2ee2c Update test tool (#8761) 8633b2ee2c is described below commit 8633b2ee2c152d073408c03febdcbe01f7d3e0f0 Author: 小陈 AuthorDate: Fri Sep 27 17:46:39 2024 +0800 Update test tool (#8761) --- .github/workflows/push-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/push-ci.yml b/.github/workflows/push-ci.yml index 76d2c8d66d..6c6e523748 100644 --- a/.github/workflows/push-ci.yml +++ b/.github/workflows/push-ci.yml @@ -257,7 +257,7 @@ jobs: needs: [ list-version, deploy ] timeout-minutes: 60 steps: - - uses: chi3316/rocketmq-test-tool/benchmark-runner@e670a74fe1f0ccbe1d28081dbf6c0fc35372b885 + - uses: apache/rocketmq-test-tool/benchmark-runner@ce372e5f3906ca1891e4918b05be14608eae608e name: Performance benchmark with: action: "performance-benchmark"
Re: [PR] Update performance benchmark test tool [rocketmq]
RongtongJin merged PR #8761: URL: https://github.com/apache/rocketmq/pull/8761 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user yuz10 (triggered by github-actions[bot]). Head commit for run: 20faf8ef7736e392e3793f03faca29490718e7dc / yuz10 <845238...@qq.com> clean code Report URL: https://github.com/apache/rocketmq/actions/runs/11067871305 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user 3424672656 (triggered by github-actions[bot]). Head commit for run: 56606d33227e1b44efbdb134b190c474e44c1f75 / wanghuaiyuan Optimize code style Report URL: https://github.com/apache/rocketmq/actions/runs/11068181221 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user 3424672656 (triggered by github-actions[bot]). Head commit for run: 56606d33227e1b44efbdb134b190c474e44c1f75 / wanghuaiyuan Optimize code style Report URL: https://github.com/apache/rocketmq/actions/runs/11068181221 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "PUSH-CI" failed!
The GitHub Actions job "PUSH-CI" on rocketmq.git has failed. Run started by GitHub user RongtongJin (triggered by RongtongJin). Head commit for run: 8633b2ee2c152d073408c03febdcbe01f7d3e0f0 / 小陈 Update test tool (#8761) Report URL: https://github.com/apache/rocketmq/actions/runs/11068399606 With regards, GitHub Actions via GitBox
[PR] [ISUUE #8762] Fix dLedger lock granularity issue causing LMQ message loss [rocketmq]
bxfjb opened a new pull request, #8763: URL: https://github.com/apache/rocketmq/pull/8763 ### Which Issue(s) This PR Fixes Fixes #8762 ### Brief Description ### How Did You Test This Change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Enhancement] Implementation of kvcq message heap estimation [rocketmq]
LetLetMe opened a new issue, #8764: URL: https://github.com/apache/rocketmq/issues/8764 ### Before Creating the Enhancement Request - [X] I have confirmed that this should be classified as an enhancement rather than a bug/feature. ### Summary kcCq implements the capability of message accumulation estimation. kcCq 实现消息的堆积估算能力 ### Motivation kcCq implements the capability of message accumulation estimation. kcCq 实现消息的堆积估算能力 ### Describe the Solution You'd Like nothing ### Describe Alternatives You've Considered nothing ### Additional Context nothing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GH] (rocketmq): Workflow run "E2E test for pull request" failed!
The GitHub Actions job "E2E test for pull request" on rocketmq.git has failed. Run started by GitHub user 3424672656 (triggered by 3424672656). Head commit for run: daf3d1a666f3a9d1a9e314a124468e1d05f84ba2 / lizhimins <707364...@qq.com> [maven-release-plugin] prepare for next development iteration (#8750) Report URL: https://github.com/apache/rocketmq/actions/runs/11068219210 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Maven" failed!
The GitHub Actions job "Build and Run Tests by Maven" on rocketmq.git has failed. Run started by GitHub user 3424672656 (triggered by github-actions[bot]). Head commit for run: c7abdd38cbb12fa53700944e3ff04cbbcfed542c / wanghuaiyuan Optimize code style Report URL: https://github.com/apache/rocketmq/actions/runs/11066156274 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Maven" failed!
The GitHub Actions job "Build and Run Tests by Maven" on rocketmq.git has failed. Run started by GitHub user chi3316 (triggered by chi3316). Head commit for run: 319426d2e16b57d7e422e93a1fce828f0e29ef53 / Jamie Update test tool Report URL: https://github.com/apache/rocketmq/actions/runs/11066698707 With regards, GitHub Actions via GitBox
[PR] Update performance benchmark test tool [rocketmq]
chi3316 opened a new pull request, #8761: URL: https://github.com/apache/rocketmq/pull/8761 ### Which Issue(s) This PR Fixes ### Brief Description Update test tool ### How Did You Test This Change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user bxfjb (triggered by bxfjb). Head commit for run: dd99891affc73f1237f53b5e15992b4911f260ac / zhaoyuhan fix unittest Report URL: https://github.com/apache/rocketmq/actions/runs/11066809698 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "E2E test for pull request" failed!
The GitHub Actions job "E2E test for pull request" on rocketmq.git has failed. Run started by GitHub user chi3316 (triggered by chi3316). Head commit for run: daf3d1a666f3a9d1a9e314a124468e1d05f84ba2 / lizhimins <707364...@qq.com> [maven-release-plugin] prepare for next development iteration (#8750) Report URL: https://github.com/apache/rocketmq/actions/runs/11066724399 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Run Integration Tests" failed!
The GitHub Actions job "Run Integration Tests" on rocketmq.git has failed. Run started by GitHub user bxfjb (triggered by bxfjb). Head commit for run: dd99891affc73f1237f53b5e15992b4911f260ac / zhaoyuhan fix unittest Report URL: https://github.com/apache/rocketmq/actions/runs/11066809705 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user yuz10 (triggered by github-actions[bot]). Head commit for run: 20faf8ef7736e392e3793f03faca29490718e7dc / yuz10 <845238...@qq.com> clean code Report URL: https://github.com/apache/rocketmq/actions/runs/11067871305 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "E2E test for pull request" failed!
The GitHub Actions job "E2E test for pull request" on rocketmq.git has failed. Run started by GitHub user yuz10 (triggered by yuz10). Head commit for run: daf3d1a666f3a9d1a9e314a124468e1d05f84ba2 / lizhimins <707364...@qq.com> [maven-release-plugin] prepare for next development iteration (#8750) Report URL: https://github.com/apache/rocketmq/actions/runs/11067875089 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "E2E test for pull request" failed!
The GitHub Actions job "E2E test for pull request" on rocketmq.git has failed. Run started by GitHub user yuz10 (triggered by yuz10). Head commit for run: daf3d1a666f3a9d1a9e314a124468e1d05f84ba2 / lizhimins <707364...@qq.com> [maven-release-plugin] prepare for next development iteration (#8750) Report URL: https://github.com/apache/rocketmq/actions/runs/11067904861 With regards, GitHub Actions via GitBox
[GH] (rocketmq-clients): Workflow run "Build" failed!
The GitHub Actions job "Build" on rocketmq-clients.git has failed. Run started by GitHub user qianye1001 (triggered by lizhimins). Head commit for run: c790665cc583a38e48f84eed84c30002968928b0 / qianye add retry Report URL: https://github.com/apache/rocketmq-clients/actions/runs/10610920821 With regards, GitHub Actions via GitBox
Re: [PR] [ISSUE #8765] fix low performance of delay message when enable rocksdb consume queue [rocketmq]
codecov-commenter commented on PR #8766: URL: https://github.com/apache/rocketmq/pull/8766#issuecomment-2378871652 ## [Codecov](https://app.codecov.io/gh/apache/rocketmq/pull/8766?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report Attention: Patch coverage is `52.5%` with `19 lines` in your changes missing coverage. Please review. > Project coverage is 47.35%. Comparing base [(`daf3d1a`)](https://app.codecov.io/gh/apache/rocketmq/commit/daf3d1a666f3a9d1a9e314a124468e1d05f84ba2?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`20faf8e`)](https://app.codecov.io/gh/apache/rocketmq/commit/20faf8ef7736e392e3793f03faca29490718e7dc?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache). | [Files with missing lines](https://app.codecov.io/gh/apache/rocketmq/pull/8766?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines | |---|---|---| | [...ache/rocketmq/store/queue/RocksDBConsumeQueue.java](https://app.codecov.io/gh/apache/rocketmq/pull/8766?src=pr&el=tree&filepath=store%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fstore%2Fqueue%2FRocksDBConsumeQueue.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL3F1ZXVlL1JvY2tzREJDb25zdW1lUXVldWUuamF2YQ==) | 52.50% | [13 Missing and 6 partials :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/8766?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop#8766 +/- ## = - Coverage 47.51% 47.35% -0.17% + Complexity 1159111548 -43 = Files 1282 1282 Lines 8984889884 +36 Branches 1155711565 +8 = - Hits 4269242565 -127 - Misses 4193142062 +131 - Partials5225 5257 +32 ``` | [Flag](https://app.codecov.io/gh/apache/rocketmq/pull/8766/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | |---|---|---| | [](https://app.codecov.io/gh/apache/rocketmq/pull/8766/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `47.35% <52.50%> (-0.17%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more. [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/rocketmq/pull/8766?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] UtilAllTest#testCalculateFileSizeInPath can't delete testfile [rocketmq]
github-actions[bot] commented on issue #7406: URL: https://github.com/apache/rocketmq/issues/7406#issuecomment-2380305146 This issue is stale because it has been open for 365 days with no activity. It will be closed in 3 days if no further activity occurs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Bug] rocketmq 2m2s集群环境下，客户端使用golang/v5.1.0-rc.1，只能消费其中一个broker的消息，另一个broker上的消息不能消费 [rocketmq]
heshiyingx opened a new issue, #8767: URL: https://github.com/apache/rocketmq/issues/8767 ### Before Creating the Bug Report - [X] I found a bug, not just asking a question, which should be created in [GitHub Discussions](https://github.com/apache/rocketmq/discussions). - [X] I have searched the [GitHub Issues](https://github.com/apache/rocketmq/issues) and [GitHub Discussions](https://github.com/apache/rocketmq/discussions) of this repository and believe that this is not a duplicate. - [X] I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ. ### Runtime platform environment 运行环境：macOS14.7(intel)，docker 使用镜像：apache/rocketmq:5.3.0 docker_compose整体文件包括配置如下： [docker_compose.zip](https://github.com/user-attachments/files/17171947/docker_compose.zip) ### RocketMQ version 使用的是官方镜像，apache/rocketmq:5.3.0 ### JDK Version openjdk version "1.8.0_422" OpenJDK Runtime Environment (Temurin)(build 1.8.0_422-b05) OpenJDK 64-Bit Server VM (Temurin)(build 25.422-b05, mixed mode) ### Describe the Bug 在双主双从的集群环境下，创建一个topic，持续的生产消息，多个消费者只能消费其中一个broker的消息。另一个broker中的消息不能被消费。 ### Steps to Reproduce - 修改docker_compose文件[docker_compose.zip](https://github.com/user-attachments/files/17171947/docker_compose.zip)，修改配置文件的挂载目录(使配置文件，能正确映射到docker 容器) - 修改配置文件中brokerIP1为本机IP - 使用docker_compose启动集群 - 进入某个容器，使用`sh mqadmin updateTopic -n -t -c -a +message.type=NORMAL`命令创建topic - 使用`https://github.com/apache/rocketmq-clients/archive/refs/tags/golang/v5.1.0-rc.1.zip`这个SDK生产消息，然后同样使用这个SDK中的示例代码，修改EndPoint和topic,使用simple_consumer中的消费者启动. - 然后就能观测到只能消费一部分消息(经排查，消费到的消息属于一个broker,未消费到的消息属于另外一个broker) ### What Did You Expect to See? 消费者能够消费集群中所有的消息 ### What Did You See Instead? 只能消费部分消息。 ### Additional Context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Consume启动报错 [rocketmq-clients]
github-actions[bot] commented on issue #824: URL: https://github.com/apache/rocketmq-clients/issues/824#issuecomment-2380312916 This issue is stale because it has been open for 30 days with no activity. It will be closed in 3 days if no further activity occurs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] I cannot run the example for go client v5 [rocketmq-clients]
github-actions[bot] commented on issue #823: URL: https://github.com/apache/rocketmq-clients/issues/823#issuecomment-2380312926 This issue is stale because it has been open for 30 days with no activity. It will be closed in 3 days if no further activity occurs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GH] (rocketmq): Workflow run "Snapshot Daily Release Automation" failed!
The GitHub Actions job "Snapshot Daily Release Automation" on rocketmq.git has failed. Run started by GitHub user lizhanhui (triggered by lizhanhui). Head commit for run: daf3d1a666f3a9d1a9e314a124468e1d05f84ba2 / lizhimins <707364...@qq.com> [maven-release-plugin] prepare for next development iteration (#8750) Report URL: https://github.com/apache/rocketmq/actions/runs/11079459354 With regards, GitHub Actions via GitBox
