[GitHub] [pulsar-test-infra] tisonkun opened a new issue, #57: Remove legacy doc-label-check action
tisonkun opened a new issue, #57: URL: https://github.com/apache/pulsar-test-infra/issues/57 We've switched to docbot. It should be safe to delete doc-label-check. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-test-infra] tisonkun commented on issue #57: Remove legacy doc-label-check action
tisonkun commented on issue #57: URL: https://github.com/apache/pulsar-test-infra/issues/57#issuecomment-1213882471 If we're assuming there're other projects using it, we should publish a tag and remove it in master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-test-infra] tisonkun opened a new issue, #58: Use Docker container action for docbot
tisonkun opened a new issue, #58: URL: https://github.com/apache/pulsar-test-infra/issues/58 https://docs.github.com/en/actions/creating-actions/creating-a-docker-container-action I don't know why we use composition and leave the [docbot workflow](https://github.com/apache/pulsar/blob/fb0f653eadcf6bf72eb8c8efcc29975da6e21267/.github/workflows/ci-documentbot.yml) checkout source code and setup golang. We can make the use experience like pulsarbot action. cc @maxsxu -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-test-infra] tisonkun commented on issue #58: Use Docker container action for docbot
tisonkun commented on issue #58: URL: https://github.com/apache/pulsar-test-infra/issues/58#issuecomment-1213891090 See also https://github.com/tisonkun/pulsar-test-infra/tree/docker-action for a prototype, the docker image size is 12.9MB. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-test-infra] nodece opened a new issue, #59: [docbot] Cannot to label for PR correctly
nodece opened a new issue, #59: URL: https://github.com/apache/pulsar-test-infra/issues/59 https://user-images.githubusercontent.com/16235121/184477262-a02197c7-162d-40b5-b4ac-e86d9ffaf2ce.png";> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-test-infra] tisonkun commented on issue #59: [docbot] Cannot to label for PR correctly
tisonkun commented on issue #59: URL: https://github.com/apache/pulsar-test-infra/issues/59#issuecomment-1214075030 cc @maxsxu @nodece if you can provide concrete reproduce steps, we may investigate the root cause more accurate. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-test-infra] tisonkun commented on issue #59: [docbot] Cannot to label for PR correctly
tisonkun commented on issue #59: URL: https://github.com/apache/pulsar-test-infra/issues/59#issuecomment-1214097267 Related trace: https://github.com/apache/pulsar/runs/7816260130?check_suite_focus=true It seems the content analyze part is correct. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-test-infra] nodece commented on issue #59: [docbot] Cannot to label for PR correctly
nodece commented on issue #59: URL: https://github.com/apache/pulsar-test-infra/issues/59#issuecomment-1214126113 This log looks confused. I have a few questions: - What are `Issue labels`? Is it equal to `PR labels`? - Why do the issue labels miss the `doc-label-missing`? I think there is something wrong with the diff labels. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-test-infra] nodece commented on issue #59: [docbot] Cannot to label for PR correctly
nodece commented on issue #59: URL: https://github.com/apache/pulsar-test-infra/issues/59#issuecomment-1214126877 > if you can provide concrete reproduce steps, we may investigate the root cause more accurate. I'm not sure when happened. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-test-infra] nodece commented on issue #59: [docbot] Cannot to label for PR correctly
nodece commented on issue #59:
URL:
https://github.com/apache/pulsar-test-infra/issues/59#issuecomment-1214130678
I noticed the following code. Why are different types distinguished? I think
all four types should be used the same way for labeling.
```
func (a *Action) Run(actionType string) error {
a.event = actionType
switch actionType {
case "opened", "edited":
return a.onPullRequestOpenedOrEdited()
case "labeled", "unlabeled":
return a.onPullRequestLabeledOrUnlabeled()
}
return nil
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar-test-infra] nodece commented on a diff in pull request #56: avoid duplicate docbot comment
nodece commented on code in PR #56:
URL: https://github.com/apache/pulsar-test-infra/pull/56#discussion_r945160245
##
docbot/main.go:
##
@@ -309,6 +309,11 @@ func (a *Action) onPullRequestOpenedOrEdited() error {
}
}
+ if _, exist := currentLabelsSet[a.config.GetLabelMissing()]; exist &&
checkedCount == 0 {
Review Comment:
Please move this section to line 326.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar-test-infra] nodece commented on a diff in pull request #56: avoid duplicate docbot comment
nodece commented on code in PR #56:
URL: https://github.com/apache/pulsar-test-infra/pull/56#discussion_r945161884
##
docbot/main.go:
##
@@ -418,6 +423,11 @@ func (a *Action) onPullRequestLabeledOrUnlabeled() error {
}
}
+ if _, exist := currentLabelsSet[a.config.GetLabelMissing()]; exist &&
checkedCount == 0 {
Review Comment:
Please move this section to line 411.
##
docbot/main.go:
##
@@ -418,6 +423,11 @@ func (a *Action) onPullRequestLabeledOrUnlabeled() error {
}
}
+ if _, exist := currentLabelsSet[a.config.GetLabelMissing()]; exist &&
checkedCount == 0 {
Review Comment:
Please move this section to line 441.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar-test-infra] tisonkun commented on a diff in pull request #56: avoid duplicate docbot comment
tisonkun commented on code in PR #56:
URL: https://github.com/apache/pulsar-test-infra/pull/56#discussion_r945165238
##
docbot/main.go:
##
@@ -309,6 +309,11 @@ func (a *Action) onPullRequestOpenedOrEdited() error {
}
}
+ if _, exist := currentLabelsSet[a.config.GetLabelMissing()]; exist &&
checkedCount == 0 {
Review Comment:
I can move this section inside `if a.config.GetEnableLabelMissing() &&
checkedCount == 0`, but cannot after `AddLabelsToIssue`. Otherwise, the docbot
will never create comment because we check label missing just after we add it,
and it will be always true.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar-test-infra] tisonkun commented on a diff in pull request #56: avoid duplicate docbot comment
tisonkun commented on code in PR #56:
URL: https://github.com/apache/pulsar-test-infra/pull/56#discussion_r945165322
##
docbot/main.go:
##
@@ -309,6 +309,11 @@ func (a *Action) onPullRequestOpenedOrEdited() error {
}
}
+ if _, exist := currentLabelsSet[a.config.GetLabelMissing()]; exist &&
checkedCount == 0 {
Review Comment:
Updated at 1cab1c23a53446c107d594f177e945e5ffa09733
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar-test-infra] nodece commented on a diff in pull request #56: avoid duplicate docbot comment
nodece commented on code in PR #56:
URL: https://github.com/apache/pulsar-test-infra/pull/56#discussion_r945166295
##
docbot/main.go:
##
@@ -311,6 +311,11 @@ func (a *Action) onPullRequestOpenedOrEdited() error {
// Add missing label
if a.config.GetEnableLabelMissing() && checkedCount == 0 {
+ if _, exist := currentLabelsSet[a.config.GetLabelMissing()];
exist {
+ logger.Infoln("Already added missing label.")
+ return fmt.Errorf("%s", MessageLabelMissing)
Review Comment:
```suggestion
return nil;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar-test-infra] nodece commented on a diff in pull request #56: avoid duplicate docbot comment
nodece commented on code in PR #56:
URL: https://github.com/apache/pulsar-test-infra/pull/56#discussion_r945166378
##
docbot/main.go:
##
@@ -420,6 +425,11 @@ func (a *Action) onPullRequestLabeledOrUnlabeled() error {
// Add missing label
if a.config.GetEnableLabelMissing() && checkedCount == 0 {
+ if _, exist := currentLabelsSet[a.config.GetLabelMissing()];
exist {
+ logger.Infoln("Already added missing label.")
+ return fmt.Errorf("%s", MessageLabelMissing)
Review Comment:
```suggestion
return nil
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [pulsar-test-infra] nodece commented on pull request #56: avoid duplicate docbot comment
nodece commented on PR #56: URL: https://github.com/apache/pulsar-test-infra/pull/56#issuecomment-1214186102 Sorry that I forget to review the return value. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [VOTE] Pulsar Release 2.8.4 Candidate 1
+1 (non-binding) Is 2.8.4 a long term support release? Yunze Xu 于2022年8月12日周五 16:20写道： > This is the first release candidate for Apache Pulsar, version 2.8.4. > > It fixes the following issues: > > https://github.com/apache/pulsar/pulls?q=is%3Amerged+is%3Apr+label%3Arelease%2F2.8.4 > > *** Please download, test and vote on this release. This vote will stay > open > for at least 72 hours *** > > Note that we are voting upon the source (tag), binaries are provided for > convenience. > > Source and binary files: > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.8.4-candidate-1/ > > SHA-512 checksums: > > c3d26704f2cfb3365c29d4110612ca7351084f8bee3c306d5e906b3f9b22c7557cc5baf12f74f8c222baccae3310691419eda5b47fdf9cd6c5281b70134ab5eb > apache-pulsar-2.8.4-bin.tar.gz > 28160ee94dccfb74dfb56e0e5d0e08870c6612659507333a52b5660ecbf060a75d1eed667cffd8596f9468de95055b78916b932db0e0d4c2745868d55429ee98 > apache-pulsar-2.8.4-src.tar.gz > > Maven staging repo: > https://repository.apache.org/content/repositories/orgapachepulsar-1170/ > > The tag to be voted upon: > v2.8.4-candidate-1 (02ee5616866d4eda8dd94f85d9d9b71c459f248d) > https://github.com/apache/pulsar/releases/tag/v2.8.4-candidate-1 > > Pulsar's KEYS file containing PGP keys we use to sign the release: > https://dist.apache.org/repos/dist/dev/pulsar/KEYS > > Docker images: > > > https://hub.docker.com/layers/pulsar/bewaremypower/pulsar/2.8.4/images/sha256-fba51a75c0f2ca79fbff7b254f80f641fcda661fd702f8149bbfdd5994078e3a > > > https://hub.docker.com/layers/pulsar-all/bewaremypower/pulsar-all/2.8.4/images/sha256-42d4b41e5869edc6242bb49d6a1687bd6d191a6385637122edc5c7b2c44ee46f > > Please download the source package, and follow the Release Candidate > Validation[1] to validate the release > > [1] https://github.com/apache/pulsar/wiki/Release-Candidate-Validation > > Thanks, > Yunze > > > > > -- BR, Qiang Huang
Re: [VOTE] Pulsar Release 2.7.5 Candidate 2
+1 (non-binding) Haiting Jiang 于2022年8月13日周六 00:27写道： > Here is the docker images: > > > https://hub.docker.com/layers/271293864/jason918/pulsar/2.7.5/images/sha256-8863429ae891cfdac609455992105427188ccadef98723819c882644084748c9 > > > https://hub.docker.com/layers/271294049/jason918/pulsar-all/2.7.5/images/sha256-db90c52962abd7314973242e272b91be36c1ac34a206db482575fc65d9f3abaf > > Thanks, > Haiting > > On 2022/08/12 07:50:43 Haiting Jiang wrote: > > This is the second release candidate for Apache Pulsar, version 2.7.5. > > > > It fixes the following issues: > > https://github.com/apache/pulsar/compare/v2.7.4...v2.7.5-candidate-2 > > > > *** Please download, test and vote on this release. This vote will stay > open > > for at least 72 hours *** > > > > Note that we are voting upon the source (tag), binaries are provided for > > convenience. > > > > Source and binary files: > > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.7.5-candidate-2/ > > > > SHA-512 checksums: > > > > > 50ab0c3fd954fd1911c84917be411302e8b4dd1afca8b6af67df6d4f4f74ea43125dd18c2bc84aa62b635ff3ca21c54d59085a89ef029d15e7f078854958edb5 > > apache-pulsar-2.7.5-bin.tar.gz > > > 9d8a327a728b917513b492cda585c11423705261e7f6a13d1c9e33a991b4e6f60b9aae7179a24c56e1c7bf0136d9fb64199e8b7952f2a63e8ed98bb518a05988 > > apache-pulsar-2.7.5-src.tar.gz > > > > Maven staging repo: > > https://repository.apache.org/content/repositories/orgapachepulsar-1171/ > > > > The tag to be voted upon: > > v2.7.5-candidate-2 (43a8436ca6cd6604cd25174d7388390bcd5d6b12) > > https://github.com/apache/pulsar/releases/tag/v2.7.5-candidate-2 > > > > Pulsar's KEYS file containing PGP keys we use to sign the release: > > https://dist.apache.org/repos/dist/dev/pulsar/KEYS > > > > Please download the source package, and follow the README to build > > and run the Pulsar standalone service. > > > > Thanks, > > Haiting > > > -- BR, Qiang Huang
[jira] [Commented] (PULSAR-22) flink消费pulsar时出现错误，但是仍可消费到数据
[
https://issues.apache.org/jira/browse/PULSAR-22?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17579311#comment-17579311
]
Qiang Huang commented on PULSAR-22:
---
Could you please submit an issue in [https://github.com/apache/pulsar/issues] ?
Also please provide a reproduce step, thank you.
> flink消费pulsar时出现错误，但是仍可消费到数据
>
>
> Key: PULSAR-22
> URL: https://issues.apache.org/jira/browse/PULSAR-22
> Project: Pulsar
> Issue Type: Bug
> Environment: kafka(json) -> flink(Stream) -> custom
> Serializer -> kop -> pulsar -> flink(stream) keyshared consum(问题所在区域)
>Reporter: 海洋饼干
>Priority: Major
>
> [Source Data Fetcher for Source: kafkaSource (4/4)#0] ERROR
> org.apache.flink.connector.pulsar.source.reader.split.PulsarPartitionSplitReaderBase
> - Error in polling message from pulsar consumer.
> java.util.concurrent.ExecutionException:
> org.apache.pulsar.client.api.PulsarClientException$TransactionConflictException:
>
> \{"errorMsg":"org.apache.pulsar.transaction.common.exception.TransactionConflictException:
>
> [persistent://public/default/HB0002_rdwBaseCtr_aaamore5005-test-pulsar-partition-0][flink-source]
> Transaction:(1,12) try to ack batch message:810:24 in pending ack
> status.","reqId":4357026755663389273,
> "remote":"tk01-bd-test-pulsar-7-139/192.168.7.139:6650",
> "local":"/192.168.34.54:9756"}
> at
> java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
> at
> org.apache.flink.connector.pulsar.source.reader.split.PulsarUnorderedPartitionSplitReader.pollMessage(PulsarUnorderedPartitionSplitReader.java:98)
> at
> org.apache.flink.connector.pulsar.source.reader.split.PulsarPartitionSplitReaderBase.fetch(PulsarPartitionSplitReaderBase.java:110)
> at
> org.apache.flink.connector.pulsar.source.reader.split.PulsarUnorderedPartitionSplitReader.fetch(PulsarUnorderedPartitionSplitReader.java:56)
> at
> org.apache.flink.connector.base.source.reader.fetcher.FetchTask.run(FetchTask.java:58)
> at
> org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:142)
> at
> org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:105)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)
> Caused by:
> org.apache.pulsar.client.api.PulsarClientException$TransactionConflictException:
>
> \{"errorMsg":"org.apache.pulsar.transaction.common.exception.TransactionConflictException:
>
> [persistent://public/default/HB0002_rdwBaseCtr_aaamore5005-test-pulsar-partition-0][flink-source]
> Transaction:(1,12) try to ack batch message:810:24 in pending ack
> status.","reqId":4357026755663389273,
> "remote":"tk01-bd-test-pulsar-7-139/192.168.7.139:6650",
> "local":"/192.168.34.54:9756"}
> at
> org.apache.pulsar.client.impl.ClientCnx.getPulsarClientException(ClientCnx.java:1177)
> at
> org.apache.pulsar.client.impl.ClientCnx.handleAckResponse(ClientCnx.java:431)
> at
> org.apache.pulsar.common.protocol.PulsarDecoder.channelRead(PulsarDecoder.java:150)
> at
> org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
> at
> org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
> at
> org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
> at
> org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:327)
> at
> org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:299)
> at
> org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
> at
> org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
> at
> org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
> at
> org.apache.pulsar.shade.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
> at
> org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:
Re: [DISCUSS] Create a new Github Project to track the flaky tests
+1. Flaky tests failures bother contributors a lot. Resolving flak test failures can greatly improve development efficiency. PengHui Li 于2022年8月13日周六 10:06写道： > I thought maybe we can close the flaky test with the Stale label. > All of them are created very early and have no recurrence within a month. > Maybe fixed by some PRs, but not sure about it. > > Currently, I haven't added them to the Github Project. > But still open in the Github issue list. > > Another one is the add-to-project CI does not work for classic projects. > Lishen https://github.com/yaalsn is trying to improve the > pulsar-test-infra > to > support adding issues/PRs to classic projects automatically. > > Best, > Penghui > > On Sat, Aug 13, 2022 at 1:50 AM Lari Hotari wrote: > > > Thank you Penghui, really useful way to coordinate the fixing of flaky > > tests! > > > > -Lari > > > > On Wed, Aug 10, 2022 at 11:35 AM PengHui Li wrote: > > > > > Hi all, > > > > > > For better tracking flaky test fix, I have tried to create a Github > > > Project under the Pulsar repo > > https://github.com/apache/pulsar/projects/11 > > > (It can be removed if we don't like this way) > > > This will help us to have a more intuitive view of all the flaky tests, > > > how many are in progress, in the review stage, and approved. > > > > > > The project is public for all the contributors, so if you want > > > to contribute some flaky tests fixes, > > > you can go to the Github Project to peek up items in the Todo column. > > > > > > And I also created a PR https://github.com/apache/pulsar/pull/17038 to > > > add the PRs and issues > > > with `flaky-tests` label to this project automatically. > > > > > > BTW, I also have some questions about the Github Project automation. As > > > the description of > > > column `Review in progress, it said the PR with request changes would > go > > > to this column > > > automatically. But it doesn't work. I'm not sure why. > > > [image: image.png] > > > > > > Best, > > > Penghui > > > > > > > > > -- BR, Qiang Huang
Re: [DISCUSS] Switch to the Temurin JDK in the Docker image
LGTM. They are very similar. The OpenJDK provides source-code, and the Temurin JDK provides builds of the source code. I would like to point out that they have different licenses. - The OpenJDK implementation is licensed under the GPL-2.0-only with a linking exception. [0] - The Eclipse Temurin™ project provides code and processes that support the building of runtime binaries and associated technologies that are high performance, enterprise-caliber, cross-platform, open-source licensed, and Java SE TCK-tested for general use across the Java ecosystem.[1] - [0] https://en.wikipedia.org/wiki/OpenJDK https://openjdk.org/legal/ - [1] https://projects.eclipse.org/projects/adoptium.temurin Zixuan Liu 于2022年8月12日周五 15:46写道： > Hi tison, > > Great catch! > > The Temurin JDK is OpenJDK distribution from Adoptium, the old JDK from > Ubuntu, they should all be built on the OpenJDK open source project, so I > think should be fully compatible. > > Each Temurin release has passed the relevant Oracle Java Compatibility Kit > (JCK) to demonstrate that it is a compatible implementation of the Java > specification. In addition, Temurin releases must pass the AQAvit quality > verification suite[1] to ensure they are ready for production usage. > > [1] - https://adoptium.net/docs/qvs-policy > > Thanks, > Zixuan > > > tison 于2022年8月12日周五 15:23写道： > > > +1 > > > > Thanks for bringing this topic :) > > > > In Pulsar usages, these two distributions should not be quite different. > > Did you investigate the compatibility more? > > > > Best, > > tison. > > > > > > Zixuan Liu 于2022年8月12日周五 15:17写道： > > > > > Hi all, > > > > > > I noticed we are using OpenJDK in our Docker image, I suggest that we > > > switch to the Temurin JDK, because our CI runs on the Temurin JDK, we > > need > > > to keep the same JDK everywhere to avoid unexpected problems. Thanks, > > > Zixuan > > > > > > -- BR, Qiang Huang
Re: [Discuss] PIP 198: Standardize PR Naming Convention using GitHub Actions
I agree that the customized one is better. +1 on the customized one. Jun M 于2022年8月12日周五 10:51写道： > +1 on the customized one. > > > Cheers > momo-jun > > -- BR, Qiang Huang
Re: [DISCUSS] Skip unnecessary tests when there are only cpp/python related changes
+1. It can help reduce a lot of useless duplication of test cases. Michael Marshall 于2022年8月11日周四 09:58写道： > Great suggestion, +1. > > - Michael > > On Wed, Aug 10, 2022 at 8:38 PM Dave Fisher wrote: > > > > Yes, please! > > > > Sent from my iPhone > > > > > On Aug 10, 2022, at 5:39 PM, PengHui Li wrote: > > > > > > +1 > > > > > > Best, > > > Penghui > > > > > >> On Wed, Aug 10, 2022 at 10:52 PM Yunze Xu > > > >> wrote: > > >> > > >> LGTM > > >> > > >> Thanks, > > >> Yunze > > >> > > >> > > >> > > >> > > >>> 2022年8月10日 15:36，Zike Yang 写道： > > >>> > > >>> Hi, Pulsar community > > >>> > > >>> Currently, Java tests consume significant CI resources. And it is not > > >>> necessary to run all the tests for changes that are only on the C++ > or > > >>> python parts of the code. I have created a PR [0] to improve the CI > by > > >>> skipping unnecessary tests when there are only CPP/Python changes. > > >>> This can significantly increase the efficiency of CI when testing the > > >>> C++/Python part of the code. > > >>> > > >>> After this PR gets merged, we will skip java unit tests, integration > > >>> tests(the part only for java codes), and go function tests when there > > >>> are only cpp/python changes. But the system test is not skipped > > >>> because there are some python function codes in that test. Perhaps in > > >>> the future, we can further optimize the system test to skip > > >>> unnecessary matrix tests for PRs with only C++ changes. > > >>> > > >>> I have created a test PR in a separate repo to verify this PR. [1] > > >>> And more detail in [2]. > > >>> > > >>> Please take a look and feel free to comment on it. > > >>> > > >>> Regarding the current Pulsar CI, I have a question. Why do we need to > > >>> add doc_only check at each step when skipping code tests instead of > > >>> just skipping the whole job for PR with only doc changes? [3] Is > there > > >>> any concern? > > >>> > > >>> Please let me know what you think. Thanks! > > >>> > > >>> > > >>> [0] https://github.com/apache/pulsar/pull/16988 > > >>> [1] https://github.com/RobertIndie/pulsar-ci-test/pull/1 > > >>> [2] > > >> https://github.com/RobertIndie/pulsar-ci-test/actions/runs/2829525510 > > >>> [3] > > >> > https://github.com/apache/pulsar/blob/master/.github/workflows/pulsar-ci.yaml#L380 > > >>> > > >>> Best, > > >>> Zike Yang > > >> > > >> > > > -- BR, Qiang Huang
Re: Questions about the release process
Thanks for raising this Yunze. The docker image was added to the VOTE threads as a convenience for voters/testers. I don't believe we established a formal requirement for it, which might explain why it is not in the wiki. It should, however, be listed as an optional step if we want it to be one. I agree with putting the release process in the apache/pulsar git repo. > we do need to assure that the PMC fully reviews changes Currently, any committer has access to modify the release docs, so this introduces a new requirement. One way to enforce who reviews a PR is with a CODEOWNERS file. We could create a GitHub group with PMC members and then require that any PR that touches the release process documentation file is approved by a PMC member. I am not sure that this is the "right" direction though. > Do not make the release docs part of versioned docs. I agree. Thanks, Michael On Fri, Aug 12, 2022 at 10:32 PM Yunze Xu wrote: > > > The RM should ask a PMC member to help them add their KEY. > > Do not make the release docs part of versioned docs. > I agree. > > Thanks, > Yunze > > > > > > 2022年8月12日 23:10，Dave Fisher 写道： > > > > Hi - > > > > One change that needs to be made is regarding the KEYS file. > > > > We should drop the use of > > https://dist.apache.org/repos/dist/dev/pulsar/KEYS instead we should > > carefully update https://dist.apache.org/repos/dist/release/pulsar/KEYS > > > > The two KEYS files are currently out of sync. The release file had to be > > hand reconstructed at the beginning of the year and I’ve had to deal with > > new Release Manager KEYS that were not copied during the Release Process. > > (Recently Apache Infra has been scanning release and is informing PMCs when > > their releases are broken.) > > > > The RM should ask a PMC member to help them add their KEY. I’m willing to > > do it and I’m certain other PMC members would do the same. > > > > The VOTE threads can then always refer to a proper KEYS file that will > > always be correct. RMs should also make sure that their KEY does not expire > > while the release is active which could be for several years. If your KEY > > is revoked at some point then please let the PMC know. > > > > I like moving the Release Docs to the codebase, but we do need to assure > > that the PMC fully reviews changes. The reviews that count before squash > > and merge must be from PMC members. The reason is that the Pulsar PMC is > > responsible for assuring that Pulsar releases comply with Apache Release > > Policies. > > > > Do not make the release docs part of versioned docs. There should be only > > the current policy. If other products of the Pulsar project require > > different release docs it is fine to have separate docs. > > > > All The Best, > > Dave > > > >> On Aug 12, 2022, at 7:41 AM, tison wrote: > >> > >> Hi Penghui & Yunze, > >> > >> I ever wrote developer guides for TiDB[1] and Apache Kvrocks (Incubating), > >> including the release guide for the latter[2]. > >> > >> Just for your information, I'm preparing the proposal to bring a developer > >> guide page (series docs). Perhaps start in the next month. > >> > >> Although, it cannot help the current status, and I don't want to discuss > >> details on this topic here. Again, just for your information :) > >> > >> Best, > >> tison. > >> > >> [1] https://pingcap.github.io/tidb-dev-guide/ > >> [2] https://kvrocks.apache.org/community/how-to-release > >> > >> > >> Yunze Xu 于2022年8月12日周五 21:57写道： > >> > >>> Yeah, I agree. It’s better to move the release process to the codebase. > >>> > >>> Regarding the automatic validation program, we can have that for some > >>> common verifications like the GPG verification, which only requires a > >>> simple > >>> command if you have downloaded the binary. > >>> > >>> Thanks, > >>> Yunze > >>> > >>> > >>> > >>> > 2022年8月12日 18:12，PengHui Li 写道： > > Thanks for raising this question. > > Maybe we'd better move the release process doc and validation doc > to the codebase, not the wiki pages. > > - Only committers can update the wiki pages > - The changes without review > > After moving to the pulsar codebase > > - Everyone can contribute to the validation doc > - The release process doc update can get reviewers to review > > I think there are multiple issues that need to be resolved for the > >>> release > process > > - Have the Python client(Linux, osx) at the RC stage, I think currently > >>> we > only have the C++ client for RC, but push to pypi after the RC gets > >>> passed > - Add validation process for the Python and C++ client > - Add the Go function and Python function validation process > - Add a script for building images for RC > - Add images validation process > > And another point is can we have an automatic validation program to > >>> reduce > the burden on validators? > I'm not
[GitHub] [pulsar-test-infra] michaeljmarshall commented on issue #58: Use Docker container action for docbot
michaeljmarshall commented on issue #58: URL: https://github.com/apache/pulsar-test-infra/issues/58#issuecomment-1214284219 I agree with moving this to a docker action. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [DISCUSS] PIP-186: Introduce two phase deletion protocol based on system topic
Good idea. I suggest the naming endings with `Seconds` like sendDelayOfTwoPhaseDeletionInSeconds,` reconsumeLaterOfTwoPhaseDeletionInSeconds`. >private int sendDelaySecondsOfTwoPhaseDeletion = 60; >private int reconsumeLaterSecondsOfTwoPhaseDeletion = 600; Yan Zhao 于2022年8月12日周五 10:45写道： > > I suggest to include 'topic' in the flag, we have too many entities in > > Pulsar > > Thanks, change it to `topicTwoPhaseDeletionEnabled`. > -- BR, Qiang Huang
[GitHub] [pulsar-test-infra] michaeljmarshall commented on pull request #46: Add LICENSE to project's root directory
michaeljmarshall commented on PR #46: URL: https://github.com/apache/pulsar-test-infra/pull/46#issuecomment-1214284523 Thanks for confirming @tisonkun. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-test-infra] michaeljmarshall merged pull request #46: Add LICENSE to project's root directory
michaeljmarshall merged PR #46: URL: https://github.com/apache/pulsar-test-infra/pull/46 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-test-infra] michaeljmarshall closed issue #45: Add LICENSE file to root of project directory
michaeljmarshall closed issue #45: Add LICENSE file to root of project directory URL: https://github.com/apache/pulsar-test-infra/issues/45 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: Questions about the release process
Sent from my iPhone > On Aug 13, 2022, at 9:35 PM, Michael Marshall wrote: > > Thanks for raising this Yunze. The docker image was added to the VOTE > threads as a convenience for voters/testers. I don't believe we > established a formal requirement for it, which might explain why it is > not in the wiki. It should, however, be listed as an optional step if > we want it to be one. > > I agree with putting the release process in the apache/pulsar git repo. > >> we do need to assure that the PMC fully reviews changes > > Currently, any committer has access to modify the release docs, so > this introduces a new requirement. One way to enforce who reviews a PR > is with a CODEOWNERS file. We could create a GitHub group with PMC > members and then require that any PR that touches the release process > documentation file is approved by a PMC member. I am not sure that > this is the "right" direction though. There is already a Pulsar PMC / private group in GitHub that is maintained by ASF infra. We would need to briefly discuss with them if we choose this approach > >> Do not make the release docs part of versioned docs. > > I agree. > > Thanks, > Michael > > >> On Fri, Aug 12, 2022 at 10:32 PM Yunze Xu >> wrote: >> >>> The RM should ask a PMC member to help them add their KEY. >>> Do not make the release docs part of versioned docs. >> I agree. >> >> Thanks, >> Yunze >> >> >> >> >>> 2022年8月12日 23:10，Dave Fisher 写道： >>> >>> Hi - >>> >>> One change that needs to be made is regarding the KEYS file. >>> >>> We should drop the use of >>> https://dist.apache.org/repos/dist/dev/pulsar/KEYS instead we should >>> carefully update https://dist.apache.org/repos/dist/release/pulsar/KEYS >>> >>> The two KEYS files are currently out of sync. The release file had to be >>> hand reconstructed at the beginning of the year and I’ve had to deal with >>> new Release Manager KEYS that were not copied during the Release Process. >>> (Recently Apache Infra has been scanning release and is informing PMCs when >>> their releases are broken.) >>> >>> The RM should ask a PMC member to help them add their KEY. I’m willing to >>> do it and I’m certain other PMC members would do the same. >>> >>> The VOTE threads can then always refer to a proper KEYS file that will >>> always be correct. RMs should also make sure that their KEY does not expire >>> while the release is active which could be for several years. If your KEY >>> is revoked at some point then please let the PMC know. >>> >>> I like moving the Release Docs to the codebase, but we do need to assure >>> that the PMC fully reviews changes. The reviews that count before squash >>> and merge must be from PMC members. The reason is that the Pulsar PMC is >>> responsible for assuring that Pulsar releases comply with Apache Release >>> Policies. >>> >>> Do not make the release docs part of versioned docs. There should be only >>> the current policy. If other products of the Pulsar project require >>> different release docs it is fine to have separate docs. >>> >>> All The Best, >>> Dave >>> On Aug 12, 2022, at 7:41 AM, tison wrote: Hi Penghui & Yunze, I ever wrote developer guides for TiDB[1] and Apache Kvrocks (Incubating), including the release guide for the latter[2]. Just for your information, I'm preparing the proposal to bring a developer guide page (series docs). Perhaps start in the next month. Although, it cannot help the current status, and I don't want to discuss details on this topic here. Again, just for your information :) Best, tison. [1] https://pingcap.github.io/tidb-dev-guide/ [2] https://kvrocks.apache.org/community/how-to-release Yunze Xu 于2022年8月12日周五 21:57写道： > Yeah, I agree. It’s better to move the release process to the codebase. > > Regarding the automatic validation program, we can have that for some > common verifications like the GPG verification, which only requires a > simple > command if you have downloaded the binary. > > Thanks, > Yunze > > > > >> 2022年8月12日 18:12，PengHui Li 写道： >> >> Thanks for raising this question. >> >> Maybe we'd better move the release process doc and validation doc >> to the codebase, not the wiki pages. >> >> - Only committers can update the wiki pages >> - The changes without review >> >> After moving to the pulsar codebase >> >> - Everyone can contribute to the validation doc >> - The release process doc update can get reviewers to review >> >> I think there are multiple issues that need to be resolved for the > release >> process >> >> - Have the Python client(Linux, osx) at the RC stage, I think currently > we >> only have the C++ client for RC, but push to pypi after the RC gets > passed >> - Add
Re: [DISCUSS] Allow the producer to connect to the topic with producer_request_hold backlog policy
Great points, Penghui.
> To optimize, I think we can only let the producer connect to the broker
> and the broker should tell the producer the backlog is exceeded.
In looking at the `CommandProducerSuccess` protobuf message, we
already have the `producer_ready` boolean field. It was added for the
exclusive producer case, and it seems to match this logic exactly,
though the client wouldn't know "why" the producer was not ready. I
think this field meets our needs because the producer just needs to
know that it is connected and should not send messages yet.
> If we have too many producers, try to reconnect to the broker again and
> again. It is also a non-negligible cost.
This is a really important point. The current protocol implementation
leads to unnecessary network traffic, and it will be worse for
topics with many producers. Note that the lookup part of the protocol
introduces additional load on all of the brokers serving these
requests.
> Looks like we need to fix the client-side to make sure users will not get
> ProducerBlockedQuotaExceededError when creating the producer with
> producer_hold_request backlog policy. I have tested it locally, the behavior
> can be confirmed
Thanks for confirming this case. I think it would make sense to update
the behavior on the producer_requests_hold feature so that the future
is incomplete until the producer is ready to produce, just like the
exclusive producer implementation.
Ultimately, there are two cases this feature needs to handle.
1. A new producer connecting to a topic that has exceeded its quota.
This case is trivial because the broker tells the producer it is
connected but it cannot send any messages (i.e. producer_ready=false),
and the client holds the producer's future until the broker sends
producer_ready=true.
2. An existing producer gets disconnected due to an exceeded quota. In
this case, it'd be easy enough for the producer to stop sending
messages, but because the client application already has a reference
to this producer, the application will be able to submit messages
until the client's buffer is full, at which point the send is blocked
or gets an exception. I think that would work, but it could be worth
extending the client so that an application could reactively discover
that the topic's quota is exceeded using a listener. Additionally, we
could disable the send timeouts when the producer enters this "hold"
state so that messages.
In case number 2, it probably makes sense to extend the protocol so
that the broker sends a protocol message indicating the producer
should stop sending messages. This would be more elegant than
disconnecting the producer and making it look up the topic again.
Thanks,
Michael
On Fri, Aug 12, 2022 at 5:50 AM PengHui Li wrote:
>
> > The producer fails
> the pending messages when the policy is producer_exception and the
> producer does nothing when the policy is producer_request_hold
>
> Eventually, it will fail [0] the user's create producer request because of
> the operation timeout [1].
>
> > The primary advantage for this solution is that the broker does not
> need to hold a producer's messages in memory for some undefined time.
>
> Yes, I agree. And changing the protocol will also affect other clients.
>
> To optimize, I think we can only let the producer connect to the broker
> and the broker should tell the producer the backlog is exceeded.
> The producer can only send one message to test. Only push out more messages
> after the first message gets the response. Just a rough idea, not for now.
> If we have too many producers, try to reconnect to the broker again and
> again.
> It is also a non-negligible cost.
>
> Looks like we need to fix the client-side to make sure users will not get
> ProducerBlockedQuotaExceededError when creating the producer with
> producer_hold_request backlog policy. I have tested it locally, the behavior
> can be confirmed
>
> ```
> org.apache.pulsar.client.api.PulsarClientException$ProducerBlockedQuotaExceededError:
> {"errorMsg":"org.apache.pulsar.broker.service.BrokerServiceException$TopicBacklogQuotaExceededException:
> Cannot create producer on topic with backlog quota
> exceeded","reqId":1841236212888635356, "remote":"localhost/127.0.0.1:64805",
> "local":"/127.0.0.1:64809"}
>
> at
> org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:1052)
> at
> org.apache.pulsar.client.impl.ProducerBuilderImpl.create(ProducerBuilderImpl.java:88)
> at
> org.apache.pulsar.broker.service.BacklogQuotaManagerTest.createProducer(BacklogQuotaManagerTest.java:664)
> at
> org.apache.pulsar.broker.service.BacklogQuotaManagerTest.testProducerException(BacklogQuotaManagerTest.java:1091)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethod
[GitHub] [pulsar-test-infra] tisonkun commented on a diff in pull request #56: avoid duplicate docbot comment
tisonkun commented on code in PR #56:
URL: https://github.com/apache/pulsar-test-infra/pull/56#discussion_r945234643
##
docbot/main.go:
##
@@ -311,6 +311,11 @@ func (a *Action) onPullRequestOpenedOrEdited() error {
// Add missing label
if a.config.GetEnableLabelMissing() && checkedCount == 0 {
+ if _, exist := currentLabelsSet[a.config.GetLabelMissing()];
exist {
+ logger.Infoln("Already added missing label.")
+ return fmt.Errorf("%s", MessageLabelMissing)
Review Comment:
We should return the error here. Still the pr needs a valid doc label. We
just skip the create comment step.
##
docbot/main.go:
##
@@ -420,6 +425,11 @@ func (a *Action) onPullRequestLabeledOrUnlabeled() error {
// Add missing label
if a.config.GetEnableLabelMissing() && checkedCount == 0 {
+ if _, exist := currentLabelsSet[a.config.GetLabelMissing()];
exist {
+ logger.Infoln("Already added missing label.")
+ return fmt.Errorf("%s", MessageLabelMissing)
Review Comment:
See https://github.com/apache/pulsar-test-infra/pull/56#discussion_r945234643
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
