[GitHub] [pulsar-test-infra] Anonymitaet commented on pull request #70: Fix sync and remove labels

Anonymitaet commented on PR #70: URL: https://github.com/apache/pulsar-test-infra/pull/70#issuecomment-1240329162 @maxsxu could you please review this PR from a technical perspective? Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: Pulsar CI congested, master branch build broken

If my assumption of the GitHub usage metrics bug in the GitHub Actions build job queue fairness algorithm is correct, what would help is running the flaky unit test group outside of Pulsar CI workflow. In that case, the impact of the usage metrics would be limited. The example of https://github

[GitHub] [pulsar-client-node] rkaw92 commented on issue #230: await flush() does not actually flush

rkaw92 commented on issue #230: URL: https://github.com/apache/pulsar-client-node/issues/230#issuecomment-1240333920 I currently rely on back-off, but it's not optimal. As for filling the queue, I'd think that issuing exactly as many `send()` operations as should fit in the buffer (exactly

[GitHub] [pulsar-test-infra] tisonkun commented on pull request #70: Fix sync and remove labels

tisonkun commented on PR #70: URL: https://github.com/apache/pulsar-test-infra/pull/70#issuecomment-1240336055 Can I summarize the issue on the current master is that the `doc` related labels can be more than they should be? That's, no comments mangled regression as https://github.co

[GitHub] [pulsar-test-infra] tisonkun commented on pull request #70: Fix sync and remove labels

tisonkun commented on PR #70: URL: https://github.com/apache/pulsar-test-infra/pull/70#issuecomment-1240338681 I think we should try to decide which input is the source. Right now, 1. A change on labels manually by committers can trigger PR description modified. 2. A change in PR d

[GitHub] [pulsar-test-infra] maxsxu commented on a diff in pull request #70: Fix sync and remove labels

maxsxu commented on code in PR #70: URL: https://github.com/apache/pulsar-test-infra/pull/70#discussion_r965611984 ## docbot/action.go: ## @@ -47,11 +48,17 @@ func NewActionWithClient(ctx context.Context, ac *ActionConfig, client *github.C } } +const openedActionType

Re: Pulsar CI congested, master branch build broken

Thanks Lari for the detailed explanation. This is kind of an emergency situation and I believe your plan is the way to go now. I already prepared a pull for moving the flaky suite out of the Pulsar CI workflow: https://github.com/nicoloboschi/pulsar/pull/8 I can take care of the execution of the p

[GitHub] [pulsar-test-infra] Anonymitaet commented on pull request #70: Fix sync and remove labels

Anonymitaet commented on PR #70: URL: https://github.com/apache/pulsar-test-infra/pull/70#issuecomment-1240351605 @tisonkun thanks! "One source" (only modifying PR description triggers bot) is an ideal situation, but we can not enforce ppl not to update labels from **Label** icon bec

[GitHub] [pulsar-client-node] rkaw92 commented on issue #230: await flush() does not actually flush

rkaw92 commented on issue #230: URL: https://github.com/apache/pulsar-client-node/issues/230#issuecomment-1240355308 Aha! https://github.com/apache/pulsar-client-node/blob/master/src/ProducerConfig.cc#L103 Setting `maxPendingMessagesAcrossPartitions` actually calls `pulsar_pr

[GitHub] [pulsar-client-node] rkaw92 opened a new pull request, #231: Respect maxPendingMessagesAcrossPartitions

rkaw92 opened a new pull request, #231: URL: https://github.com/apache/pulsar-client-node/pull/231 Fixes #230 This change should help avoid `ProducerQueueIsFull` errors when producing to a partitioned topic with batchingEnabled. -- This is an automated message from the Apache Git

Re: Pulsar CI congested, master branch build broken

This is the pull for step 2. https://github.com/apache/pulsar/pull/17539 This is the script I'm going to use to cancel pending workflows. https://github.com/nicoloboschi/pulsar-validation-tool/blob/master/pulsar-scripts/pulsar-gha/cancel-workflows.js I'm going to run the script in minutes. I adv

Re: Pulsar CI congested, master branch build broken

Thank you Nicolo. There's lazy consensus, let's go forward with the action plan. -Lari On 2022/09/08 08:16:05 Nicolò Boschi wrote: > This is the pull for step 2. https://github.com/apache/pulsar/pull/17539 > > This is the script I'm going to use to cancel pending workflows. > https://github.com/

[GitHub] [pulsar-test-infra] nodece commented on pull request #70: Fix remove labels

nodece commented on PR #70: URL: https://github.com/apache/pulsar-test-infra/pull/70#issuecomment-1240448306 We have discussed this offline, using PR description as the only input, thanks all. -- This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [pulsar-test-infra] tisonkun commented on a diff in pull request #70: Fix remove labels

tisonkun commented on code in PR #70: URL: https://github.com/apache/pulsar-test-infra/pull/70#discussion_r965721600 ## docbot/action.go: ## @@ -47,11 +47,16 @@ func NewActionWithClient(ctx context.Context, ac *ActionConfig, client *github.C } } +const ( + open

[GitHub] [pulsar-test-infra] nodece commented on a diff in pull request #70: Fix remove labels

nodece commented on code in PR #70: URL: https://github.com/apache/pulsar-test-infra/pull/70#discussion_r965725119 ## docbot/action.go: ## @@ -47,11 +47,16 @@ func NewActionWithClient(ctx context.Context, ac *ActionConfig, client *github.C } } +const ( + opened

[GitHub] [pulsar-test-infra] nodece commented on pull request #70: Fix remove labels

nodece commented on PR #70: URL: https://github.com/apache/pulsar-test-infra/pull/70#issuecomment-1240491889 To publish to the Apache repository, we need PMC operations. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [pulsar-manager] DimitryVlasenko opened a new issue, #487: Set password returns 404

DimitryVlasenko opened a new issue, #487: URL: https://github.com/apache/pulsar-manager/issues/487 **Pulsar manager was deployed using official helm chart** **when trying to set an admin password as per** https://github.com/apache/pulsar-manager#access-pulsar-manager root@puls

Re: [VOTE] PIP-196 Segmented transaction buffer snapshot

Hi, community This proposal has some updates. The latest version of the proposal can be found here . Feel free to comment on this doc. Sincerely, Xiangying On Wed, Sep 7, 2022 at 4:55 PM Xiangying Meng wrote:

[GitHub] [pulsar] jinxiaoyi created a discussion: pulsar-daemon start standalone

GitHub user jinxiaoyi created a discussion: pulsar-daemon start standalone when i start pulsar2.10.1 use ./pulsar-daemon start standalone 3181 port is already use. how can i fix it. may i change the port? GitHub link: https://github.com/apache/pulsar/discussions/17550 This is an

[GitHub] [pulsar-test-infra] maxsxu commented on a diff in pull request #70: Fix remove labels

maxsxu commented on code in PR #70: URL: https://github.com/apache/pulsar-test-infra/pull/70#discussion_r966069645 ## docbot/action.go: ## @@ -130,10 +137,13 @@ func (a *Action) checkLabels() error { for label, checked := range expectedLabelsMap {

[GitHub] [pulsar-test-infra] nodece merged pull request #70: Fix remove labels

nodece merged PR #70: URL: https://github.com/apache/pulsar-test-infra/pull/70 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.ap

[GitHub] [pulsar-test-infra] tisonkun commented on pull request #70: Fix remove labels

tisonkun commented on PR #70: URL: https://github.com/apache/pulsar-test-infra/pull/70#issuecomment-1240869571 I noticed that this PR doesn't have a diff to remove PR edit logic. Will take a closer look now. -- This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [pulsar-test-infra] tisonkun commented on issue #47: Remove Doc Bot Logic To Update PR Descriptions

tisonkun commented on issue #47: URL: https://github.com/apache/pulsar-test-infra/issues/47#issuecomment-1240874814 After #69 & #70 this issue has been resolved. We no longer edit PR description in docbot. -- This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [pulsar-test-infra] nodece commented on pull request #70: Fix remove labels

nodece commented on PR #70: URL: https://github.com/apache/pulsar-test-infra/pull/70#issuecomment-1240875788 `addAndCleanupHelpComment()` method is safe, which doesn't add comments more than once. -- This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [pulsar-test-infra] nodece commented on issue #47: Remove Doc Bot Logic To Update PR Descriptions

nodece commented on issue #47: URL: https://github.com/apache/pulsar-test-infra/issues/47#issuecomment-1240879014 Fixed #69. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[GitHub] [pulsar-test-infra] nodece closed issue #47: Remove Doc Bot Logic To Update PR Descriptions

nodece closed issue #47: Remove Doc Bot Logic To Update PR Descriptions URL: https://github.com/apache/pulsar-test-infra/issues/47 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [DISCUSS] Consumer reconnection causes repeated consumption messages

Hi Haiting When using cumulative ack, we can save the maximum received MessageId on the consumer client side to filter the message duplication caused by reconnection, if the consumer client process restarts the maximum received MessageId will not exist in the consumer client. This requires the use

Re: Pulsar CI congested, master branch build broken

Dear community, The plan has been executed. The summary of our actions is: 1. We cancelled all pending jobs (queue and in-progress) 2. We removed the required checks to be able to merge improvements on the CI workflow 3. We merged a couple of improvements: 1. workarounded the possible bug trigg

[GitHub] [pulsar] gitfy added a comment to the discussion: Pulsar functions : python zip with deps wheel

GitHub user gitfy added a comment to the discussion: Pulsar functions : python zip with deps wheel I found the solution for the above issue. The expectation is to have the zip file with all the wheel files for the dependent libraries. Internally pulsar is running the command to install the de

Re: [VOTE] PIP-196 Segmented transaction buffer snapshot

Hi, Xiangying +1(non-binding) This PIP overall LGTM! It solves the problem of snapshots not being able to scale. We can make some optimizations later : 1. Merge transaction snapshot segments to reduce the number of segments and the index's size. 2. Add snapshot segments to memory as TB requires re

Re: Pulsar CI congested, master branch build broken

Thank you, Lari and Nicolò! Best, tison. Nicolò Boschi 于2022年9月9日周五 02:41写道： > Dear community, > > The plan has been executed. > The summary of our actions is: > 1. We cancelled all pending jobs (queue and in-progress) > 2. We removed the required checks to be able to merge improvements on the

Re: Pulsar CI congested, master branch build broken

Great work. Thank you, Lari and Nicolò. BR, Haiting On Fri, Sep 9, 2022 at 9:36 AM tison wrote: > > Thank you, Lari and Nicolò! > Best, > tison. > > > Nicolò Boschi 于2022年9月9日周五 02:41写道： > > > Dear community, > > > > The plan has been executed. > > The summary of our actions is: > > 1. We cance

Re: Pulsar CI congested, master branch build broken

Fantastic, thank you Lari and Nicolò! - Michael On Thu, Sep 8, 2022 at 9:03 PM Haiting Jiang wrote: > > Great work. Thank you, Lari and Nicolò. > > BR, > Haiting > > On Fri, Sep 9, 2022 at 9:36 AM tison wrote: > > > > Thank you, Lari and Nicolò! > > Best, > > tison. > > > > > > Nicolò Boschi 于

Re: [VOTE] PIP-196 Segmented transaction buffer snapshot

+1(binding) I have done the review on gdoc And please also update the github issue(PIP). Thanks, Penghui On Fri, Sep 9, 2022 at 9:31 AM 丛搏 wrote: > Hi, Xiangying > +1(non-binding) > > This PIP overall LGTM! It solves the problem of snapshots not being > able to scale. > We can make some optim

Re: [DISCUSS] Consumer reconnection causes repeated consumption messages

Hi Bo, Overall it makes sense to me. It is basically the same as broker side deduplication mechanism when producing messages, which uses `sequenceId`. In your case, messageId is used for deduplication. It should work as long as the received messageId increases monotonically. So we should be caref

Re: Enable "Update branch" Github button in pull requests

Hi Nicolo, Thanks for your proposal. After Pulsar enabled the `update branch` bottom, I found a lot of Prs are blocked by running unit tests due to a lot of unit tests rerun by the `update branch` action. The update branch action will cause the CI resource busy and waste a lot of resources if

Re: [DISCUSS] Consumer reconnection causes repeated consumption messages

I agree that reducing unnecessary duplicates is a good goal. For the topic unloading case, it might help to think about how we can improve the protocol. The current handover is very blunt. A new solution could focus on decreasing duplicate message delivery while also focusing on decreasing time wh

[GitHub] [pulsar] michaeljmarshall added a comment to the discussion: pulsar-daemon start standalone

GitHub user michaeljmarshall added a comment to the discussion: pulsar-daemon start standalone This is the bookkeeper port. You can update it in the `conf/standalone.conf` by setting the following to a different port: ```conf bookiePort=3181 ``` GitHub link: https://github.com/apache/pulsar

Re: Enable "Update branch" Github button in pull requests

That is interesting feedback, Hang. I wasn't familiar with the button when it appeared (even though I had read this email thread), and I definitely hit the "update branch" button several times thinking I was resolving conflicts. Perhaps this new button contributed to the CI congestion we saw recen

Re: Enable "Update branch" Github button in pull requests

Good points Hang. Yes, let's remove the "update branch" button since it seems to lead to unnecessary load for CI. -Lari On 2022/09/09 03:55:57 Hang Chen wrote: > Hi Nicolo, > Thanks for your proposal. After Pulsar enabled the `update > branch` bottom, I found a lot of Prs are blocked by run

[GitHub] [pulsar] michaeljmarshall added a comment to the discussion: How do I convert a None-partitioned topic to Partitioned topic

GitHub user michaeljmarshall added a comment to the discussion: How do I convert a None-partitioned topic to Partitioned topic There is not currently a way to convert non-partitioned topics to partitioned topics. If you want to use the same name as a topic name and there is no data in the cur

Re: [DISCUSS] User-friendly acknowledgeCumulative API on a partitioned topic or multi-topics

Tarun is a colleague at Nutanix who is very eager to do his first patch in an apache Project :-) Regards, Shivji Kumar Jha http://www.shivjijha.com/ +91 8884075512 On Wed, 7 Sept 2022 at 17:05, Yunze Xu wrote: > Sure. I’m glad to see that. Just a little confused about who is Tarun? > > Thanks,

Re: Enable "Update branch" Github button in pull requests

That's a good observation. We don't have data so we can't be sure about that, but I think that it can be a possible cause of the recent congestion. I opened an INFRA ticket - https://issues.apache.org/jira/browse/INFRA-23679 (.asf.yaml file doesn't support this option) I will come back to this thr