Re: Zookeeper exception handler in Pulsar

Hi Yong, Thanks for sharing your findings. Would it make sense to also share the issues with some detailed log messages in GH issues so that others that experience these problems would be able to find the later fixes for this problem and track the status? > a BadVersion exception. At this mom

[VOTE] PIP-196 Segmented transaction buffer snapshot

Hi, community I,d like to start a vote for the PIP-196 Segmented transaction buffer snapshot. And the discussion can be found here . Sincerely, Xiangying

Re: Pulsar CI congested, master branch build broken

Thanks Lari! Does this issue cause the tests for PRs like https://github.com/apache/pulsar/pull/17198 to be hang? On 2022/09/06 14:41:07 Dave Fisher wrote: > We are going to need to take actions to fix our problems. See > https://issues.apache.org/jira/browse/INFRA-23633?focusedCommentId=176007

[GitHub] [pulsar] Anonymitaet added a comment to the discussion: [Design] Pulsar All Releases Page

GitHub user Anonymitaet added a comment to the discussion: [Design] Pulsar All Releases Page The reason for highlighting release managers is that this might be an incentive for them (improve visibility). GitHub link: https://github.com/apache/pulsar/discussions/17310#discussioncomment-358304

[GitHub] [pulsar] Anonymitaet added a comment to the discussion: [Design] Pulsar All Releases Page

GitHub user Anonymitaet added a comment to the discussion: [Design] Pulsar All Releases Page We'll implement it as discussed above. I've created an issue to track the progress. https://github.com/apache/pulsar/issues/17514 GitHub link: https://github.com/apache/pulsar/discussions/17310#disc

Re: CloudEvents binding requires standard for Pulsar

Devin, thanks for bringing up this discussion. I have one high level question: what is the goal that we want to achieve ? something like: 1) Use CloudEvents format natively in Pulsar Schema registry, so that Pulsar clients can register their schema using that format 2) Publish on some HTTP endpoin

Re: Zookeeper exception handler in Pulsar

Hi Lari, I file an issue here https://github.com/apache/pulsar/issues/17516 >This sounds like a severe issue that could lead to data loss. Is that correct? What are the implications of this? Yes. What we met before is the consumer gets stuck and the ledger could not read from the tiered storage,

[DISCUSS][PIP-206] Refresh different authentication data

Hi all, This PIP improves how to refresh different authentication data. PIP link: https://github.com/apache/pulsar/issues/17517. Thanks, Zixuan

Re: [DISCUSS] User-friendly acknowledgeCumulative API on a partitioned topic or multi-topics

Sure. I’m glad to see that. Just a little confused about who is Tarun? Thanks, Yunze > On Sep 6, 2022, at 17:40, Shivji Kumar Jha wrote: > > ++ Tarun > > Hi Yunze, > > We would love to have this. > > ```java > // the key is the partitioned topic name like my-topic-partition-0 > void ackno

Re: [DISCUSS] Improvements on the release process

Good suggestion. I will update in the PR soon. Thanks, Yunze > On Sep 6, 2022, at 15:46, Haiting Jiang wrote: > > There are a lot of work before current release process, maybe we should > also include these: > > 1. Start a discussion on the mail list about the release. We can provide a > te

[GitHub] [pulsar-client-node] rkaw92 opened a new issue, #230: await flush() does not actually flush

rkaw92 opened a new issue, #230: URL: https://github.com/apache/pulsar-client-node/issues/230 Hi, I have a problem with the Producer where I do this in a loop: * `send()` a given number of messages * `await flush()`, so that the send queue is empty * `send()` more messages...

[GitHub] [pulsar-client-node] rkaw92 commented on pull request #200: convert worker threads to async functions

rkaw92 commented on PR #200: URL: https://github.com/apache/pulsar-client-node/pull/200#issuecomment-1239305793 Hi, these changes seem great, but I'm now using current master HEAD and I think I'm seeing an issue that is related to this change: #230 I'm not saying it worked well befo

Re: [DISCUSS] Apache Pulsar 2.10.2 release

Hi all, Put some updates here: I have cherry-picked all merged PRs to branch-2.10. And currently there are 199 commits for 2.10.2 release, see [1]. And 22 PRs are postponed to 2.10.3 due to some reasons, see [2]. There are 2 PRs left open as release blocker, see [3]. I will cut 2.10.2 RC1 as soon

Re: [DISCUSS] Pulsar 3.0 - dreaming of the ability to rename objects

Il giorno mar 6 set 2022 alle ore 07:46 Lari Hotari ha scritto: > > I have been thinking of the need for supporting renaming tenants, namespaces > and topics while doing some initial design for "Pulsar 3.0 brainstorming: > Going beyond PIP-45 Pluggable metadata interface", thread > https://list

Re: [DISCUSS] PIP 194 : Pulsar client: seek command add epoch

Hi Qiang, We have a brainstorming session on this PIP over Zoom with Penghui, Hang, and more people, and I'm jotting down here our feedback. Before I do that, I just want to write my own understanding of the document, for other readers: # Context Pulsar, as opposed to other distributed / streami

Re: Pulsar CI congested, master branch build broken

The problem with CI is becoming worse. The build queue is 235 jobs now and the queue time is over 7 hours. We will need to start shedding load in the build queue and get some fixes in. https://issues.apache.org/jira/browse/INFRA-23633 continues to contain details about some activities. I have cr

Re: Pulsar CI congested, master branch build broken

> selecting which jobs to process Do you have a patch to implement this? IIRC it requires interacting with outside service or at least we may add an ok-to-test label. Besides, it increases committers/PMC members' workload - be aware of it, or most of contributions will stall. Best, tison. Lari

Re: Pulsar CI congested, master branch build broken

One possible way forward: 1. Cancel all existing builds in_progress or queued 2. Edit .asf.yaml and drop the "required checks" requirement for merging PRs. 3. Wait for build to run for .asf.yaml change, merge it 4. Disable all workflows 5. Process specific PRs manually to improve the situation.

Re: Pulsar CI congested, master branch build broken

On 2022/09/07 16:59:33 tison wrote: > > selecting which jobs to process > > Do you have a patch to implement this? IIRC it requires interacting with > outside service or at least we may add an ok-to-test label. Very good idea, I didn't think that far ahead. It seems that Apache Spark has some so

Re: Pulsar CI congested, master branch build broken

Today Pulsar repo runs almost up to one worflow run at the same time. It's a new situation I didn't notice before. > drop the "required checks" This can be dangerous to the repo status. I think the essential problem we meet here is about prioritizing specific PR, instead of releasing the guard to

Re: Pulsar CI congested, master branch build broken

Here is another patch that can reduce unnecessary workload: https://github.com/apache/pulsar/pull/17529 We don't create flaky-test issues/PRs frequently; it's about tens in one month. The project owner should be able to handle it manually in minutes per month (since candidates are already labeled)

Re: Pulsar CI congested, master branch build broken

On 2022/09/07 17:27:45 tison wrote: > Today Pulsar repo runs almost up to one worflow run at the same time. It's > a new situation I didn't notice before. > > > drop the "required checks" > > This can be dangerous to the repo status. I think the essential problem we > meet here is about prioritiz

Re: [DISCUSS] Improvements on the release process

> If the PR can be cherry-picked directly, we should also check the > CI status of the branch after we push them directly. In order to catch basic compilation or checkstyle errors, I always run the following before pushing a cherry picked commit to an upstream branch: mvn -Pcore-modules,-main -T

[GitHub] [pulsar-client-node] jbmusso commented on issue #61: Support for Avro schema on `createProducer`

jbmusso commented on issue #61: URL: https://github.com/apache/pulsar-client-node/issues/61#issuecomment-1239747593 I was wondering the same. What would be required to add Avro support via a `schema` property? Thanks. -- This is an automated message from the Apache Git Service. To respon

[GitHub] [pulsar-client-node] Matt-Esch commented on issue #230: await flush() does not actually flush

Matt-Esch commented on issue #230: URL: https://github.com/apache/pulsar-client-node/issues/230#issuecomment-1239819312 If the async flush method isn't flushing correctly, this is likely to be an issue in the underlying pulsar library. I don't doubt that it's possible for the library to no

[GitHub] [pulsar-client-node] Matt-Esch commented on a diff in pull request #200: convert worker threads to async functions

Matt-Esch commented on code in PR #200: URL: https://github.com/apache/pulsar-client-node/pull/200#discussion_r965248019 ## src/Consumer.cc: ## @@ -243,160 +235,225 @@ class ConsumerReceiveWorker : public Napi::AsyncWorker { private: Napi::Promise::Deferred deferred; - p

[DISCUSS] Consumer reconnection causes repeated consumption messages

Hello, Pulsar community: Now the consumer does not filter messages that have already been consumed. After consumer reconnection, the broker will dispatch messages to the consumer from the markDeletePosition. In Failover and Exclusive subscription type, all messages in a topic will be dispatched t

[GitHub] [pulsar] gitfy created a discussion: Pulsar functions : python zip with deps wheel

GitHub user gitfy created a discussion: Pulsar functions : python zip with deps wheel Hi, I am trying setup a python zip based on the documentation. I have the attached wheel file in the deps folder which gets properly picked up for installing the dependencies. But my problem is ``` "Downlo

[GitHub] [pulsar] gitfy edited a discussion: Pulsar functions : python zip with deps wheel

GitHub user gitfy edited a discussion: Pulsar functions : python zip with deps wheel Hi, I am trying setup a python zip based on the documentation. I have the attached wheel file in the deps folder which gets properly picked up for installing the dependencies. But my problem is ``` "Downloa

[GitHub] [pulsar] trevordowdle added a comment to the discussion: Unsubscribe subscription [Rest API] working?

GitHub user trevordowdle added a comment to the discussion: Unsubscribe subscription [Rest API] working? Thank you, yes I will try this and get back to you and then I'll create an issue in the project to get the documentation fixed. Had a busy week and haven't got around to it yet, but will s

[GitHub] [pulsar-test-infra] nodece opened a new pull request, #70: Fix sync and remove labels

nodece opened a new pull request, #70: URL: https://github.com/apache/pulsar-test-infra/pull/70 ### Motivation 1. If different labels are selected multiple times, the previously selected labels cannot be deleted correctly. 2. When the committer wants to manually add the document la

Re: Pulsar CI congested, master branch build broken

My current assumption of the CI slowness problem is that the usage metrics for Apache Pulsar builds on GitHub side is done incorrectly and that is resulting in apache/pulsar builds getting throttled. This assumption might be wrong, but it's the best guess at the moment. The facts that support t

Re: [DISCUSS] Improvements on the release process

Good point, Michael. Compile errors happen quite often when cherry-picking commits, even though there is no conflict but compile still may fail due to some dependency PR missing, or when we try to resolve some simple conflicts. Thanks, Haiting On Thu, Sep 8, 2022 at 2:26 AM Michael Marshall wr

Re: [DISCUSS] Consumer reconnection causes repeated consumption messages

>From the user's perspective, I think we should always avoid delivering repeated messages. But can the broker tell if the reconnection is caused by topic unloading or consumer client process restarting? For the latter case, the message should be redelivered, it's the whole point of user explicit ac