Discussing negative acknowledgment in batch default behavior

egradation to it? Have you turned it on in your clusters? Thanks a lot for your time. Asaf Mesika

Re: Discussing negative acknowledgment in batch default behavior

Resurfacing this as it seems (to me) an important correction to be made. Would love your opinions on this. Thanks! Asaf On Wed, May 11, 2022 at 1:16 PM Asaf Mesika wrote: > Hello all, > > I have a question regarding a feature introduced in 2.6.0 called Negative > Acknowledgem

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

1. "Added NONE delivery semantics and delete autoAck config." - Added --> add 2. I suggest calling it `MANUAL` `ProcessingGuarantee` instead of NONE. As you carefully explained, ProcessingGuarantee comes does to the fact that the function executor calls acknowledge, in specific timing: - `AT_MO

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

ut some suggestions below, ptal. > > > > 1. API changes should also contain the changes of `Function.proto`, > including new `ProcessingGuarantees` option and `autoAck`. > > 2. Please be sure the other language runtimes (like Python, Golang) do > support similar `record.ack()`

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

luding new `ProcessingGuarantees` option and `autoAck`. > > 2. Please be sure the other language runtimes (like Python, Golang) do > support similar `record.ack()` function from the context, if no, it might > have some API changes for different runtime we well. > > > > > >

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

Still new to the PIP process: Shouldn't this PIP be added here ? On Thu, May 12, 2022 at 11:52 AM Baozi wrote: > Hi Pulsar community, > > I open a https://github.com/apache/pulsar/issues/15560 for Function add > NONE delivery se

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

; > 5. Regarding Test Plan > > * I would add: Validate the test of autoAck=false still works (you > haven't > > broken anything) > > * I would add: Validate existing ProcessingGuarantee test for AtMostOnce, > > AtLeastOnce, ExactlyOnce still works (when autoA

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

te the test of autoAck=false still works (you > haven't > > broken anything) > > * I would add: Validate existing ProcessingGuarantee test for AtMostOnce, > > AtLeastOnce, ExactlyOnce still works (when autoAck=true) > > > Nice, I added to PIP. > > > Thank

Re: [DISCUSS] PIP-166: Function add NONE delivery semantics

ia logger (WARN)? > > > I tend to fail. Although this breaks the current logic. but the current > implementation can be considered is a bug. > > > It will flood their logs if they used it wrong. Maybe write to log once? > > > Agree, I changed PIP. > > Thanks, >

Re: [VOTE] PIP-166: Function add MANUAL delivery semantics

+1 On Mon, Jun 6, 2022 at 4:04 AM Baodi Shi wrote: > Hi Pulsar Community, > > > I would like to start a VOTE on "Function add MANUAL delivery semantics" > (PIP-166). > > > The proposal can be read at https://github.com/apache/pulsar/issues/15560 > > and the discussion thead is available at > > h

Mark delete rate limiter bug?

Hi, In the Managed Cursor implementation, I saw that the position state is persisted in the ledger whenever a user calls acknowledge, be it individual or cumulative. For performance reasons there is a rate limiter, defaulting to 1 persist action per second. The bug: 17:00:00.003 - user ack a sin

Re: Mark delete rate limiter bug?

s). > # Default is 60 seconds > managedLedgerCursorPositionFlushSeconds=60 > > -- > Matteo Merli > > > On Wed, Jun 15, 2022 at 8:45 AM Asaf Mesika wrote: > > > > Hi, > > > > In the Managed Cursor implementation, I saw that the position state is > > persisted in the ledger w

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

Hi, I did a quick reading and I couldn't understand the gist of this change: The shadow topic doesn't really have it's own messages, or it's own ledgers right? When it reads messages, it reads from the original topic ledgers. So the only thing you need to do is sync the "metadata" - ledgers list?

Re: [DISCUSS] PIP-181: Pulsar Shell

Looks awesome. I was wondering if you have thought about another direction which is supplying a GraalVM compiled binary. This should take care of load time. So in effect, you can have a single command, which boots in microseconds, and has shell auto-complete? On Mon, Jul 4, 2022 at 4:34 PM Hang Ch

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

opic, so why replicate/sync messages? I managed to understand that you want to populate the BK entry cache of the topic ledgers in the shadow topic broker. Instead of reading from BK and storing it in the cache, you favor copying from the source topic broker cache memory to the shadow topic

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

to untangle. Thanks! On Tue, Jul 12, 2022 at 5:35 PM Haiting Jiang wrote: > Hi Asaf, > > On 2022/07/11 13:08:52 Asaf Mesika wrote: > > On Thu, Jun 23, 2022 at 6:26 AM Haiting Jiang > > wrote: > > > > > Hi Asaf, > > > > > > > I did a qu

Re: [DISCUSS] PIP-184: Cluster migration or Blue-Green cluster deployment support in Pulsar

Few questions "CompletableFuture asyncMigrate();" Does this method only change the status of the managed ledger? "message ManagedLedgerInfo { // Flag to check if topic is terminated and migrated to different cluster optional bool migrated = 4; }" This flag then is only changed to true wh

Re: [DISCUSS] PIP-184: Cluster migration or Blue-Green cluster deployment support in Pulsar

so, if a broker crashes then it knows about the > managed-ledger state. > So perhaps the flag means "migrationInProgress" or "migrationStarted"? On Wed, Jul 13, 2022 at 7:23 PM Rajan Dhabalia wrote: > On Wed, Jul 13, 2022 at 1:55 AM Asaf Mesika wrote: > >

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

and having no other solution. On Sun, Jul 17, 2022 at 5:47 PM Haiting Jiang wrote: > Hi Asafa, > > On 2022/07/13 08:15:59 Asaf Mesika wrote: > > Thanks for the detailed answer. I have a few follow-up questions: > > > > 1. Can you clarify how data is flowing exactly

Re: PIP-187 Add API to analyse a subscription backlog and provide a accurate value

I'm not sure I understand the context exactly: You say today we can only know the number of entries, hence we'll have a wrong number of backlog for subscription since: 1. One entry contains multiple messages (batch message) 2. Subscription may contain a filter, which requires you to read the entir

Re: [VOTE] PIP-187 Add API to analyze a subscription backlog and provide a accurate value

Sorry to barge in the vote - I forgot to send my reply on the discussion 2 days ago :) On Tue, Jul 19, 2022 at 11:22 PM Nicolò Boschi wrote: > +1, thanks > > Nicolò Boschi > > Il Mar 19 Lug 2022, 22:16 Christophe Bornet ha > scritto: > > > +1 > > > > Le mar. 19 juil. 2022 à 20:01, Andrey Yegor

Re: PIP-187 Add API to analyse a subscription backlog and provide a accurate value

On Wed, Jul 20, 2022 at 5:46 PM Enrico Olivelli wrote: > Asaf, > > Il giorno mer 20 lug 2022 alle ore 15:40 Asaf Mesika > ha scritto: > > > > I'm not sure I understand the context exactly: > > > > You say today we can only know the number of entrie

[DISCUSS] Alternatives to changing public protocol

Hi, We started discussing in PIP-180, which Penghui recommended I move to a dedicated thread. Pulsar has a public API in its binary protocol, which the clients use to communicate with it. Nonetheless, it is its public API to the server. I believe the public API should not be changed for internal

Re: PIP-187 Add API to analyse a subscription backlog and provide a accurate value

f we want to expose the accurate backlogs in the Prometheus endpoint, > > it's almost impossible. > > I don't think this is actually possible if you want to take into > consideration the filters. > We are in the case of general purpose filtering (actually we allow >

Re: Pulsar Flaky test report 2022-07-19 for PR builds in CI

Fantastic initiative. On Thu, Jul 21, 2022 at 10:59 PM Nicolò Boschi wrote: > Dear community, > > Here's a report of the flaky tests in Pulsar CI during the observation > period of 2022-07-12 - 2022-07-19 > > > https://docs.google.com/spreadsheets/d/165FHpHjs5fHccSsmQM4beeg6brn-zfUjcrXf6xAu4yQ/

Re: [DISCUSS] PIP 193 : Sink preprocessing Function

Hi, Few questions: >- preprocess-function: the preprocess function applied before the >Sink. Starts by builtin:// for built-in functions, function:// for >package function, http:// or file:// > > 1. While this function is applied only before sink? I thought it replaces the identity f

Re: [Discuss] PIP-190: Simplify Pulsar documentation release and maintenance strategy

If I understand correctly, using tags, you can automatically create the docs for the next minor version? So for older minor versions, where those files will be at? using tags from git? On Sat, Jul 23, 2022 at 7:04 AM Ma Jun wrote: > Hi, Pulsar community, > > Happy weekend! > > I'd like to open a

Re: [DISCUSS] Introduce FlowControl to metrics endpoint

Hi Jiuming, I would reiterate the problem statement to make it clear (at least for me): There are cases where a very large amount of topics exists (> 10k per broker) and are used in Pulsar. Those topics usually have multiple producers and multiple consumers. There are metrics that are in the gran

Re: [DISCUSS] Move PIPs to the codebase?

I like the idea of keeping the suggestions as files in the repo since as you mentioned, it makes the review process using PRs much more streamlined. I think keeping the status in an MD file and only there (having a single source of truth) will make it less error-prone (people might forget to move

Re: [DISCUSS] Move PIPs to the codebase?

> after approved. I don't want to say all the changes to the proposal need > > to be voted on > > the mailing list, but we should get a chance to review the changes. > > > > Thanks, > > Penghui > > > > On Mon, Aug 29, 2022 at 10:45 PM Asaf Mesika

Re: [DISCUSS] Move PIPs to the codebase?

to say all the changes to the proposal need to > be voted on > the mailing list, but we should get a chance to review the changes. > > Thanks, > Penghui > > On Mon, Aug 29, 2022 at 10:45 PM Asaf Mesika > wrote: > > > I like the idea of keeping the suggestions as fi

Re: [DISCUSS] Introduce FlowControl to metrics endpoint

efit of this could be that /metrics endpoint wouldn't be a > bottleneck and there wouldn't be a need to do any hacks to support a high > number of metrics. > There might be need to route the metrics for different namespaces/topics to > different destinations. This could be handled

Re: [DISCUSS] Alternatives to changing public protocol

sidered a special producer client , and it just inherited the basic semantic of a normal producer and extended its abilities to support some special internal usage". Replace X with any feature, thereby expanding the public API more and more with internal fields the normal user should never kn

[DISCUSS] General question regarding PIP comments

Hi, I'm fairly new to this project and I have seen 2 options to comment to a PIP: 1. In the GitHub issue. 2. In the mailing list. I was wondering what is the best practice. I know there is an ongoing initiative at the moment to switch a PIP to become a file and actually a PR for submitting it. I

Re: [DISCUSS] User-friendly acknowledgeCumulative API on a partitioned topic or multi-topics

What eventually happened with this idea? On Fri, Jul 29, 2022 at 8:02 AM PengHui Li wrote: > +1 > > Penghui > On Jul 28, 2022, 20:14 +0800, lordcheng10 <1572139...@qq.com.invalid>, > wrote: > > Nice feature! > > > > > > > > > > -- Original -- > > From: "Yunze Xu"

Re: [DISCUSS] ARM Support for Pulsar 2.11 Docker Image

What ended up for this? On Tue, Aug 9, 2022 at 11:11 AM Alexander Preuss wrote: > Hi Michael, > > Thank you for bringing up this topic. > I was just running into an issue that prevented me from using the standard > Pulsar image in Testcontainers and found this discussion. > > In my opinion, refa

Re: [DISCUSS] PIP 194 : Pulsar client: seek command add epoch

Hi Qiang, We have a brainstorming session on this PIP over Zoom with Penghui, Hang, and more people, and I'm jotting down here our feedback. Before I do that, I just want to write my own understanding of the document, for other readers: # Context Pulsar, as opposed to other distributed / streami

Re: [DISCUSS] PIP-264: Enhanced OTel-based metric system

. > > > > > > -- > > Matteo Merli > > > > > > > > On Wed, Jun 14, 2023 at 6:10 PM Devin Bost wrote: > > > > > > Thanks for the details, Devin. Curios - 'We' stands for which > company? > > > > > > Wh

Re: [DISCUSS] PIP-264: Enhanced OTel-based metric system

I've relocated the PIP content from the issue ( https://github.com/apache/pulsar/issues/20197) to a PR ( https://github.com/apache/pulsar/pull/21080) so I could add TOC and also be inlined with the new process. On Mon, Aug 28, 2023 at 5:46 PM Asaf Mesika wrote: > Thanks for taking the

[VOTE] PIP-264: Enhanced OTel-based metric system

Hi, I'm very happy to start the vote process for PIP-264. PIP is located at https://github.com/apache/pulsar/pull/21080. The PIP was at the discussion stage from April 27th (~4 months). I want to express my sincere gratitude to Matteo, Hang and Larry for taking the time to read through the *enti

Re: [VOTE] PIP-264: Enhanced OTel-based metric system

Thank you all for your review and corresponding votes. The PIP vote has passed with 3 binding +1 votes by Matteo, Lari and Hang. On Wed, Aug 30, 2023 at 3:40 PM Lari Hotari wrote: > +1 (binding) > > -Lari > > On Mon, Aug 28, 2023 at 5:55 PM Asaf Mesika wrote: > > > Hi

Re: [DISCUSS] Consistent code style (esp. ws/indent) and autotools

Opentelemetry-java employs both spotless for coding style You run "./gradlew spotlessCheck" and it shows the problems. You run "./gradlew spotlessApply" to automatically fix it. It also uses errorprone to detect bugs. I wonder if we can run it only in GitHub PRs, so we can instruct it to run only

Re: [DISCUSS] Consistent code style (esp. ws/indent) and autotools

correction > >> seems like a waste of volunteer energy. > >> > >> Best, > >> Dave > >> > >> Sent from my iPhone > >> > >>>> On Aug 31, 2023, at 9:05 PM, Zixuan Liu wrote: > >>> > >>> This idea

Re: [DISCUSS] Consistent code style (esp. ws/indent) and autotools

huge change, but to be more friendly to > developers, > > we should document the workarounds for the git blame issue. And we > should apply > > the spotless tool to every active branches. > > > > > On Sep 3, 2023, at 19:43, Asaf Mesika wrote: > > > > >

Re: [DISCUSS] PIP-309: Adding Pulsar Client Stats Reporter

I've replied in the PR it self On Fri, Oct 20, 2023 at 2:24 AM Ying wrote: > Hi dev, > > Currently, Pulsar Client can provide recorded stats for both Producer and > Consumer, but not all stats are fixed values during the statsInterval. So > start the PIP-309 to add the Pulsar Client Stats Report

Re: [DISCUSS] PIP-310: Support custom publish rate limiters

Replied in PR. On Thu, Oct 19, 2023 at 3:51 PM Girish Sharma wrote: > Hi, > Currently, there are only 2 kinds of publish rate limiters - polling based > and precise. Users have an option to use either one of them in the topic > publish rate limiter, but the resource group rate limiter only uses

Re: [DISCUSS] Roll up project status for pulsar-helm-chart

Tison, can we mark this repo as suggested? On Tue, Aug 8, 2023 at 12:24 PM Matteo Merli wrote: > Thanks Tison, > > I fully agree that we should have a clear representation of the actual > status of the Helm chart, so that users can have the correct expectation. > > In particular I think we shoul

Re: Reporting and tooling to detect thread leaks in Pulsar tests

Larry, I know there is a way to add like a Job summary, so we can write it there - do you think this can increase visibility? On Sun, Oct 29, 2023 at 4:53 AM Lari Hotari wrote: > Hi all, > > I have submitted a PR (https://github.com/apache/pulsar/pull/21450) which > includes changes to add repor

Re: Reporting and tooling to detect thread leaks in Pulsar tests

e last few weeks. Without support for > detecting > > > the resource leaks, it's really hard to keep the test suite clean. > > > > > > > > > > > > Looking forward to more reviews on > > > https://github.com/apache/pulsar/pull/21450

Re: [DISCUSS] Replace stale bot with ping-pong workflow

Tison let's start as you suggested by disabling it On Tue, May 16, 2023 at 5:13 AM Yunze Xu wrote: > +1 to me > > Thanks, > Yunze > > On Sun, May 14, 2023 at 9:28 PM Dave Fisher wrote: > > > > Hi - > > > > I have not looked at all your links but I think this is a great idea. > This will help e

Re: [DISCUSS] PIP-310: Support custom publish rate limiters

I just want to add one thing to the mix here. You can see by the amount of plugin interfaces Pulsar has, somebody "left the door open" for too long. You can agree with me that the number of those interfaces is not normal for any open source software. I know HBase for example, or Kafka - never seen

Re: [DISCUSS] Replace stale bot with ping-pong workflow

Submitted a PR to disable it: https://github.com/apache/pulsar/pull/21549 On Tue, Nov 7, 2023 at 3:58 PM Asaf Mesika wrote: > Tison let's start as you suggested by disabling it > > > On Tue, May 16, 2023 at 5:13 AM Yunze Xu wrote: > >> +1 to me >> >> Thank

Re: [DISCUSS] Introducing Apache Pulsar Office Hour Meetings

I think it's a wonderful idea. Since community engagement in the community meetings are quite low (1-3 people top), hopefully we can engage more users through those open office hours. On Fri, Nov 17, 2023 at 5:11 AM mattison chao wrote: > Dear Apache Pulsar Community, > > I hope you are doing

Re: [DISCUSS] Introducing Apache Pulsar Office Hour Meetings

I believe Mattisson wanted some feedback/suggestion before proceeding to the announcement to the users via Slack and website On Mon, Nov 20, 2023 at 5:42 PM Julien Jakubowski wrote: > Thank you for this. This will be a huge benefit to the user community! > Could you please share this on the co

[DISCUSS] PIP-320: OpenTelemetry Scaffolding

Hi, This is the first sub-PIP for parent PIP-264 ("Enhanced OTel-based metric system"). This PIPs goal is to introduce OpenTelemetry into Apache Pulsar. When this PIP is implemented, we will be able to start converting (not replacing) existing metrics

Re: [DISCUSS] PIP-320: OpenTelemetry Scaffolding

On Wed, Nov 29, 2023 at 12:18 AM Enrico Olivelli wrote: > Asaf, > > > > Il Mar 28 Nov 2023, 19:14 Asaf Mesika ha scritto: > > > Hi, > > > > This is the first sub-PIP for parent PIP-264 > > <https://github.com/apache/pulsar/pull/21080> ("En

Re: [DISCUSS] Introducing Apache Pulsar Office Hour Meetings

e: > >>> > >>> Hello, > >>> this sounds like a good idea. > >>> It is not clear to me who is going to run these meetings (also when > and how) > >>> > >>> Can you please share some more context ? Maybe I missed something &g

[VOTE] PIP-320: OpenTelemetry Scaffolding

into OpenTelemetry. Thank you! Asaf Mesika

[DISCUSS] PIP-323: Complete Backlog Quota Telemetry

picks up PIP-248 <https://github.com/apache/pulsar/issues/19601> which was left stale and unapproved, improves it a bit and will implement once approved. Thanks, Asaf Mesika

[VOTE] PIP-323: Complete Backlog Quota Telemetry

Hi, I'm starting the vote for PIP-323, since it has been reviewed by several people and all comments have been resolved. Reminder: PIP-323 is introduced to fill the gap of backlog quota telemetry. It allows the user to know when a time-based backlog quota is about to exceed, and how many times i

Re: [VOTE] PIP-320: OpenTelemetry Scaffolding

n-binding) > > > > On Mon, Dec 11, 2023 at 4:50 AM Lari Hotari wrote: > > > > > +1 (binding) > > > > > > -Lari > > > > > > On 2023/12/11 07:33:53 Asaf Mesika wrote: > > > > Hi, > > > > > > &g

Re: [VOTE] PIP-323: Complete Backlog Quota Telemetry

The vote is now closed. The PIP is approved, with 4 +1 binding votes, by Yubaio, Mattison, Penghui and Guo. On Mon, Dec 18, 2023 at 4:55 PM mattison chao wrote: > +1(binding) > > Best, > Mattison > > > On Dec 13, 2023, at 16:22, Asaf Mesika wrote: > > > >

Re: [VOTE] PIP-330: getMessagesById gets all messages

+1 (non-binding) On Tue, Jan 16, 2024 at 4:43 AM Dezhi Liu wrote: > +1 (non-binding) > > Thanks, > Dezhi Liu > > On 2024/01/15 09:33:48 Zixuan Liu wrote: > > Hi Pulsar Community, > > > > Voting for PIP-330: getMessagesById gets all messages > > > > PIP: https://github.com/apache/pulsar/pull/2187

Re: Ability to decrease partition count in pulsar

Hey Girish, First, I say that I *love* this proposal and, in general, those types of proposals. This is what strides Pulsar towards being an even more next-generation messaging system. I read and have a few questions and brainstorming ideas popping into my mind: 1. The current design basically s

Re: [DISCUSS] PIP-331: WASM Function API

Hi ZiCheng, Brilliant suggestion! I replied in the PR section, which I couldn't understand. On Tue, Jan 30, 2024 at 1:18 PM dragon-zhang wrote: > Hi Pulsar Community, > > I want to add a new feature that supports run WASM bytecode to the > pulsar-functions module. > > Please see the PIP: http

PIP-264: Implementation status update

Hi, PIP-264 (approved Sep 2023) fixes many pitfalls of the current metric system inside Pulsar. We partly do so by adding another option of adding metrics: OpenTelemetry Java SDK, and then redefining the existing metrics using OTel. The first part was adding OpenTelemetry Java SDK into Pulsar in

Re: [ANNOUNCE] New Committer: Kiryl Valkovich

Congrats! You are definitely a great addition to Pulsar committers! On Thu, Feb 29, 2024 at 4:58 AM Zixuan Liu wrote: > Congrats! > > Lari Hotari 于2024年2月27日周二 14:14写道： > > > Congrats, Kiryl! > > > > -Lari > > > > On Tue, 27 Feb 2024 at 06:53, tison wrote: > > > > > > The Apache Pulsar Project

Re: [DISCUSS] Clarify the relation between supported Pulsar versions and versioned docs

I personally agree it is frustrating to update in multiple places. It's not time consuming, just annoying. Maybe we can only update for the support LTS version and onwards? On Mar 6 2024, at 1:56 pm, Kiryl Valkovich wrote: > Idea: don't require updating versioned docs from contributors. > Making

Re: (Apache committer criteria) [ANNOUNCE] New Committer: Asaf Mesika

> > I have selected these examples from the most recent discussions and they > may not be the best examples to fully illustrate the point. You should respect people's time by taking the time to craft your replies in this discussion. >From my experience, I've been contributing a lot in recent year

Re: [VOTE] PIP-342: Support OpenTelemetry metrics in Pulsar client

+1 (non-binding) On Thu, Mar 14, 2024 at 8:29 PM Apurva Telang wrote: > +1 (non-binding) > > On Thu, Mar 14, 2024 at 2:12 AM mattison chao > wrote: > > > +1 (binding) > > > > Best, > > Mattison > > On Mar 14, 2024 at 15:55 +0800, Lari Hotari , wrote: > > > +1 (binding) > > > > > > -Lari > > > >

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

Hi Haiting, I've noticed Shadow Topic is *not* covered in the documentation. Can you please add documentation for it? It's such a great feature, it's almost a waste not having it documented. Once it is, I can it to the proud list of features Pulsar have in this lovely page we recently launched:

Re: [DISCUSS] PIP-184: Cluster migration or Blue-Green cluster deployment support in Pulsar

e.org/features/ On Thu, Jul 14, 2022 at 8:13 PM Rajan Dhabalia wrote: > On Thu, Jul 14, 2022 at 9:59 AM Asaf Mesika wrote: > > > > > > > No, as it's mentioned in PIP: this API will terminate the topic so, it > > will > > > not allow any new write and p

Re: Preparing for Pulsar 4.0: cleaning up the Managed Ledger interfaces

In high level I see this work as needed cleanup indeed and very much needed On Tue, 11 Jun 2024 at 20:29 Lari Hotari wrote: > Hi all, > > We have the next LTS release, Pulsar 4.0, scheduled for October. The > current master branch will most likely become the 4.0 release branch in > September. >

Re: [ANNOUNCE] Apache Pulsar 2.8.4 released

I don't see the release notes in the provided link for 2.8.4 On Mon, Sep 19, 2022 at 11:59 AM Yunze Xu wrote: > The Apache Pulsar team is proud to announce Apache Pulsar version 2.8.4. > > Pulsar is a highly scalable, low latency messaging platform running on > commodity hardware. It provides si

[DISCUSS] Pulsar Metrics - Current State and Future Directions

and if there is any other problem missing or existing functionality missed as it serves as the basis for the requirements for the solution that will be chosen. Thanks! Asaf Mesika Document link: https://docs.google.com/document/d/1vke4w1nt7EEgOvEerPEUS-Al3aqLTm9cl2wTBkKNXUA/edit?usp=sharing

Re: [DISCUSS] Add an auth data const for refresh the original auth data

I kind of lost you here Zixuan. Is there any chance you describe the authentication flow including the commands? Then specify the change you wish to make? On Thu, Aug 18, 2022 at 5:21 AM Zixuan Liu wrote: > A little confusing here is that both ProxyClient and DirectProxyHandler > pass original

Re: [DISCUSS] Pulsar Metrics - Current State and Future Directions

existing and new > contributors adjust to the new style. > > Thanks, > Michael > > [0] https://github.com/apache/pulsar/issues/12069 > > On Mon, Oct 3, 2022 at 3:36 AM Asaf Mesika wrote: > > > > Hi All, > > > > I would like to share with y

Re: [PIP-214][broker]Add broker level metrics statistics and expose to prometheus discussion

The suggestion makes sense to me as well. I've also reviewed your PR. On Sun, Oct 23, 2022 at 8:43 AM Haiting Jiang wrote: > +1 > Makes sense to me. > > Thanks, > Haiting > > On Sat, Oct 22, 2022 at 3:59 AM yang yijun wrote: > > > > Hi,I have a suggestion to improve the broker. > > > > For deta

Re: [PIP-214][broker]Add broker level metrics statistics and expose to prometheus discussion

One thing to note: You mentioned a PIP number but I'm not sure this is the right process you follow here for that. On Sun, Oct 23, 2022 at 1:12 PM Asaf Mesika wrote: > The suggestion makes sense to me as well. I've also reviewed your PR. > > On Sun, Oct 23, 2022 at 8:

Re: [DISCUSS] Remove timestamp from Prometheus metrics

Great addition! It also reduces the Prometheus response size. On Wed, Sep 7, 2022 at 7:47 AM Michael Marshall wrote: > Merged. Depending on whether [0] will be cherry picked to release > branches, I will cherry pick [1] or [2] to all active release > branches. > > Thanks, > Michael > > [0] https

Pulsar in OpenSSF?

Hi, I recently bumped into the Pulsar page in the Scorecards project of OpenSSF. I was wondering if anybody has any context/insight on that. Thanks, Asaf

Re: [DISCUSS] PIP-218: Consumer batchReceive() single partition every receive

Regarding the implementation suggested in the PIP: It said that you will read the messages from all partitions assigned, and once you see a message from a different partition you halt and supply the batch. My question is: Why can't we simply read the messages from one partition (say 10 messages) a

Re: [PIP-214][broker]Add broker level metrics statistics and expose to prometheus discussion

ced how to deal with the empty label [1] > > > > If this can work. We don't need to add new indicator names. > > Otherwise, we might need "pulsar_tenant_*" ,"pulsar_namespace_*", > > "pulsar_cluster_*" in the future. > > > > [0

Re: [DISCUSS] PIP-218: Consumer batchReceive() single partition every receive

Ok. I would list what you wrote in your original PIP for context. What is your future improvement you plan? On Thu, Nov 3, 2022 at 11:46 AM 丛搏 wrote: > Good question! > It is a scene for our subsequent optimization, and we are implementing > it with minimal changes. > > Asaf Me

Re: [DISCUSS] General question regarding PIP comments

2022 at 5:07 PM Enrico Olivelli wrote: > Asaf, > good question! > > Il giorno gio 1 set 2022 alle ore 15:01 Asaf Mesika > ha scritto: > > > > Hi, > > > > I'm fairly new to this project and I have seen 2 options to comment to a > > PIP: > >

Re: [DISCUSS] PIP-222: Add CommandPartitionedTopicMetadata metrics

Hi Jiuming, I have a few questions: Motivation > > Currently, there's no way to track CommandPartitionedTopicMetadata > requests. There's no metrics that indicate that a broker is handling > CommandPartitionedTopicMetadata requests. > > Misconfigured clients might flood brokers with > CommandPart

Re: [DISCUSS] The use of consumer redeliverUnacknowledgedMessages method

Ok, I'll try to summarize what I read here to make sure we're all on the same page :) Exclusive and Failover subscription types are subscriptions that guarantee two things: 1. Single active consumer per topic (partition). 2. Message processing in the order they were written to the topic (partition

Re: [DISCUSS] Pulsar 3.0 - dreaming of the ability to rename objects

I don’t have a solution to your problem, but I can share some thoughts I had when reading your suggestion: (1) Simplicity I’m relatively new to Pulsar, and I try to learn it as I go. One thing I can say for sure: Pulsar is complicated, both user-facing and internals. Just the other way, I read a

Re: [DISCUSS] Back pressure in Apache Pulsar

Mattison, can you provide more context here? 1st link describes a complicated situation in which the BK node Pulsar is working against is overwhelmed, and we would like to transmit this back-pressure back to consumers/producers. This creates an issue as you need to notify the load balancer of this

Re: [PROPOSAL] Website precommit and move the source of docs to the site repo

> > We don't have a pre-commit workflow for any website-related changes. > Thus, we don't detect broken links or syntax errors when reviewing new > patches[4][5][6]. When you say "pre-commit work-flow,” do you mean what exactly? Usually, a pre-commit is something you run locally upon committing. D

Re: [DISCUSS] The use of consumer redeliverUnacknowledgedMessages method

ace holding those > > commands above. > > > It's a transformative idea. I'd like +1. See what other contributors think. > > > 2022年11月30日 00:19，Asaf Mesika 写道： > > > Ok, I'll try to summarize what I read here to make sure we're all on the > > sa

Re: [DISCUSS] General question regarding PIP comments

I submitted a PR for this here: https://github.com/apache/pulsar/pull/18974 On 1 Sep 2022 at 17:07:29, Enrico Olivelli wrote: > Asaf, > good question! > > Il giorno gio 1 set 2022 alle ore 15:01 Asaf Mesika > ha scritto: > > > Hi, > > > I'm fairly

Re: Pulsar in OpenSSF?

Bumping to verify it wasn’t missed On 24 Oct 2022 at 22:06:36, Asaf Mesika wrote: > Hi, > > I recently bumped into the Pulsar page > <https://metrics.openssf.org/grafana/d/default/metric-dashboard?orgId=1&var-PackageURL=pkg%3Agithub/apache/pulsar> > in the Scorecards

Re: [DISCUSS] PIP 194 : Pulsar client: seek command add epoch

P connection > > (with its adverse effects on other consumers and produces "riding" on that > > connection) > > c. Reset the cursor to the newly requested position. > > d. Continue dispatching messages from newly requested positions to > > consumers.

Re: [DISCUSS] PIP-231: Add metric for topic load failed

I have several context-related questions: 1. When a topic fails to load, what happens exactly at that stage? Does it switch over to another broker? Is there a retry on the current broker? 2. Is there a way to provide a gauge showing how many topics are unloaded due to non-recoverable

Re: [DISCUSS][doc][improve] Update Pulsar site landing page content #349

I'm pasting the context described in the PR also here: Pulsar’s landing page has a key role in getting new people “onboard” Apache Pulsar - effectively getting people from “Pulsar? Never heard about it, what is it good for?” to “Wow, that’s an amazi

Re: [DISCUSS] Registering Jackson Java 8 support modules by default for all Pulsar components, including client

Can the scenarios of breaking changes for users be described in the PIP if they exist? If this is a breaking change, it might be a good thing to point it out in the PIP and in the release notes ("If you did this, then perhaps you want to do that"). Context question since I'm not familiar with func

Re: [DISCUSS] Make the default value of param "--get-subscription-backlog-size" of admin API "topics stats" true

Small question regarding this: The code for calculation is: long estimateBacklogFromPosition(PositionImpl pos) { synchronized (this) { long sizeBeforePosLedger = ledgers.headMap(pos.getLedgerId()).values() .stream().mapToLong(LedgerInfo::getSize).sum(); LedgerI

Re: [DISCUSS] PIP-239: Retry Mechanism per Message

>From my knowledge, when a client does negative ack, after a certain amount of time, it sends the following command to the broker containing the negatively acked messages ids: message CommandRedeliverUnacknowledgedMessages { required uint64 consumer_id = 1; repeated MessageIdData message_i

Re: [DISCUSS] Redundant ServiceUrlProvider design and improper use of PIP-121

and the client will poll the ConnectInfoProvider and check if something was changed? On Fri, Jan 20, 2023 at 11:19 AM Yunze Xu wrote: > > I think the `updateServiceUrl` is not the initial purpose of exposing to > the Client API. > > I agree. We might need an API like > > ```java > ClientBuilder

  1   2   3   >