Re: [Vote] PIP-245: Make subscriptions of non-persistent topic non-durable

Bump > 2023年2月13日 14:56，Jiuming Tao <mailto:jm...@streamnative.io>> 写道： > > Hi all, > > I would like to start a VOTE on `PIP-245: Make subscriptions of > non-persistent topic non-durable`. > > Motivation: > > There are two types of subscriptions for a

Re: [Vote] PIP-245: Make subscriptions of non-persistent topic non-durable

t; >> Thanks, >> Baodi Shi >> >> >> 在 2023年2月16日 21:44:17 上，Asaf Mesika 写道： >> >>> +1 (non-binding) >>> >>> >>> On Thu, Feb 16, 2023 at 11:11 AM Jiuming Tao >>> >>> wrote: >>> >>> >>> I’v

Re: [Vote] PIP-245: Make subscriptions of non-persistent topic non-durable

about compatibility in more detail? WDYT? > Then we could start the vote again. > > [0] https://lists.apache.org/thread/2bjg39zh7z38bzbnqngbo5l4jzkjttrq > > Thanks, > Zike Yang > > On Wed, Feb 15, 2023 at 1:34 PM Tao Jiuming wrote: >> >> >> bump >>

Re: [DISCUSS] PIP-245: Subscriptions expiration for NonPersistentTopic only

ote: >> >>> Agree，+1 >>> >>> >>> Regards >>> Jiwei Guo (Tboy) >>> >>> On Thu, Feb 9, 2023 at 7:07 PM Jiuming Tao >>> wrote: >>>> >>>> >>>> It makes sense >>>> >>&

[Vote] PIP-245: Make subscriptions of non-persistent topic non-durable

Hi all, I would like to start a VOTE on `PIP-245: Make subscriptions of non-persistent topic non-durable`. Motivation: There are two types of subscriptions for a topic: Durable and Non-durable. We create a Consumer with a Durable subscription and a Reader with a Non-durable subscription. But

Re: [DISCUSS] PIP-245: Subscriptions expiration for NonPersistentTopic only

non-durable to > subscribe.” > > > > Thanks, > Baodi Shi > > 在 2023年2月9日 15:10:12 上，Jiuming Tao 写道： > >> Agree. But we need to take care of the compatibility. How do we handle >> >> this compatibility? >> >> >> How ab

Re: [DISCUSS] PIP-245: Subscriptions expiration for NonPersistentTopic only

, for non-persistent topic, it should not create a durable >> subscription. > > Agree. But we need to take care of the compatibility. How do we handle > this compatibility? > > BR, > Zike Yang > > On Thu, Feb 9, 2023 at 2:43 PM Jiuming Tao > wrote: >> >>

Re: [DISCUSS] PIP-245: Subscriptions expiration for NonPersistentTopic only

eyDispatcherMultipleConsumers`, I’m not sure that >> if it matters when we auto delete the subscriptions which has no >> consumers connected. >> >> It doesn't matter IMO. >> >> Thanks, >> Zike Yang >> >> >> >> Zike Yang >

Re: [DISCUSS] PIP-245: Subscriptions expiration for NonPersistentTopic only

out if there is a strong reason to leave the durable subscription > after > all the consumers are disconnected. If there is no strong reason, I tend to > fix > incorrect behavior directly. > > Best, > Penghui > > > > > On Tue, Feb 7, 2023 at 4:14 PM Jiu

Re: [DISCUSS] PIP-245: Subscriptions expiration for NonPersistentTopic only

ve the durable subscription > after > all the consumers are disconnected. If there is no strong reason, I tend to > fix > incorrect behavior directly. > > Best, > Penghui > > > > > On Tue, Feb 7, 2023 at 4:14 PM Jiuming Tao <mailto:jm...@streamnative.i

[DISCUSS] PIP-245: Subscriptions expiration for NonPersistentTopic only

Hi, I’ve opened a new PIP to discuss: Currently, in Pulsar, we have a configuration named `subscriptionExpirationTimeMinutes` to manage the subscription expiration of `PersistentTopic` and `NonPersistentTopic`. When we set a value which is greater than 0 to `subscriptionExpirationTimeMinutes`

Re: [ANNOUNCE] New Committer: Baodi Shi

Congrats > 2023年1月18日 21:35，Yunze Xu 写道： > > The Project Management Committee (PMC) for Apache Pulsar has invited > Baodi Shi (https://github.com/shibd) to become a committer and we are > pleased to announce that he has accepted. > > Being a committer enables easier contribution to the project

Re: [ANNOUNCE] Bo Cong as new PMC member in Apache Pulsar

Congrats！ > 2023年1月18日 21:50，PengHui Li 写道： > > Hi all, > > The Apache Pulsar Project Management Committee (PMC) has invited Bo Cong > (https://github.com/congbobo184) as a member of the PMC and we are > pleased to announce that he has accepted. > > He is very active in the community in the pa

Re: [DISCUSS] PIP-235: Add metric for subscription back size

Hi Xiao, Seems `backlogSize` already existed in the `SubscriptionStatsImpl`, but didn’t expose in Prometheus, We just need to expose it in Prometheus. It should exposed in Prometheus when the configuration named `exposeTopicLevelMetricsInPrometheus` and `exposeSubscriptionBacklogSizeInPrometh

Re: [Vote] PIP-231: Add metric for topic load failed

bump Jiuming Tao 于2022年12月27日周二 16:27写道： > Hello pulsar community, > > I'm starting the VOTE for PIP-231: Add metric for topic load failed. > > Motivation: > Currently, we have a `topic_load_times` metric to track how long a topic > load succeeds. > But when loadi

[Vote] PIP-231: Add metric for topic load failed

Hello pulsar community, I'm starting the VOTE for PIP-231: Add metric for topic load failed. Motivation: Currently, we have a `topic_load_times` metric to track how long a topic load succeeds. But when loading a topic, there may be some chances the topic load failed due to MetadataStore or Bookke

Re: [DISCUSS] PIP-232: Introduce thread monitor to check if thread is blocked for long time.

With this proposal, I just remembered something. Apache SkyWalking provided a mechanism which called: Thread Profiling The core of the mechanism is start a schedul

Re: [DISCUSS] PIP-232: Introduce thread monitor to check if thread is blocked for long time.

+1 It’s a valuable feature that could help us to improve the stability of pulsar Thanks, Tao Jiuming > 2022年12月19日 20:17，adobewjl 写道： > > Hello pulsar community, > I've opened `PIP-232: Introduce thread monitor to check if thread is blocked > for long time.` to discuss. > For more details, ple

Re: [DISCUSS] Proposal required for Admin API/CLI and metrics changes

+1 The metrics, CLI, Admin API changes will directly face our users. I think a proposal will ask the whole community to consider the rationality of modifying it Not only the people in review know it Thanks, Tao Jiuming > 2022年12月7日 11:38，PengHui Li 写道： > > Hi all, > > I would like to start

Re: [DISCUSS] PIP-231: Add metric for topic load failed

nsient error thus I’m > alerted on something less important? How do you see this in production I > guess is the question. > > > Thanks! > > Asaf > > On 19 Dec 2022 at 10:19:39, Jiuming Tao > wrote: > >> Hello pulsar community, >> >> I

[DISCUSS] PIP-231: Add metric for topic load failed

Hello pulsar community, I've opened `PIP-231: Add metric for topic load failed` to discuss. Motivation: Currently, we have topic_load_times metric to tracking how long a topic load succeed. But when loading a topic, there are

Re: [DISCUSS] PIP-223: Add metrics for all Rest Endpoints

erRestEndpointMetrics" to > "exposePerRestEndpointMetricsInPrometheus"? > > Thanks, > Michael > > On Thu, Nov 24, 2022 at 11:30 PM Haiting Jiang > wrote: > > > > +1, Great work. > > > > > > Haiting &g

Re: [DISCUSS] PIP-223: Add metrics for all Rest Endpoints

"enablePerRestEndpointMetrics" to > "exposePerRestEndpointMetricsInPrometheus"? > > Thanks, > Michael > > On Thu, Nov 24, 2022 at 11:30 PM Haiting Jiang > wrote: > > > > +1, Great work. > > > > > > Haiting &g

Re: [Vote] PIP-223: Add metrics for all Rest Endpoints

Michael > > On Thu, Dec 8, 2022 at 12:39 AM Jiuming Tao > wrote: > > > > bump, need one more binding > > > > Jiuming Tao 于2022年11月28日周一 16:47写道： > > > > > Dear Pulsar Community, > > > > > > Please review and vote on this PIP. > &g

Re: [Vote] PIP-223: Add metrics for all Rest Endpoints

bump, need one more binding Jiuming Tao 于2022年11月28日周一 16:47写道： > Dear Pulsar Community, > > Please review and vote on this PIP. > > PIP link: https://github.com/apache/pulsar/issues/18560 > > Discuss thread: > https://lists.apache.org/thread/z74vcn0yolzzrcc4ftonm9j3nb

Re: [Vote] PIP-223: Add metrics for all Rest Endpoints

bump Jiuming Tao 于2022年11月28日周一 16:47写道： > Dear Pulsar Community, > > Please review and vote on this PIP. > > PIP link: https://github.com/apache/pulsar/issues/18560 > > Discuss thread: > https://lists.apache.org/thread/z74vcn0yolzzrcc4ftonm9j3nbk4pzxm > > Thanks, > Tao Jiuming >

[Vote] PIP-223: Add metrics for all Rest Endpoints

Dear Pulsar Community, Please review and vote on this PIP. PIP link: https://github.com/apache/pulsar/issues/18560 Discuss thread: https://lists.apache.org/thread/z74vcn0yolzzrcc4ftonm9j3nbk4pzxm Thanks, Tao Jiuming

Re: [DISCUSS] PIP-223: Add metrics for all Rest Endpoints

, like 200 and 307. > > As for `pulsar_broker_rest_endpoint_failed`, after a second thought. I > think it would be useful for operators to set up system alarms. So +1 > on this. > > Thanks, > Haiting > > On Thu, Nov 24, 2022 at 12:52 AM Jiuming Tao > wrote: > > > > Hi Haiting, > &

[Result][Vote] PIP-222: Add CommandPartitionedTopicMetadata metrics

Thanks for vote this PIP, closing the vote 3 bindings: Enrico Olivelli Haiting Jiang Penghui Li 1 non-bindings: Ran Gao Thanks, Tao Jiuming

Re: [VOTE] PIP-222: Add CommandPartitionedTopicMetadata metrics

Bump > 2022年11月22日 下午4:03，Ran Gao 写道： > > +1 (unbinding) > > Thanks, > Ran > > On 2022/11/17 09:06:50 Jiuming Tao wrote: >> Dear Pulsar Community, >> >> Please review and vote on this PIP. >> >> PIP link: https://github.com/apache/pul

[RESULT][Vote] PIP-211: Introduce offload throttling

Thanks for vote this PIP, closing the vote 3 bindings: Enrico Olivelli Haiting Jiang Penghui Li Thanks, Tao Jiuming

Re: [DISCUSS] PIP-223: Add metrics for all Rest Endpoints

> Thanks, > Haiting > > On Wed, Nov 23, 2022 at 12:00 PM Jiuming Tao > wrote: >> >> Hi Penghui, >> >>> We'd better add an option to enable or disable the endpoint-level metrics. >>> And keep it as disabled as default. >> >>

Re: [DISCUSS] PIP-223: Add metrics for all Rest Endpoints

we can > get the request path from the handle method > `public void handle(String path, Request baseRequest, HttpServletRequest > request, HttpServletResponse response)` > > Thanks, > Penghui > > On Tue, Nov 22, 2022 at 5:20 PM Jiuming Tao > wrote: > >> Hi Ha

Re: [DISCUSS] PIP-223: Add metrics for all Rest Endpoints

se to me. > About the metric "pulsar_broker_rest_endpoint_failed", please provide > a more clear definition of "fail". Are redirects like 403 included? > > Thanks, > Haiting > > On Tue, Nov 22, 2022 at 3:17 AM Jiuming Tao > wrote: >> >> Hi pulsar community, >> &

[DISCUSS] PIP-223: Add metrics for all Rest Endpoints

Hi pulsar community, I’ve opened a PIP to discuss: PIP-223: Add metrics for all Rest Endpoints The PIP link: https://github.com/apache/pulsar/issues/18560 Thanks, Tao Jiuming

[VOTE] PIP-211: Introduce offload throttling

Dear Pulsar Community, Please review and vote on this PIP. PIP link: https://github.com/apache/pulsar/issues/18004 Discuss thread: https://lists.apache.org/thread/3j8ldw93gyx1kblknygq35nq8g72plpx

Re: [VOTE] PIP-222: Add CommandPartitionedTopicMetadata metrics

ommandPartitionedTopicMetadata > Enrico > > Il giorno gio 17 nov 2022 alle ore 10:36 Jiuming Tao > ha scritto: > > > > Hi Enrico, > > > > Tracking all the cmds will lead to a lot of changes, BrokerInterceptors > only triggered when the cmd entering. > >

Re: [VOTE] PIP-222: Add CommandPartitionedTopicMetadata metrics

11月17日 下午5:27，Enrico Olivelli 写道： > > Tao, > before voting on this proposal, I think that we should change the title. > Now the PIP refers to tracking all the commands and not only > CommandPartitionedTopicMetadata, is this correct? > > Enrico > > Il giorno gio 17 n

[VOTE] PIP-222: Add CommandPartitionedTopicMetadata metrics

Dear Pulsar Community, Please review and vote on this PIP. PIP link: https://github.com/apache/pulsar/issues/18319 Discuss thread: https://lists.apache.org/thread/sybl4nno4503w42hzt7b5lsyk6m2rbo6

Re: [DISCUSS] PIP-211: Introduce offload throttling

uess that interruption could force the offloader > to need to restart the task of offloading a ledger, which is very > inefficient. > > Thanks, > Michael > > On Thu, Nov 10, 2022 at 12:50 PM Jiuming Tao > wrote: >> >> Hi Michael, >> >> >>&g

Re: [DISCUSS] PIP-222: Add CommandPartitionedTopicMetadata metrics

at is causing it > > What do you think ? > > Enrico > > Il giorno gio 10 nov 2022 alle ore 13:37 Jiuming Tao > ha scritto: >> >> Hi Asaf, >> >>> After reaching the end of the PIP I discovered a link to an issue that >>> gives an ex

Re: [DISCUSS] PIP-211: Introduce offload throttling

h ledgers to offload. Perhaps the managed ledger > could "offer" ledgers up that need to be offloaded, and the new > service would only need to consume those events. > > Although, a new service would complicate the pulsar deployment. > > Thanks, > Michael > >

Re: [DISCUSS] PIP-222: Add CommandPartitionedTopicMetadata metrics

command which seems > reasonable by default > * The client can utilize the bi-directional nature of Pulsar Protocol and > get notified when partitions change (subscribe) therefore limiting the need > to pull based on frequency. > > Thanks, > > Asaf > > > > O

Re: [DISCUSS] PIP-211: Introduce offload throttling

Sent from my iPhone > >> On Nov 1, 2022, at 10:27 AM, Jiuming Tao >> wrote: >> >> Hi pulsar community, >> >> I opened a PIP to discuss: PIP-211: Introduce offload throttling >> >> PIP link: https://github.com/apache/pulsar/issues/18004 >> <https://github.com/apache/pulsar/issues/18004> >> >> Thanks, >> Tao Jiuming >

[DISCUSS] PIP-222: Add CommandPartitionedTopicMetadata metrics

Hi pulsar community, I’ve opened a PIP to discuss: PIP-222: Add CommandPartitionedTopicMetadata metrics The PIP link: https://github.com/apache/pulsar/issues/18319 Thanks, Tao Jiuming

[DISCUSS] PIP-211: Introduce offload throttling

Hi pulsar community, I opened a PIP to discuss: PIP-211: Introduce offload throttling PIP link: https://github.com/apache/pulsar/issues/18004 Thanks, Tao Jiuming

Fwd: [DISCUSS] Introduce FlowControl to metrics endpoint

min), the network gets >>> surged up due to the /metrics response, thereby causing latency to >> messages >>> produced or consumed from that broker. >>> >>> The solution proposed is to throttle the /metrics response based on a >>> pre-configured rate limit. &

[DISCUSS] Introduce FlowControl to metrics endpoint

produced or consumed from that broker. >> >> The solution proposed is to throttle the /metrics response based on a >> pre-configured rate limit. >> >> Points to consider for this discussion from the participants: >> >> 1. Did you happen to experience such dif

Re: [Discussion] PIP 198 - How to define [type] and [scope]?

A A Thanks Tao Jiuming > 2022年8月22日 下午12:47，Yu 写道： > > Hi developers, > > Two quick questions need your vote! > > Which do you prefer? > > > > # 1. Use "branch" or "BP"? > > Choice A: [fix][broker][branch-2.9] xxx > Choice B: [fix][broker][BP-2.9] xxx > > ~

Re: [ANNOUNCE] Jiwei Guo as a new PMC member in Pulsar

Congratulations! Yu 于2022年8月19日周五 11:02写道： > Congrats Jiwei! > > On Fri, Aug 19, 2022 at 10:45 AM Zixuan Liu wrote: > > > Congrats! > > > > Zixuan > > > > Jun M 于2022年8月19日周五 10:16写道： > > > > > Congrats to Jiwei! > > > > > > > > > Cheers, > > > Jun > > > > > >

Re: [DISCUSS] Introduce FlowControl to metrics endpoint

bump Jiuming Tao 于2022年8月8日周一 18:19写道： > Hi Pulsar community, > > In the situation of expose metrics data which has a big size, it will lead > to: > 1. A sudden increase of network usage > 2. The latency of pub/sub rising > > For the purpose of resolving these problems,

[DISCUSS] Introduce FlowControl to metrics endpoint

Hi Pulsar community, In the situation of expose metrics data which has a big size, it will lead to: 1. A sudden increase of network usage 2. The latency of pub/sub rising For the purpose of resolving these problems, I’d opened a PR: https://github.com/apache/pulsar/pull/16452

Re: Mark delete rate limiter bug?

Entries will filtered by ManagedCursorImpl#filterReadEntries Both of individual_ack and cumulative_ack will update ManagedCursorImpl#individualDeletedMessages, messages will be filtered which has already acked. Thanks, Tao Jiuming > 2022年6月15日 下午11:45，Asaf Mesika 写道： > > Hi, > > In the Mana

Re: [VOTE] [PIP-154] Max active transaction limitation for transaction coordinator

+1 Thanks, Tao Jiuming > 2022年4月24日 上午10:00，丛搏 写道： > > Hi Pulsar community, > > This is the voting thread for PIP-154. It will stay open for at least 48 > hours. > > The proposal can be found: https://github.com/apache/pulsar/issues/15133 > > Discuss thread: > https://lists.apache.org/threa

Re: [VOTE] PIP-155: Drop support for Python2

+1 Thanks, Tao Jiuming > 2022年4月20日 下午8:42，Matteo Merli 写道： > > https://github.com/apache/pulsar/issues/15185 > > ## Motivation > > Python 2.x has been deprecated for many years now and it was > officially end-of-lifed 2.5 years ago > (https://www.python.org/doc/sunset-python-2/). > > We have

Re: PIP-156: Build and Run Pulsar Server on Java 17

+1 Thanks, Tao Jiuming > 2022年4月19日 上午2:25，Heesung Sohn 写道： > > Dear Pulsar dev community, > > My name is Heesung Sohn, and I would like to open a discussion here > about PIP-156: > Build and Run Pulsar Server on Java 17 > . > > > @Nicolò Boschi,

Re: [discuss] Support fail-fast strategy when broker rate-limited

sableAutoread` is Producer/Consumer > isolation issue, > since they shares the connection. For example, if topic-A is rate-limited, > topic-B in the same > client is also affected. > > Thanks, > Haiting > > On 2022/04/07 07:13:50 Jiuming Tao wrote: >> Hi all: >

Re: [discuss] Support fail-fast strategy when broker rate-limited

` instead. > 2022年4月7日 下午3:13，Jiuming Tao 写道： > > Hi all: > According to the issue (https://github.com/apache/pulsar/issues/15038), > when producer send a message to broker and broker rate-limited at that time, > the request will blocked in buffer, after broker read the

[discuss] Support fail-fast strategy when broker rate-limited

Hi all: According to the issue (https://github.com/apache/pulsar/issues/15038), when producer send a message to broker and broker rate-limited at that time, the request will blocked in buffer, after broker read the message and send receipt to producer, the producer maybe timeout already

Re: [DISCUSS] PrometheusMetricsServlet performance improvement

gt; > Thanks, > Penghui > > On Sun, Feb 27, 2022 at 12:22 AM Jiuming Tao > wrote: > >> Hi all, >> https://github.com/apache/pulsar/pull/14453 < >> https://github.com/apache/pulsar/pull/14453> please take a look. >> >> Thanks, >> Tao Jium

Re: [DISCUSS] PrometheusMetricsServlet performance improvement

Hi all, https://github.com/apache/pulsar/pull/14453 <https://github.com/apache/pulsar/pull/14453> please take a look. Thanks, Tao Jiuming > 2022年2月24日 上午1:05，Jiuming Tao 写道： > > Hi all, >> >> 2. When there are hundreds MB metrics data collected, it causes high

Re: [DISCUSS] PrometheusMetricsServlet performance improvement

when does this happen? > >> PrometheusMetricsGenerator#generate will be invoked once in a period(such >> as 1 minute), the result will be cached and returned for every metrics >> collect request in the period directly. > > Since there are tradeoffs to the cache duratio

Re: [VOTE] Pulsar Release 2.8.3 Candidate 3

+1 > 2022年2月25日 上午11:44，Michael Marshall 写道： > > This is the third release candidate for Apache Pulsar, version 2.8.3. > > It fixes the following issues: > https://github.com/apache/pulsar/compare/v2.8.2...v2.8.3-candidate-3 > > *** Please download, test and vote on this release. This vote wi

Re: [DISCUSS] PrometheusMetricsServlet performance improvement

Hi all, > > 2. When there are hundreds MB metrics data collected, it causes high heap > memory usage, high CPU usage and GC pressure. In the > `PrometheusMetricsGenerator#generate` method, it uses > `ByteBufAllocator.DEFAULT.heapBuffer()` to allocate memory for writing > metrics data. The defa

[DISCUSS] PrometheusMetricsServlet performance improvement

Hi all, 1. I have learned that the /metrics endpoint will be requested by more than one metrics collect system. In the condition, I want to reimplement `PromethuesMetricsServlet` by sliding window. PrometheusMetricsGenerator#generate will be invoked once in a period(such as 1 minute), the result w