Re: [ANNOUNCE] New Committer: Marvin Cai

Congrats! --- Haiting Jiang On 2021/12/13 09:46:10 linlin wrote: > The Apache Pulsar Project Management Committee (PMC) has invited Marvin Cai > https://github.com/MarvinCai to become a committer and we are pleased to > announce that he has accepted. > > Marvin has joined the com

Re: [DISCUSSION] PIP-117: Change Pulsar standalone defaults

+1 (non-binding) Out of the scope of this PIP, a follow up data migration tool could be added to help migration from old settings without data loss. --- Haiting On 2021/12/15 01:04:21 Hang Chen wrote: > +1 > > Thanks, > Hang > > PengHui Li 于2021年12月15日周三 07:52写道： > > > > +1 > > > > Penghui >

Re: [DISCUSSION] PIP-120: Enable client memory limit by default

+1 (non-binding) Haiting On 2021/12/14 19:20:02 Matteo Merli wrote: > https://github.com/apache/pulsar/issues/13306 > > > Pasted below for quoting convenience. > > > > > ## Motivation > > In Pulsar 2.8, we have introduced a setting to control the amount of memory > used by a client

Re: PIP 116: Create Pulsar Writing Style Guide

+1, Great stuff. I am wondering if there is some tool to check it automatically? like code-style check. Thanks, Haiting On 2021/12/15 03:17:45 Yu wrote: > Hi Pulsar enthusiasts, > > > As you may notice, there are more and more documentation contributions > nowadays, which is great! > > > Si

Re: [DISCUSSION] PIP-118: Do not restart brokers when ZooKeeper session expires

+1 Great PIP. Long expected. Small question about new parameter "zookeeperSessionExpiredPolicy", now we have Etcd implementation for metadata store, is it better to use "metadataStoreExpiredPolicy" ? On 2021/12/14 18:03:20 Matteo Merli wrote: > https://github.com/apache/pulsar/issues/13304 >

Re: [DISCUSSION] PIP-117: Change Pulsar standalone defaults

+1 > > @Haiting > Pulsar Standalone is usually used for local development, I am not sure you > need a migration tool. > You can still keep the configuration to use previously (standalone.conf) > > Enrico > > Il giorno mer 15 dic 2021 alle ore 10:56 Haiting Jiang < >

[DISCUSSION] Produce chunk messages failed when topic level maxMessageSize is set

geSize issue [3] when batching is enabled (non-batching case is solved with PR [4]). Any suggestions or other use cases of topic level maxMessageSize will be appreciated. Thanks, Haiting Jiang [1] https://github.com/apache/pulsar/issues/13360 [2] https://github.com/apache/pulsar/pull/8

Re: [PR] Pulsar non root docker image

> 1. Pulsar configuration is read in only from configuration files in > `/pulsar/conf`. A non root user must be able to update these files to > have run with custom configuration. About the configurations, I also see similar require like this lately [0]. IMHO, update any configs without redeploy

Re: [VOTE] PIP-117: Change Pulsar standalone defaults

+1 Thanks, Haiting On 2021/12/23 05:35:03 Michael Marshall wrote: > +1 > > - Michael > > On Wed, Dec 22, 2021 at 6:18 PM Sijie Guo wrote: > > > > +1 > > > > On Tue, Dec 21, 2021 at 3:49 PM Matteo Merli wrote: > > > > > This is the voting thread for PIP-117. It will stay open for at least 48h.

Re: [VOTE] PIP-123: Introduce Pulsar metadata CLI tool

+1 Thanks, Haiting On 2021/12/23 05:38:14 Michael Marshall wrote: > +1 > > - Michael > > On Wed, Dec 22, 2021 at 6:18 PM Sijie Guo wrote: > > > > +1 > > > > On Tue, Dec 21, 2021 at 3:59 PM Matteo Merli wrote: > > > > > This is the voting thread for PIP-123. It will stay open for at least 48h.

Re: [broker]loadBalancerMemoryResourceWeight config default value optimiz

e 0.5. [1] https://docs.oracle.com/en/java/javase/12/docs/api/jdk.management/com/sun/management/GcInfo.html [2] https://docs.oracle.com/javase/8/docs/technotes/guides/vm/gctuning/g1_gc.html Thanks, Haiting Jiang On 2021/12/24 01:57:37 jiangxinwei wrote: > https://github.com/apache/pulsar/pu

Re: [DISCUSS] PIP-124: Pulsar Client Shared State API

+1, Great feature. > It is not a goal to implement a Pulsar backed Database system But this does look like a database system. I believe there would be some limitations here, either in availability, consistency, performance, storage cost, etc. So I think we can provide more information about dis

[VOTE] PIP 131: Resolve produce chunk messages failed when topic level maxMessageSize is set

ize issue [3] when batching is enabled (non-batching case is solved with PR [4]). [1] https://github.com/apache/pulsar/issues/13360 [2] https://github.com/apache/pulsar/pull/8732 [3] https://github.com/apache/pulsar/issues/12958 [4] https://github.com/apache/pulsar/pull/13147 Thanks, Haiting Jiang

[DISCUSSION] PIP-131: Include message header size when check maxMessageSize of non-batch message on the client side.

quot; to enable this feature. But I think it's OK to enable this directly as it's more reasonable, and I don't see good migration plan if we add a config for this. The compatibility issue is worth discussing. And any suggestions are appreciated. [1] https://github.com/apache/pulsar/issues/13560 Thanks, Haiting Jiang

Re: [DISCUSSION] PIP-132: Include message header size when check maxMessageSize of non-batch message on the client side.

Sorry, it should be PIP-132. Thanks, Haiting Jiang On 2021/12/31 12:05:54 Haiting Jiang wrote: > https://github.com/apache/pulsar/issues/13591 > > Pasted below for quoting convenience. > > —— > > ## Motivation > > Currently, Pulsar client (Java) only checks payloa

Re: [DISCUSSION] PIP-130: Apply redelivery backoff policy for ack timeout

+1 for this PIP. Do we have plans for other languages clients? like go? Thanks, Haiting Jiang On 2021/12/27 13:24:52 PengHui Li wrote: > https://github.com/apache/pulsar/issues/13528 > > Pasted below for quoting convenience. > > - > > PIP 130: Apply redelivery b

Re: [Bug] Other brokers failed to acquire bundle ownership after unloading a namespace bundle

Hi, Please provide more information about this issue, like pulsar version, steps to steady reproduce this, the influence of this error. Thanks, Haiting Jiang On 2021/12/31 03:41:45 zhangao wrote: > Hi,  >     Currently, I found a problem about bundle ownership acquire, > After I u

Re: [VOTE] PIP-130: Apply redelivery backoff policy for ack timeout

+1 Thanks, Haiting Jiang On 2022/01/04 03:34:33 PengHui Li wrote: > This is the voting thread for PIP-130. It will stay open for at least 48 > hours. > > https://github.com/apache/pulsar/issues/13528 > > Pasted below for quoting convenience. > > - > > PIP

Re: [DISCUSSION] PIP-132: Include message header size when check maxMessageSize of non-batch message on the client side.

Good point, I think we should shrink chunk size to "ClientCnx.getMaxMessageSize() - chunkMessageHeaderSize", as we have the same header size for each chunk message. Thanks, Haiting Jiang On 2022/01/04 03:27:00 Zike Yang wrote: > But how do we handle chunked messages? The chunk

Re: Voting time period minimum

had 2 PIP votes called for 48 hours so this rule is worth reiterating! > > All the best, > Dave > > Sent from my iPhone > Thanks, Haiting Jiang

Re: [DISCUSSION] PIP-132: Include message header size when check maxMessageSize of non-batch message on the client side.

Hi mattison, Yes, this is an alternative way to solve the case of properties is too large. But I think this approach is more complex in coding and introduces new concept and configs, and I don't see the benefit of limiting the header size and payload size separately. Thanks, Haiting

Re: [ANNOUNCE] New Committer: Zhangjian He

Congrats, Zhangjian! Thanks, Haiting Jiang On 2022/01/06 02:05:53 Jia Zhai wrote: > Congratulations to Zhangjian! > Thanks for your contributions to Pulsar. > > > > On Thu, Jan 6, 2022 at 9:25 AM guo jiwei wrote: > > > Congrats to Zhangjian! > > &g

Re: [DISCUSSION] PIP-132: Support Pulsar system event.

> > - [ ] Client connected > - [ ] Client disconnected > - [ ] Consumer subscribe > - [ ] Consumer unsubscribe > - [ ] Producer publish > - [ ] Message delivered > - [ ] Message acked > > These are far from enough, and we need to continue to make supplements here. > Please fell free to give me more advice for this PIP. > > ## Reject Alternatives > > none. > > Thanks, > Mattisonchao Thanks, Haiting Jiang

Re: [DISCUSSION] PIP-132: Include message header size when check maxMessageSize of non-batch message on the client side.

take too much space (several bytes each), and broker have 10KB buffer for maxMessageSize. Thanks, Haiting Jiang On 2022/01/05 08:11:06 Zike Yang wrote: > > as we have the same header size for each chunk message. > > The message metadata of different chunks will not be the same. There &g

Re: [DISCUSSION] PIP-132: Support Pulsar system event.

he better way is to record some system internal event like node > changed, ledger deleted etc. This PIP can start with a small scope of events that requires the attention of administrators, like ledger delete failure, zk session timeout, etc. Thanks, Haiting Jiang On 2022/01/06 05:23:52 m

Re: [DISCUSSION] PIP-135: Include MetadataStore backend for Etcd

+1 On 2022/01/12 01:44:21 PengHui Li wrote: > +1 > > Penghui > > On Wed, Jan 12, 2022 at 8:39 AM mattison chao > wrote: > > > +1 > > > > On Wed, 12 Jan 2022 at 08:09, Matteo Merli wrote: > > > > > https://github.com/apache/pulsar/issues/13717 > > > > > > - > > > > > > ## Motivation > > >

Re: [VOTE] PIP-122: Change loadBalancer default loadSheddingStrategy to ThresholdShedder

+1 On 2022/01/10 06:47:44 Hang Chen wrote: > This is the voting thread for PIP-122. It will stay open for at least 48 > hours. > > https://github.com/apache/pulsar/issues/13340 > > Pasted below for quoting convenience. > > > > ### Motivation > The ThresholdShedder load balance policy sinc

Re: [VOTE] PIP-121: Pulsar cluster level auto failover on client side

+1 On 2022/01/12 00:09:26 Matteo Merli wrote: > +1 > -- > Matteo Merli > > > On Tue, Jan 11, 2022 at 12:07 PM Neng Lu wrote: > > > > +1 (non-binding) > > > > On Mon, Jan 10, 2022 at 12:40 AM PengHui Li wrote: > > > > > +1 (binding) > > > > > > Penghui > > > > > > On Mon, Jan 10, 2022 at 4:38 P

[VOTE] PIP-132: Include message header size when check maxMessageSize of non-batch message on the client side.

fig "includeHeaderInSizeCheck" to enable this feature. But I think it's OK to enable this directly as it's more reasonable, and I don't see good migration plan if we add a config for this. [1] https://github.com/apache/pulsar/issues/13560 Thanks, Haiting Jiang

Re: [Discuss] Create new issues to SDKs in different languages

+1. Great idea. I wonder if we can create issue in client repo automatically with bots for PRs labelled"component/client" in pulsar repo. This would save the extra effort for the reviewer. Thanks, Haiting Jiang On 2022/01/12 03:45:18 "r...@apache.org" wrote: > Hello ev

Re: [VOTE] PIP-129: Introduce intermediate state for ledger deletion

+1 (non) On 2022/01/14 03:23:37 mattison chao wrote: > +1 (non-binding) > > Best, > Mattison > > On Fri, 14 Jan 2022 at 11:19, Hang Chen wrote: > > > +1 (binding) > > > > Best, > > Hang > > > > Zhanpeng Wu 于2022年1月14日周五 10:37写道： > > > > > > This is the voting thread for PIP-129. It will stay

Re: [ANNOUNCE] New PMC Member - Lari Hotari

Congratulations! BR, Haiting Jiang On 2022/01/18 02:26:32 陳智弘 wrote: > I am so glad to hear this news , lots of critical issues were solved by him > and really help us a lot > > Thomas > > Aloys Zhang 於 2022年1月18日 週二 上午10:25寫道： > > > Congratulations! > > >

Re: [VOTE] PIP-132: Include message header size when check maxMessageSize of non-batch message on the client side.

Thanks for your participation. Close this vote with 3 (+1) bindings and 6 (+1) non-bindings, 0 (-1). Thanks, Haiting On 2022/01/12 01:57:59 Haiting Jiang wrote: > This is the voting thread for PIP-132. It will stay open for at least 48 > hours. > > https://github.com/apache/p

Re: [DISCUSS] The default value of maxPendingChunkedMessage

> I agree with updating the Javadoc to align with the actual code. This > will lead to fewer surprises, +1. For further improvement, I think we can deprecate `maxPendingChunkedMessage` by extending the scope of `ClientBuilder#memoryLimit` to consumers. Thanks, Haiting On 2022/02/01 05:23:06

Re: [ANNOUNCE] New Committer: Aloys Zhang

Congrats! Aloys Zhang Thanks, Haiting On 2022/02/10 09:10:02 Hang Chen wrote: > Congratulations, Aloys Zhang > > Best, > Hang > > PengHui Li 于2022年2月10日周四 12:14写道： > > > > Congratulations, Aloys Zhang > > > > Thanks, > > Penghui > > > > On Thu, Feb 10, 2022 at 11:19 AM ZhangJian He wrote: >

Re: [VOTE] Pulsar Release 2.7.5 Candidate 1

+1 Checks: - Checksum and signatures - Compile from source w JDK11 - Validate Pub/Sub and Java Functions [1] - Validate Connectors [1] - Validate Stateful Functions [1] [1] https://github.com/apache/pulsar/wiki/Release-Candidate-Validation Thanks, Haiting On 2022/02/11 13:47:50 Lari Hotari

Re: [ANNOUNCE] New Committer: Li Li

Congrats! Thanks, Haiting On 2022/02/16 10:51:52 Guangning E wrote: > Configurations! > > > Thanks, > Guangning > > Jia Zhai 于2022年2月16日周三 15:40写道： > > > Congratulations > > > > > > On Wed, Feb 16, 2022 at 11:39 AM Yu wrote: > > > > > Hi Lili, > > > > > > Congrats! Well deserved! You've m

Re: [Discuss][PIP-143] Support split paritions belonging to specified topics in a bundle

Hi Aloys, +1 for this great PIP. > The Admin CLI `bin/pulsar-admin namespaces split-bundle -b ${bundle_range}` > will add a new parameter "--topic" or "-t" for `outstanding topic` name. Do we have limitation on this "topic" parameter. Can this be a partitioned topic? If so, will this new algori

Re: [Discuss][PIP-143] Support split paritions belonging to specified topics in a bundle

> 3 topics to one bundle. This is more like increasing the transparency of > > the topic in the bundle, you can all the positions of the topics, so how > > planning for bundle splitting becomes more flexible. > > > > The new API does not necessarily have to query by topic

Re: [Discuss][PIP-143] Support split paritions belonging to specified topics in a bundle

loys > > Aloys Zhang 于2022年2月21日周一 14:39写道： > > > Hi, penghui > > > > > The new API does not necessarily have to query by topic one by one, > > we have listed all the "topic -> position" of a bundle? > > > > I see. After we got all the p

Re: [VOTE] [PIP-143] Support split bundle by specified boundaries

+1 Thanks, Haiting On 2022/02/26 02:53:23 Aloys Zhang wrote: > Hi Pulsar Community, > > I would like to start a VOTE on "Support split bundle by specified > boundaries."(PIP-143) > > Here is the design detail https://github.com/apache/pulsar/issues/13761 > > and the discussion thread > https:

Re: [DISCUSSION][PIP-146] ManagedCursorInfo compression

This is a global setting now. But I wonder if we should compress it only if the size is over a threshold? Because: 1. It's not easy for us to notice some managed cursor info is too large in advance, normally it would be found only if it have actual impact. But if we enable this compression in

Re: [DISUSS] Improve unit test stability

+1 for this great idea. Although I am not sure if there is an easy and accurate way to "find new or modified unit tests". Thanks, Haiting On 2022/03/07 03:11:47 包子 wrote: > Hi, I want to discuss how to improve the stability of unit testing.I found > that most flaky unit tests can be reproduc lo

Re: [VOTE][PIP-146] ManagedCursorInfo compression

+1 Thanks, Haiting On 2022/03/09 09:18:04 Zixuan Liu wrote: > Hi Pulsar community, > > I want to start this PIP - https://github.com/apache/pulsar/issues/14529 > > Thanks, > Zixuan >

Re: [DISCUSS] Set EntryCache evict task interval directly

+1 managedLedgerCacheEvictionIntervalMs is easier to understand. > What's more, the evict task is executed once, and then sleep waitTimeMillis > before the next round, we can use a ScheduledExecutorService to replace it. +1, Currently it works like scheduleAtFixDelay, but I think we may need sc

Re: [DISCUSS] PIP-143 : Standardize Admin REST API

+1 Thanks, Haiting On 2022/03/16 12:43:45 guo jiwei wrote: > Hello community, > > I want to discuss refactoring and standardizing REST API. Users have > encountered several deadlock problems in the API part before, like #13666 > , #12590 >

Re: [VOTE] Pulsar Release 2.10.0 Candidate 4

+1 - Compiled the source - Checked checksums and signatures - Validate Pub/Sub and Java Functions - Validate Connectors - Validate Stateful Functions Thanks Haiting On 2022/03/22 01:52:36 PengHui Li wrote: > This is the fourth release candidate for Apache Pulsar 2.10.0 > > It fixes the followin

Re: [DISSCUSS] [PIP-150] Support read the message of startMessageId position on the broker side

This PIP seems want to enable the feature of "startMessageIdInclusive" on Subscribe command, right? I notice that you only mentions Reader in this PIP. What about normal consumer ? It also has the config of `startMessageIdInclusive` in ConsumerBuilder. Thanks, Haiting On 2022/03/25 16:35:5

Re: [DISSCUSS] [PIP-151] Use the system topic to store the bundle load data

Hi Kai, Good feature. Please add more info about how to enable this in an existing cluster. Do we plan to write bundle data to both zk and system topic during rolling update? Thanks, Haiting On 2022/04/07 01:35:09 Kai Wang wrote: > Hi Pulsar community, > > I created a PIP to use the system

Re: [discuss] Support fail-fast strategy when broker rate-limited

> send receipt to producer, the producer maybe timeout already. It's always possible that client got timeout, but the message is actually successfully written, causing by network issue. IMO, the more serious problem about `disableAutoread` is Producer/Consumer isolation issue, since they shar

[DISCUSS] [PIP-152] Support subscription level dispatch rate limiter setting.

Hi Pulsar community, I created a PIP to add support for subscription level dispatch rate limiter setting. The proposal can be found: https://github.com/apache/pulsar/issues/15094 ## Motivation Currently, for message dispatch rate limiter in a subscription , we have 3 level setting : - B

Re: [VOTE] [PIP-150] Support read the message of startMessageId position on the broker side

+1 Thanks, Haiting On 2022/04/06 08:54:05 Zixuan Liu wrote: > Hi Pulsar community, > > Start voting for PIP-150: https://github.com/apache/pulsar/issues/14883 > > Thanks, > Zixuan > > - > > Discussion thread: > https://lists.apache.org/thread/n3drk2g2oy766qnbtx17knvtssy3tdyl > > ## Moti

Re: [DISCUSS] [PIP-152] Support subscription level dispatch rate limiter setting.

ery useful feature. Thank you. > > > > > > Best, > > > Mattison > > > > > > On Mon, 11 Apr 2022 at 08:55, PengHui Li wrote: > > > > > > > +1 > > > > > > > > Penghui > > > > > > > > On Sat, Apr 9, 20

Re: [DISCUSS] [PIP-152] Support subscription level dispatch rate limiter setting.

on the client side. > > > > > > > > -joe > > > > On Mon, Apr 11, 2022 at 7:54 PM Zixuan Liu wrote: > > > > > +1 > > > > > > Zixuan > > > > > > mattison chao 于2022年4月12日周二 09:24写道： > > > > > &

Re: [discuss] Support fail-fast strategy when broker rate-limited

mple, if topic-A is > >> rate-limited, > >> > topic-B in the same client is also affected. > >> > >> Exactly. The rate-limiting won't even work properly since other rate > >> limiters operating on the same connection will enable autoread and thi

Re: [ANNOUNCE] New Committer: Zike Yang

Congrats! Thanks, Haiting On 2022/04/13 09:34:23 PengHui Li wrote: > The Apache Pulsar Project Management Committee (PMC) has invited Zike Yang > https://github.com/RobertIndie to become a committer and we are pleased to > announce that he has accepted. > > Welcome and Congratulations, Zike Yang

Re: [ANNOUNCE] New Committer: Zhangmin Gao

Congrats! Thanks, Haiting On 2022/04/13 09:35:57 PengHui Li wrote: > The Apache Pulsar Project Management Committee (PMC) has invited Zhangmin > Gao > https://github.com/gaozhangmin to become a committer and we are pleased to > announce that he has accepted. > > Welcome and Congratulations, Zhan

[VOTE] [PIP-152] Support subscription level dispatch rate limiter setting.

Hi Pulsar community, This is the voting thread for PIP-152. It will stay open for at least 48 hours. The proposal can be found: https://github.com/apache/pulsar/issues/15094 Discuss thread: https://lists.apache.org/thread/r6dzr09lc42yh79vt0dvmvlv6wtz2czn Thanks, Haiting Jiang

Re: [DISCUSS] [PIP-154] Max active transaction limitation for transaction coordinator

Hi Bo, Do we have metrics like "active transaction usage percent" to set up monitor and alert. This is necessary since this critical once the limitation is reached. Thanks, Haiting On 2022/04/12 12:44:36 丛搏 wrote: > pipHi Pulsar community, > > I open a pip to discuss Max active transaction li

Re: [VOTE] [PIP-154] Max active transaction limitation for transaction coordinator

+1 Thanks, Haiting On 2022/04/24 02:00:39 丛搏 wrote: > Hi Pulsar community, > > This is the voting thread for PIP-154. It will stay open for at least 48 > hours. > > The proposal can be found: https://github.com/apache/pulsar/issues/15133 > > Discuss thread: > https://lists.apache.org/thread/

Re: [VOTE] [PIP-152] Support subscription level dispatch rate limiter setting.

, > > Hang > > > > Lin Lin 于2022年4月23日周六 21:52写道： > > > > > > +1 > > > > > > Lin Lin > > > > > > On 2022/04/19 03:27:39 Haiting Jiang wrote: > > > > Hi Pulsar community, > > > > > > > > This

Re: [ANNOUNCE] New Committer: Nicolò Boschi

Congrats! BR, Haiting On 2022/04/26 23:48:25 PengHui Li wrote: > The Apache Pulsar Project Management Committee (PMC) has invited Nicolò > Boschi > https://github.com/nicoloboschi to become a committer and we are pleased to > announce that he has accepted. > > Welcome and Congratulations, Nicolò

Re: [DISCUSSION] PIP-156: Enable system topic by default

+1 Thanks, Haiting On 2022/04/26 12:07:32 guo jiwei wrote: > Hi community: >In PIP-39 > , > we have introduced namespace event to support topic level policy. This > feature was released in Pulsar 2.6. > >Then PIP-92

Re: [VOTE] [PIP-154] Max active transaction limitation for transaction coordinator

+1 Thanks, Haiting On 2022/04/24 02:00:39 丛搏 wrote: > Hi Pulsar community, > > This is the voting thread for PIP-154. It will stay open for at least 48 > hours. > > The proposal can be found: https://github.com/apache/pulsar/issues/15133 > > Discuss thread: > https://lists.apache.org/thread/

Re: [ANNOUNCE] new Committer Qiang Zhao (mattisonchao)

Congratulations! BR, Haiting On 2022/05/04 06:28:24 Enrico Olivelli wrote: > The Project Management Committee (PMC) for Apache Pulsar > has invited Qiang Zhao to become a committer and we are pleased > to announce that he has accepted. > > Qiang Zhao (with GH id mattisonchao) contributed lots

Re: [DISCUSS] PIP-105 extension: per-consumer filtering

It looks like we can add something like `EntryDispatcher` before the EntryFilter. Mixing entry filtering and consumer selecting seems a little confusing. The `EntryDispatcher` could works as a consumer selector in `PersistentDispatcherMultipleConsumers`. It accepts an entry and a consumer li

Re: [VOTE] PIP-156: Enable system topic by default

+1 Regards, Haiting On 2022/05/11 13:26:09 guo jiwei wrote: > This is the voting thread for PIP-166. It will stay open for at least 48h. > > https://github.com/apache/pulsar/issues/15333 > > --- > ## Motivation > In [PIP-39]( > https://github.com/apache/pulsar/wiki/P

Re: [DISCUSS] [PIP-165] Auto release client useless connections

Hi Yubiao, It seems one possible downside of this pip is increasing produce latency when topic ownership changed, any way to avoid that? Thanks, Haiting On 2022/05/26 06:31:37 Yubiao Feng wrote: > I open a pip to discuss Auto release client useless connections, could you > help me review > >

Re: [DISCUSS] [PIP-165] Auto release client useless connections

s to reconnect to ownership. Do you > have a good idea. > > On Fri, May 27, 2022 at 10:15 AM Haiting Jiang > wrote: > > > Hi Yubiao, > > > > It seems one possible downside of this pip is increasing produce latency > > when topic ownership changed, any way to

Re: [DISCUSS] [PIP-165] Auto release client useless connections

t; released repeatedly. We can declare a policy to either command lookup using > 192.168.0.3(already created connection) or not release the service-URL > connection. > > Yubiao Feng > Thanks > > On Fri, May 27, 2022 at 6:31 PM Haiting Jiang > wrote: > > > Hi

Re: [VOTE] PIP-168: Support zero-copy of NIC to NIC on Proxy

+1 Thanks, Haiting On 2022/05/26 13:40:30 zhaocong wrote: > Hi Pulsar Community, > > > I would like to start a VOTE on "Support zero-copy of NIC to NIC on Proxy" > (PIP-168). > > > The proposal can be read at https://github.com/apache/pulsar/issues/15631 > > and the discussion thead is avail

Re: About 2.10.1, 2.9.2, 2.8.4 has this been cherry picked yet? [fix][java-client] Fix performance regression with message listener

Hi Dave, Yes, it's cherry-picked on Apr 20. Here are the commits: https://github.com/apache/pulsar/commit/dcecd86225e2ca681edb0cbf922919539c7174f9 https://github.com/apache/pulsar/commit/0aad9a12339de434ce149cd57d524a3b45b1acc8 https://github.com/apache/pulsar/commit/c7124415b118bc37ee917f4b4c532e

Re: [ANNOUNCE] New Committer: Dezhi Liu

Congrats! BR, Haiting On 2022/06/07 06:46:00 Hang Chen wrote: > The Project Management Committee (PMC) for Apache Pulsar has invited > Dezhi Liu (https://github.com/liudezhi2098) to become a committer and > we are pleased to announce that he has accepted. > > Dezhi Liu (with Github id liudezhi20

Re: [DISCUSS] PIP-175: Extend time based release process

> I was actually not thinking of changing the denomination of 2.11. On > one hand, it could make sense for being the first Java 17 release, but > on the other, we'd be releasing 2 LTSs super close between each other > and we'd have to support 1 release more for the time being. > > I'd like to hea

Re: [DISCUSS] PIP-174: Provide new implementation for broker dispatch cache

> Instead of dealing with individual caches for each topic, let's adopt > a model where > there is a single cache space for the broker. If all topics share the same cache space, and we won't recycle space by topic. So when some topics are rebalanced to other brokers, the previous broker keeps its

Re: [VOTE] PIP-165: Auto release client useless connections

+1 Haiting On 2022/06/14 10:28:22 Xiangying Meng wrote: > +1 > > On Tue, Jun 14, 2022 at 3:52 PM Gavin Gao wrote: > > > +1 > > > > > > On 2022/06/01 07:11:51 Yubiao Feng wrote: > > > Hi Pulsar Community: > > > > > > There were some mistakes in the last email, so I have corrected them. > > > >

[DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

.com/apache/pulsar/pull/11960 [3] https://github.com/apache/pulsar/issues/16153 BR, Haiting Jiang

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

hy you need to change the > protocol to introduce shadow message id. Can you please explain that to me? > Is CommandSend used only internally between Pulsar Clusters or used by a > Pulsar Client? > > Thanks, > > Asaf > > On Tue, Jun 21, 2022 at 11:00 AM Haiting Jiang > wrot

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

Hi Penghui & Asaf: Please allow me to provide some more detailes about **metadata** synchronization between source topic and shadow topic. 1.When shadow topic initializes, it will read from metadata store path "/managed-ledgers/{source_topic_ledger_name}", which contains all the managed ledge

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

Hi Dave, On 2022/06/23 03:59:35 Dave Fisher wrote: > > On Jun 21, 2022, at 1:00 AM, Haiting Jiang wrote: > > > > Hi Pulsar community: > > > > I open a pip to discuss "Shadow Topic, an alternative way to support > > readonly topic ownership." >

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

d is a Pulsar marker or not optional bool marker = 8 [default = false]; // message id for shadow topic optional MessageIdData shadow_message_id = 9; } ``` So there won't be any broker-to-broker directly protobuf command interactions. Thanks, Haiting > Thanks, > Penghui > &g

Re: [DISCUSS] PIP-181: Pulsar Shell

+1, Great feature. Thanks, Haiting On 2022/07/01 16:56:08 Nicolò Boschi wrote: > Yes it's not a good idea to add such new features to active release > branches. > However, the shell will work with older cluster versions (as soon as the > java client is compatible). Also you will be able to downlo

Re: [DISCUSS] PIP-181: Provide new load balance placement strategy implementation for ModularLoadManagerStrategy

PR #16319 does the similar thing. Just sync the info to this mail thread. IMO, the real problem is the definition of broker load (aka resource usage) is not consistent in bundle shedder and bundle assignment. The load balance module should have a better abstraction that contains both shedding

Re: [DISCUSS] [PIP-179] Support the admin API to check unknown request parameters

On 2022/06/22 02:52:44 Yubiao Feng wrote: > Hi, Pulsar community: > > I open a pip to discuss "Support the admin API to check unknown request > parameters" > > Proposal Link: https://github.com/apache/pulsar/issues/16135 > > ### Motivation > > The design of the Admin API is now such that if

Re: [VOTE] PIP-181: Pulsar Shell

+1 (non-binding) Thanks, Haiting On 2022/07/04 14:45:28 Nicolò Boschi wrote: > Dear Community, > > I would like to start a VOTE on "PIP-181: Pulsar Shell" > > The proposal can be read at [0] and the discussion thread is available at > [1]. > Voting will stay open for at least 48h. > > [0] http

[VOTE] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership

sar/issues/16153 [1] https://lists.apache.org/thread/o9k7trfmxrz89b0woybnshonpkq8ybw1 Thanks, Haiting Jiang

Re: [ANNOUNCE] New Committer: Zixuan Liu

Congratulates, Zixuan! BR, Haiting On 2022/07/07 10:03:36 Yu wrote: > Hi team, > > The Project Management Committee (PMC) for Apache Pulsar has invited > Zixuan Liu (https://github.com/nodece) to become a committer > and we are pleased to announce that he has accepted. > > Being a committer ena

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

will have to resolve edge cases while developing or after going > to production. > > Every feature must be fully integrated with the rest of Pulsar > > Enrico > > Il giorno mer 29 giu 2022 alle ore 08:40 Haiting Jiang > ha scritto: > > > > Hi Penghui > &

Re: [VOTE] [PIP-182] Provide new load balance placement strategy implementation for ModularLoadManagerStrategy

+1 Thanks, Haiting On 2022/07/08 06:30:19 Qiang Huang wrote: > Hi Pulsar Community, > > I would like to start a VOTE on "Provide new load balance placement > strategy implementation for ModularLoadManagerStrategy" (PIP-182). > The proposal can be read at https://github.com/apache/pulsar/issues/1

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

Hi Asaf, On 2022/07/11 13:08:52 Asaf Mesika wrote: > On Thu, Jun 23, 2022 at 6:26 AM Haiting Jiang > wrote: > > > Hi Asaf, > > > > > I did a quick reading and I couldn't understand the gist of this change: > > > The shadow topic doesn't reall

Re: [VOTE] Enable GitHub Discussion for user-facing discussion

+1 Thanks, Haiting On 2022/07/12 10:00:27 tison wrote: > Hi, > > In the previous email[1] I started a discussion about enabling GitHub > Discussions in apache/pulsar repository and we meet a consensus to avoid > making development discussions truth happen on the sources. It's also a > requiremen

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

t really, the replicator (producer) part is in source topic, and the "not really write to BK" part is in shadow topic. For the replicator, nothing much needs to be changed. Mostly, we just extend the replicating target from other broker clusters to current cluster, and adding a messa

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

> topic is in charge to push out messages (replicate) to a remote cluster > > topic, using the normal client (producer) for this. > > If I understand correctly, here, we re-use the replicator, which pushes out > > messages to a topic, but those messages won't really be writt

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

ure. > > > there are not much difference to reset the cursor in persistent > replicator or > in the raw reader. > > Do you want to control it in the shadow topic? or by users? > Maybe we can make some changes to the replicator to support skip backlogs. > > Thanks, &g

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

ses of data > replication e2e and modifying it only to go to the cache, and not write it. > It's less entanglement IMO. > 2. You keep the encapsulation and make it easier for client implementations > and people reasoning about the protocol. > > IMO changing the protocol requ

Re: [VOTE] PIP-187 Add API to analyze a subscription backlog and provide a accurate value

+1 Thanks, Haiting On 2022/07/23 02:00:32 PengHui Li wrote: > +1 > > Penghui > > On Wed, Jul 20, 2022 at 9:41 PM Asaf Mesika wrote: > > > Sorry to barge in the vote - I forgot to send my reply on the discussion 2 > > days ago :) > > > > > > On Tue, Jul 19, 2022 at 11:22 PM Nicolò Boschi > >

Re: [DISCUSS] Alternatives to changing public protocol

Hi Asaf, > (the field is not generic, > it's specifically named shadow_message_id). As Penghui suggested, this field name is changed to `message_id` for potential generic usage. :) > The second problem is clients: Every such field will eventually trickle > down to the clients, which will need

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

, Jul 11, 2022 at 9:09 PM Asaf Mesika wrote: > > > On Thu, Jun 23, 2022 at 6:26 AM Haiting Jiang > > wrote: > > > > > Hi Asaf, > > > > > > > I did a quick reading and I couldn't understand the gist of this > > change: > > &g

Re: [DISCUSS] Apache Pulsar 2.8.4 release

Hi Enrico, > We should also one last release out of 2.7 branch. > I hope that someone will volunteer I volunteer to be the release manager for 2.7.5. I will start the discuss mail shortly. Thanks, Haiting On 2022/07/21 16:56:50 Enrico Olivelli wrote: > Thank you Yunze > > I agree that it is ti

Re: [VOTE]PIP-189: No batching if only one message in batch.

+1 Thanks, Haiting On 2022/07/25 02:23:20 Anon Hxy wrote: > Dear Community, > > I would like to start a VOTE on "PIP-189: No batching if only one message > in batch." > > The proposal can be read at [0] and the discussion thread is available at > [1] and the PR link is available at [2] > > Voti

  1   2   3   >