Re: [Discuss][PIP-143] Support split paritions belonging to specified topics in a bundle

[Aloys Zhang]

Hi penghui and haiting,

I try to figure out how the (anti-)affinity works.

> if I understand correctly, it looks like if we have a partitioned topic
with 10
> partitions under a namespace with 16 bundles, if applies the
anti-affinity policy,
> partition-0 map to bundle 0, partition-1 map to bundle 1, and so on.
> Of course, it is not necessary for every partitioned topic to start from
bundle 0,
> we can use the partition-0 hash to determine the start bundle index.

As penghui described, I think this is a mechanism for assigning topics to a
bundle that controls how a topic is mapped to a bundle.

> IMO, this affinity serves the purpose of isolating an abnormal topic to
some spare
> brokers.  These brokers host these kind of topics only. Here are some
cases :

And as haiting mentioned here, It's more like an isolation policy that
decides topics can be owned by which broker.
So, I am a little confused about how the (anti-)affinity works now.

Back to this PIP which aims to solve the problem that a small number of
topics in a bundle have a load that exceeds the average.
So, we can
1. get the positions for the topics with we are interested( need a new API)
2. calculate the position to split this bundle(also need a new API)
I think this way is enough for solving the problem.

And the (anti-)affinity way needs more discussion or maybe we can introduce
a new PIP for it.
What do you think?@penghui @haiting


Thanks,
Aloys

Aloys Zhang  于2022年2月21日周一 14:39写道：

> Hi, penghui
>
> >  The new API does not necessarily have to query by topic one by one,
> we have listed all the "topic -> position" of a bundle?
>
> I see. After we got all the positions of the topics we want to split in a
> bundle, it's quite easy for us to decide how to it.
>
> Haiting Jiang  于2022年2月20日周日 12:05写道：
>
>> > Do you have an example for affinity? I don't fully understand how this
>> is
>> > used
>> > in practice.
>>
>> IMO, this affinity serves the purpose of isolating an abnormal topic to
>> some spare
>> brokers.  These brokers host these kind of topics only. Here are some
>> cases :
>>
>> 1. A topic may have unexpected short spike traffic flows periodically and
>> causing broker overloads and negative impact on other topics.
>> Until we have more proper solutions, we can always isolate these topics
>> first,
>>  and make the service recover time as small as possible.
>>
>> 2. Some users may encounter some bugs in brokers, and we can isolate the
>> topic to
>> exclusive brokers, and use more radical approach to locate the bug, like
>> enable debug
>> level logs or even add some temporary code patch.
>>
>> 3. User may already have configured failure domain and anti-affinity
>> namespace, but with
>> business logic code changes, some topic may need to migrate from one
>> namespace
>> to another. This will take some time for user to change the client side
>> config.
>> In the meanwhile, we can isolate the topic first.
>>
>> Thanks,
>> Haiting
>>
>> On 2022/02/18 15:26:09 PengHui Li wrote:
>> > Hi Haiting,
>> >
>> > > I think this approach have more potential with abnormal topic
>> isolation.
>> > If we can introduce
>> > some kind of bundle isolation strategy, (like broker-bundle affinity and
>> > anti-affinity mechanism), we can easily isolate some unexpected traffic
>> to
>> > some empty brokers.
>> > IMO, this would improve the stability of broker cluster.
>> >
>> > if I understand correctly, it looks like if we have a partitioned topic
>> > with 10
>> > partitions under a namespace with 16 bundles, if applies the
>> anti-affinity
>> > policy,
>> > partition-0 map to bundle 0, partition-1 map to bundle 1, and so on.
>> > Of course, it is not necessary for every partitioned topic to start from
>> > bundle 0,
>> > we can use the partition-0 hash to determine the start bundle index.
>> >
>> > Do you have an example for affinity? I don't fully understand how this
>> is
>> > used
>> > in practice.
>> >
>> > Best,
>> > Penghui
>> >
>> > On Fri, Feb 18, 2022 at 11:16 PM PengHui Li  wrote:
>> >
>> > > Hi Aloys,
>> > >
>> > > >  Do you mean that
>> > > 1. First, add a new API, maybe `getHashPositioin`,  to get the hash
>> > > position in a bundle
>> > > 2. Then use this position to split the overloaded bundle
>> > > If so, when we split a bundle with multi partitions of a topic, we
>> need to
>> > > call the `getHashPositioin` multi times to get the middle position of
>> all
>> > > these positions.
>> > >
>> > > Yes, this want I mean. In this way, users can control to assign 1
>> topic or
>> > > 3 topics to one bundle. This is more like increasing the transparency
>> of
>> > > the topic in the bundle, you can all the positions of the topics, so
>> how
>> > > planning for bundle splitting becomes more flexible.
>> > >
>> > > The new API does not necessarily have to query by topic one by one,
>> > > we have listed all the "topic -> position" of a bundle?
>> > >
>> > > Thanks,
>> > > Penghui
>> > >
>> > > On Fri, Feb 18, 2022 at 4:51 PM Haiting Jiang <

[GitHub] [pulsar-site] urfreespace opened a new pull request #5: fix: use waves lib with cdn

[GitBox]

urfreespace opened a new pull request #5:
URL: https://github.com/apache/pulsar-site/pull/5


   Signed-off-by: LiLi 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-site] urfreespace merged pull request #5: fix: use waves lib with cdn

[GitBox]

urfreespace merged pull request #5:
URL: https://github.com/apache/pulsar-site/pull/5


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: Improve quality and efficiency of image creation

[Yu]

Hi Enrico, thanks for your suggestion! I've created an INFRA ticket on ASF:
https://issues.apache.org/jira/browse/INFRA-22912.

On Fri, Feb 18, 2022 at 4:13 PM Enrico Olivelli  wrote:

> Can we verify if there is some license for opensource projects or
> especially for Apache?
>
> Maybe you can open a INFRA ticket on ASF jira
>
>
> Enrico
>
>
> Il Mer 16 Feb 2022, 04:23 Yu  ha scritto:
>
>> Hi Enrico, do you mean the price of LucidChart? Here it is:
>>
>> [image: image.png]
>>
>>
>> https://lucid.app/pricing/lucidchart?gclid=EAIaIQobChMI3oGGyKGD9gIVnpFmAh1_gwr1EAAYASAAEgI7q_D_BwE#/pricing?utm_source=google&utm_medium=cpc&utm_campaign=_chart_en_tier3_mixed_search_brand_bmm_&km_CPC_CampaignId=1484560204&km_CPC_AdGroupID=60168113031&km_CPC_Keyword=%2Blucidchart%20%2Bprice&km_CPC_MatchType=b&km_CPC_ExtensionID=&km_CPC_Network=g&km_CPC_AdPosition=&km_CPC_Creative=442433233166&km_CPC_TargetID=aud-536921399221:kwd-402463205246&km_CPC_Country=9061446&km_CPC_Device=c&km_CPC_placement=&km_CPC_target=
>>
>>
>>
>>
>> On Tue, Feb 15, 2022 at 8:17 PM Enrico Olivelli 
>> wrote:
>>
>>> Yu,
>>>
>>> Il Mar 15 Feb 2022, 13:12 Yu  ha scritto:
>>>
 Bumping up this thread, any thoughts? Thanks

 On Fri, Jan 28, 2022 at 1:00 AM Yu  wrote:

 > Hi Pulsarers,
 >
 >
 > As you may notice, the images in Pulsar documentation are in
 inconsistent
 > styles, which affect user experience. This issue may be caused by:
 >
 > 1) Pulsar community does not have design style guides.
 >
 > 2) Original images are not shared, so that contributors can not be
 able to
 > re-use the exiting images, and then they create their own.
 >
 >
 > Solving this issue brings many benefits, such as:
 >
 > - Build a trusting relationship with Pulsar users.
 >
 > - Save image contributors' efforts and time in the design process.
 >
 > - Help design work communicates clearly and looks professional.
 >
 >
 > And for 2), are there any good solutions?
 >
 > Hosting original images on GitHub repo may not be a pretty good idea
 > since:
 >
 > - Collaborating synchronously online is impossible. You need to
 download
 > the original file, re-use/edit it, and upload it back. Few
 contributors
 > would like to do it.
 >
 > - Occupy much space.
 >
 >
 > Can we choose a professional tool to create/edit/host images and
 > collaborate online synchronously?
 >
 > Just my two cents: for example, can ASF give the community an email
 > address and register an account on LucidChart? PMC can manage the
 > LucidChart Pulsar account and all image assets and grant different
 access
 > to contributors.
>>>
>>>
>>> How much does it cost?
>>>
>>> Enrico
>>>
>>> So that contributors can re-use/collaborate in an
 > efficient manner.
 >
 >
 > Please correct me if my understanding is incorrect, I'd love your
 > feedback, thanks!
 >

>>>

Re: [Discuss][PIP-143] Support split paritions belonging to specified topics in a bundle

[Haiting Jiang]

> 2. calculate the position to split this bundle(also need a new API)
Should this be "positions"? We are going to split one bundle into 
multi-bundles,  
in most cases, bundle number will be position number + 1, right? 

> And the (anti-)affinity way needs more discussion or maybe we can introduce
> a new PIP for it.
+1, this is not in the scope of this PIP.


Thanks,
Haiting

On 2022/02/21 08:57:01 Aloys Zhang wrote:
> Hi penghui and haiting,
> 
> I try to figure out how the (anti-)affinity works.
> 
> > if I understand correctly, it looks like if we have a partitioned topic
> with 10
> > partitions under a namespace with 16 bundles, if applies the
> anti-affinity policy,
> > partition-0 map to bundle 0, partition-1 map to bundle 1, and so on.
> > Of course, it is not necessary for every partitioned topic to start from
> bundle 0,
> > we can use the partition-0 hash to determine the start bundle index.
> 
> As penghui described, I think this is a mechanism for assigning topics to a
> bundle that controls how a topic is mapped to a bundle.
> 
> > IMO, this affinity serves the purpose of isolating an abnormal topic to
> some spare
> > brokers.  These brokers host these kind of topics only. Here are some
> cases :
> 
> And as haiting mentioned here, It's more like an isolation policy that
> decides topics can be owned by which broker.
> So, I am a little confused about how the (anti-)affinity works now.
> 
> Back to this PIP which aims to solve the problem that a small number of
> topics in a bundle have a load that exceeds the average.
> So, we can
> 1. get the positions for the topics with we are interested( need a new API)
> 2. calculate the position to split this bundle(also need a new API)
> I think this way is enough for solving the problem.
> 
> And the (anti-)affinity way needs more discussion or maybe we can introduce
> a new PIP for it.
> What do you think?@penghui @haiting
> 
> 
> Thanks,
> Aloys
> 
> Aloys Zhang  于2022年2月21日周一 14:39写道：
> 
> > Hi, penghui
> >
> > >  The new API does not necessarily have to query by topic one by one,
> > we have listed all the "topic -> position" of a bundle?
> >
> > I see. After we got all the positions of the topics we want to split in a
> > bundle, it's quite easy for us to decide how to it.
> >
> > Haiting Jiang  于2022年2月20日周日 12:05写道：
> >
> >> > Do you have an example for affinity? I don't fully understand how this
> >> is
> >> > used
> >> > in practice.
> >>
> >> IMO, this affinity serves the purpose of isolating an abnormal topic to
> >> some spare
> >> brokers.  These brokers host these kind of topics only. Here are some
> >> cases :
> >>
> >> 1. A topic may have unexpected short spike traffic flows periodically and
> >> causing broker overloads and negative impact on other topics.
> >> Until we have more proper solutions, we can always isolate these topics
> >> first,
> >>  and make the service recover time as small as possible.
> >>
> >> 2. Some users may encounter some bugs in brokers, and we can isolate the
> >> topic to
> >> exclusive brokers, and use more radical approach to locate the bug, like
> >> enable debug
> >> level logs or even add some temporary code patch.
> >>
> >> 3. User may already have configured failure domain and anti-affinity
> >> namespace, but with
> >> business logic code changes, some topic may need to migrate from one
> >> namespace
> >> to another. This will take some time for user to change the client side
> >> config.
> >> In the meanwhile, we can isolate the topic first.
> >>
> >> Thanks,
> >> Haiting
> >>
> >> On 2022/02/18 15:26:09 PengHui Li wrote:
> >> > Hi Haiting,
> >> >
> >> > > I think this approach have more potential with abnormal topic
> >> isolation.
> >> > If we can introduce
> >> > some kind of bundle isolation strategy, (like broker-bundle affinity and
> >> > anti-affinity mechanism), we can easily isolate some unexpected traffic
> >> to
> >> > some empty brokers.
> >> > IMO, this would improve the stability of broker cluster.
> >> >
> >> > if I understand correctly, it looks like if we have a partitioned topic
> >> > with 10
> >> > partitions under a namespace with 16 bundles, if applies the
> >> anti-affinity
> >> > policy,
> >> > partition-0 map to bundle 0, partition-1 map to bundle 1, and so on.
> >> > Of course, it is not necessary for every partitioned topic to start from
> >> > bundle 0,
> >> > we can use the partition-0 hash to determine the start bundle index.
> >> >
> >> > Do you have an example for affinity? I don't fully understand how this
> >> is
> >> > used
> >> > in practice.
> >> >
> >> > Best,
> >> > Penghui
> >> >
> >> > On Fri, Feb 18, 2022 at 11:16 PM PengHui Li  wrote:
> >> >
> >> > > Hi Aloys,
> >> > >
> >> > > >  Do you mean that
> >> > > 1. First, add a new API, maybe `getHashPositioin`,  to get the hash
> >> > > position in a bundle
> >> > > 2. Then use this position to split the overloaded bundle
> >> > > If so, when we split a bundle with multi partitions of a topic, we
> >> need 

Received error from server: Namespace bundle for topic xxx not served by this instance. Please redo the lookup. Request is denied

[tamer Abdlatif]

Hi Team,

We are running our consumer application as pods in Kubernetes ..3/5 of our
consumers are working fine , however ,2/5 are receiving the subject error .

Any idea what could be the root cause for that error ?

Your quick response is highly appreciated

Thanks and best regards,
Tamer

Re: [Discuss][PIP-143] Support split paritions belonging to specified topics in a bundle

[PengHui Li]

> And the (anti-)affinity way needs more discussion or maybe we can
introduce
a new PIP for it.

+1

Thanks,
Penghui

On Mon, Feb 21, 2022 at 6:47 PM Haiting Jiang 
wrote:

> > 2. calculate the position to split this bundle(also need a new API)
> Should this be "positions"? We are going to split one bundle into
> multi-bundles,
> in most cases, bundle number will be position number + 1, right?
>
> > And the (anti-)affinity way needs more discussion or maybe we can
> introduce
> > a new PIP for it.
> +1, this is not in the scope of this PIP.
>
>
> Thanks,
> Haiting
>
> On 2022/02/21 08:57:01 Aloys Zhang wrote:
> > Hi penghui and haiting,
> >
> > I try to figure out how the (anti-)affinity works.
> >
> > > if I understand correctly, it looks like if we have a partitioned topic
> > with 10
> > > partitions under a namespace with 16 bundles, if applies the
> > anti-affinity policy,
> > > partition-0 map to bundle 0, partition-1 map to bundle 1, and so on.
> > > Of course, it is not necessary for every partitioned topic to start
> from
> > bundle 0,
> > > we can use the partition-0 hash to determine the start bundle index.
> >
> > As penghui described, I think this is a mechanism for assigning topics
> to a
> > bundle that controls how a topic is mapped to a bundle.
> >
> > > IMO, this affinity serves the purpose of isolating an abnormal topic to
> > some spare
> > > brokers.  These brokers host these kind of topics only. Here are some
> > cases :
> >
> > And as haiting mentioned here, It's more like an isolation policy that
> > decides topics can be owned by which broker.
> > So, I am a little confused about how the (anti-)affinity works now.
> >
> > Back to this PIP which aims to solve the problem that a small number of
> > topics in a bundle have a load that exceeds the average.
> > So, we can
> > 1. get the positions for the topics with we are interested( need a new
> API)
> > 2. calculate the position to split this bundle(also need a new API)
> > I think this way is enough for solving the problem.
> >
> > And the (anti-)affinity way needs more discussion or maybe we can
> introduce
> > a new PIP for it.
> > What do you think?@penghui @haiting
> >
> >
> > Thanks,
> > Aloys
> >
> > Aloys Zhang  于2022年2月21日周一 14:39写道：
> >
> > > Hi, penghui
> > >
> > > >  The new API does not necessarily have to query by topic one by one,
> > > we have listed all the "topic -> position" of a bundle?
> > >
> > > I see. After we got all the positions of the topics we want to split
> in a
> > > bundle, it's quite easy for us to decide how to it.
> > >
> > > Haiting Jiang  于2022年2月20日周日 12:05写道：
> > >
> > >> > Do you have an example for affinity? I don't fully understand how
> this
> > >> is
> > >> > used
> > >> > in practice.
> > >>
> > >> IMO, this affinity serves the purpose of isolating an abnormal topic
> to
> > >> some spare
> > >> brokers.  These brokers host these kind of topics only. Here are some
> > >> cases :
> > >>
> > >> 1. A topic may have unexpected short spike traffic flows periodically
> and
> > >> causing broker overloads and negative impact on other topics.
> > >> Until we have more proper solutions, we can always isolate these
> topics
> > >> first,
> > >>  and make the service recover time as small as possible.
> > >>
> > >> 2. Some users may encounter some bugs in brokers, and we can isolate
> the
> > >> topic to
> > >> exclusive brokers, and use more radical approach to locate the bug,
> like
> > >> enable debug
> > >> level logs or even add some temporary code patch.
> > >>
> > >> 3. User may already have configured failure domain and anti-affinity
> > >> namespace, but with
> > >> business logic code changes, some topic may need to migrate from one
> > >> namespace
> > >> to another. This will take some time for user to change the client
> side
> > >> config.
> > >> In the meanwhile, we can isolate the topic first.
> > >>
> > >> Thanks,
> > >> Haiting
> > >>
> > >> On 2022/02/18 15:26:09 PengHui Li wrote:
> > >> > Hi Haiting,
> > >> >
> > >> > > I think this approach have more potential with abnormal topic
> > >> isolation.
> > >> > If we can introduce
> > >> > some kind of bundle isolation strategy, (like broker-bundle
> affinity and
> > >> > anti-affinity mechanism), we can easily isolate some unexpected
> traffic
> > >> to
> > >> > some empty brokers.
> > >> > IMO, this would improve the stability of broker cluster.
> > >> >
> > >> > if I understand correctly, it looks like if we have a partitioned
> topic
> > >> > with 10
> > >> > partitions under a namespace with 16 bundles, if applies the
> > >> anti-affinity
> > >> > policy,
> > >> > partition-0 map to bundle 0, partition-1 map to bundle 1, and so on.
> > >> > Of course, it is not necessary for every partitioned topic to start
> from
> > >> > bundle 0,
> > >> > we can use the partition-0 hash to determine the start bundle index.
> > >> >
> > >> > Do you have an example for affinity? I don't fully understand how
> this
> > >> is
> > >> > used
> > >>

Pulsar - Transaction Message Detail Question

[yutao chen]

Hi, I'm very interested in Pulsar transaction message, and I found this
issue 
Would you please tell me why pulsar finally chose to replace sidecar's
appraoch with marker's appraoch, which seems different from the conclusion
in the previous PIP-31

[PIP] #14395 Making SchemaRegistry implementation configurable

[Aparajita Singh]

Hi,
Please review this proposal: https://github.com/apache/pulsar/issues/14395

-- 
Thanks,
Aparajita

[GitHub] [pulsar-helm-chart] bsheltonihs opened a new pull request #236: To address the function role vs clusterrole issue

[GitBox]

bsheltonihs opened a new pull request #236:
URL: https://github.com/apache/pulsar-helm-chart/pull/236


   Fixes #230
   
   ### Motivation
   
   *When functions are enabled it deploys only with clusterrole and 
clusterrolebinding. However the use case I have is I am unable to those within 
a namespace.*
   
   ### Modifications
   
   *I followed the example in the already approved file 
"broker-cluster-role-binding.yaml" for the approach to deal with this issue. It 
allows for a role and rolebinding or clusterrole and clusterrolebinding and 
doesn't force you into only one.*
   
   ### Verifying this change
   
   - [ ] Make sure that the change passes the CI checks.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-helm-chart] toneill818 opened a new pull request #237: Cert Fix

[GitBox]

toneill818 opened a new pull request #237:
URL: https://github.com/apache/pulsar-helm-chart/pull/237


   
   Fix templates to work with cert-manager 1.5.4 since that is the version that 
the script will install


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-helm-chart] toneill818 closed pull request #237: Cert Fix

[GitBox]

toneill818 closed pull request #237:
URL: https://github.com/apache/pulsar-helm-chart/pull/237


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [ANNOUNCE] Apache Pulsar Go Client 0.8.0 released

[Enrico Olivelli]

Hi,
Did you store the source code tarball on the dist.apache.org website?

For a valid Apache release we must release the source tarball in the
official repo.


Enrico

Il Lun 21 Feb 2022, 04:25 r...@apache.org  ha
scritto:

> The Apache Pulsar team is proud to announce Apache Pulsar Go Client version
> 0.8.0.
>
> Pulsar is a highly scalable, low latency messaging platform running on
> commodity hardware. It provides simple pub-sub semantics over topics,
> guaranteed at-least-once delivery of messages, automatic cursor management
> for
> subscribers, and cross-datacenter replication.
>
> For Pulsar release details and downloads, visit:
> https://github.com/apache/pulsar-client-go/releases/tag/v0.8.0
>
> Release Notes are at:
> https://github.com/apache/pulsar-client-go/blob/master/CHANGELOG.md
>
> We would like to thank the contributors that made the release possible.
>
> Regards,
>
> The Pulsar Team
>

[GitHub] [pulsar-helm-chart] toneill818 opened a new pull request #238: Cert Fix

[GitBox]

toneill818 opened a new pull request #238:
URL: https://github.com/apache/pulsar-helm-chart/pull/238


   ### Motivation
   
   In the scripts folder there is a script to install [cert-manager 
1.5.4](https://cert-manager.io/v1.5-docs/usage/certificate/), but the chart's 
templates do not work with that version.
   
   ### Modifications
   
   - Moved `keyAlgorithm`, `keySize`, and `keyEncoding` under the `privateKey` 
object
   - Moved `organization` under subject and renamed to `organizations`
   - Bumped Chart Version
   
   ### Verifying this change
   
   - [ ] Make sure that the change passes the CI checks.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [ANNOUNCE] New Committer: Li Li

[Dianjin Wang]

Congratulations!

Best,
Dianjin Wang


On Tue, Feb 15, 2022 at 3:08 PM PengHui Li  wrote:

> The Apache Pulsar Project Management Committee (PMC) has invited Li Li
> https://github.com/urfreespace to become a committer and we are pleased to
> announce that he has accepted.
>
> Li Li has done a great contribution to Pulsar Website, documentation.
>
> Welcome and Congratulations, Li Li!
>
> Please join us in congratulating and welcoming Li Li onboard!
>
> Best Regards,
> Penghui Li on behalf of the Pulsar PMC
>

Re: [ANNOUNCE] New Committer: Li Li

[guo jiwei]

Congrats


Regards
Jiwei Guo (Tboy)


On Tue, Feb 22, 2022 at 8:28 AM Dianjin Wang 
wrote:

> Congratulations!
>
> Best,
> Dianjin Wang
>
>
> On Tue, Feb 15, 2022 at 3:08 PM PengHui Li  wrote:
>
> > The Apache Pulsar Project Management Committee (PMC) has invited Li Li
> > https://github.com/urfreespace to become a committer and we are pleased
> to
> > announce that he has accepted.
> >
> > Li Li has done a great contribution to Pulsar Website, documentation.
> >
> > Welcome and Congratulations, Li Li!
> >
> > Please join us in congratulating and welcoming Li Li onboard!
> >
> > Best Regards,
> > Penghui Li on behalf of the Pulsar PMC
> >
>

Re: [Discuss][PIP-143] Support split paritions belonging to specified topics in a bundle

[Aloys Zhang]

>Should this be "positions"? We are going to split one bundle into
multi-bundles,
in most cases, bundle number will be position number + 1, right?

Sure, it should be “positions” or “positionList”

PengHui Li  于2022年2月21日周一 20:50写道：

> > And the (anti-)affinity way needs more discussion or maybe we can
> introduce
> a new PIP for it.
>
> +1
>
> Thanks,
> Penghui
>
> On Mon, Feb 21, 2022 at 6:47 PM Haiting Jiang 
> wrote:
>
> > > 2. calculate the position to split this bundle(also need a new API)
> > Should this be "positions"? We are going to split one bundle into
> > multi-bundles,
> > in most cases, bundle number will be position number + 1, right?
> >
> > > And the (anti-)affinity way needs more discussion or maybe we can
> > introduce
> > > a new PIP for it.
> > +1, this is not in the scope of this PIP.
> >
> >
> > Thanks,
> > Haiting
> >
> > On 2022/02/21 08:57:01 Aloys Zhang wrote:
> > > Hi penghui and haiting,
> > >
> > > I try to figure out how the (anti-)affinity works.
> > >
> > > > if I understand correctly, it looks like if we have a partitioned
> topic
> > > with 10
> > > > partitions under a namespace with 16 bundles, if applies the
> > > anti-affinity policy,
> > > > partition-0 map to bundle 0, partition-1 map to bundle 1, and so on.
> > > > Of course, it is not necessary for every partitioned topic to start
> > from
> > > bundle 0,
> > > > we can use the partition-0 hash to determine the start bundle index.
> > >
> > > As penghui described, I think this is a mechanism for assigning topics
> > to a
> > > bundle that controls how a topic is mapped to a bundle.
> > >
> > > > IMO, this affinity serves the purpose of isolating an abnormal topic
> to
> > > some spare
> > > > brokers.  These brokers host these kind of topics only. Here are some
> > > cases :
> > >
> > > And as haiting mentioned here, It's more like an isolation policy that
> > > decides topics can be owned by which broker.
> > > So, I am a little confused about how the (anti-)affinity works now.
> > >
> > > Back to this PIP which aims to solve the problem that a small number of
> > > topics in a bundle have a load that exceeds the average.
> > > So, we can
> > > 1. get the positions for the topics with we are interested( need a new
> > API)
> > > 2. calculate the position to split this bundle(also need a new API)
> > > I think this way is enough for solving the problem.
> > >
> > > And the (anti-)affinity way needs more discussion or maybe we can
> > introduce
> > > a new PIP for it.
> > > What do you think?@penghui @haiting
> > >
> > >
> > > Thanks,
> > > Aloys
> > >
> > > Aloys Zhang  于2022年2月21日周一 14:39写道：
> > >
> > > > Hi, penghui
> > > >
> > > > >  The new API does not necessarily have to query by topic one by
> one,
> > > > we have listed all the "topic -> position" of a bundle?
> > > >
> > > > I see. After we got all the positions of the topics we want to split
> > in a
> > > > bundle, it's quite easy for us to decide how to it.
> > > >
> > > > Haiting Jiang  于2022年2月20日周日 12:05写道：
> > > >
> > > >> > Do you have an example for affinity? I don't fully understand how
> > this
> > > >> is
> > > >> > used
> > > >> > in practice.
> > > >>
> > > >> IMO, this affinity serves the purpose of isolating an abnormal topic
> > to
> > > >> some spare
> > > >> brokers.  These brokers host these kind of topics only. Here are
> some
> > > >> cases :
> > > >>
> > > >> 1. A topic may have unexpected short spike traffic flows
> periodically
> > and
> > > >> causing broker overloads and negative impact on other topics.
> > > >> Until we have more proper solutions, we can always isolate these
> > topics
> > > >> first,
> > > >>  and make the service recover time as small as possible.
> > > >>
> > > >> 2. Some users may encounter some bugs in brokers, and we can isolate
> > the
> > > >> topic to
> > > >> exclusive brokers, and use more radical approach to locate the bug,
> > like
> > > >> enable debug
> > > >> level logs or even add some temporary code patch.
> > > >>
> > > >> 3. User may already have configured failure domain and anti-affinity
> > > >> namespace, but with
> > > >> business logic code changes, some topic may need to migrate from one
> > > >> namespace
> > > >> to another. This will take some time for user to change the client
> > side
> > > >> config.
> > > >> In the meanwhile, we can isolate the topic first.
> > > >>
> > > >> Thanks,
> > > >> Haiting
> > > >>
> > > >> On 2022/02/18 15:26:09 PengHui Li wrote:
> > > >> > Hi Haiting,
> > > >> >
> > > >> > > I think this approach have more potential with abnormal topic
> > > >> isolation.
> > > >> > If we can introduce
> > > >> > some kind of bundle isolation strategy, (like broker-bundle
> > affinity and
> > > >> > anti-affinity mechanism), we can easily isolate some unexpected
> > traffic
> > > >> to
> > > >> > some empty brokers.
> > > >> > IMO, this would improve the stability of broker cluster.
> > > >> >
> > > >> > if I understand correctly, it looks like if we have a partiti

[GitHub] [pulsar-client-node] nkurihar closed issue #188: When installing the Pulsar Node.js client, the operation steps need to be optimized

[GitBox]

nkurihar closed issue #188:
URL: https://github.com/apache/pulsar-client-node/issues/188


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-client-node] nkurihar commented on issue #188: When installing the Pulsar Node.js client, the operation steps need to be optimized

[GitBox]

nkurihar commented on issue #188:
URL: 
https://github.com/apache/pulsar-client-node/issues/188#issuecomment-1047407354


   Thanks for fixing the document 👍 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org