Re: [VOTE] Pulsar Release 2.9.2 Candidate 2

[PengHui Li]

It's not a regression in 2.9.2, we should not block the 2.9.2 release.
Instead, we can have the fix in 2.9.3.

Penghui

On Thu, Feb 3, 2022 at 8:42 PM Nicolò Boschi  wrote:

> Hi Ran, thanks for driving the release.
>
> I haven't tested the rc yet but I firmly believe we should include this
> pull [1] which fixes a regression introduced in Pulsar 2.9.0
>
> [1] https://github.com/apache/pulsar/pull/14097
>
>
>
> Il giorno mer 2 feb 2022 alle ore 08:34 Enrico Olivelli <
> eolive...@gmail.com>
> ha scritto:
>
> > (sorry for the late reply, I am still testing, I had some other
> > priorities).
> >
> > I hope that the community will test this RC and report back
> >
> >
> > Enrico
> >
> > Il giorno mar 25 gen 2022 alle ore 15:07 Ran Gao  ha
> > scritto:
> > >
> > > Sorry, the 2.9.2 release candidate-1 has a wrong sign certificate.
> > >
> > > This is the second release candidate for Apache Pulsar, version 2.9.2.
> > >
> > > *** Please download, test, and vote on this release. This vote will
> stay
> > > open
> > > for at least 72 hours ***
> > >
> > > Note that we are voting upon the source (tag), binaries are provided
> for
> > > convenience.
> > >
> > > Source and binary files:
> > >
> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.9.2-candidate-2/
> > >
> > > SHA-512 checksums:
> > >
> > >
> >
> 563f65582c5307b4ef1e0322958ed19d7c181fb8bb8d7b8cab06ab0a6adb5520f7d18b6f97960b93c3318815529a8b8721e00e9cc9484532a2e5ed3221450094
> > >  ./apache-pulsar-2.9.2-bin.tar.gz
> > >
> >
> 60d1049611b938b0ddc769132124d43820728afc8a06813a5ec9efc095c5497c59d9bbcaaf7df5b0c0e97e051d66f59c1f8ee08885d05ca2c635773e0283770a
> > >  ./apache-pulsar-2.9.2-src.tar.gz
> > >
> > > Maven staging repo:
> > >
> https://repository.apache.org/content/repositories/orgapachepulsar-1136
> > >
> > > The tag to be voted upon:
> > > v2.9.2-candidate-2 (8a5d2253b888b3b865a2aedf635d672821c7)
> > > https://github.com/apache/pulsar/releases/tag/v2.9.2-candidate-2
> > >
> > > Pulsar's KEYS file containing PGP keys we use to sign the release:
> > > https://dist.apache.org/repos/dist/dev/pulsar/KEYS
> > >
> > > Please download the source package, and follow the README to build
> > > and run the Pulsar standalone service.
> >
>
>
> --
> Nicolò Boschi
>

[OUTREACH] Feb '22 Edition of 'Happenings in the Neighborhood' is out now

[Aaron Williams]

Hello Apache Pulsar Neighbors,

In this issue

of *Happenings, *we talk about a new PMC member, 2 new committers, a call
to help improve the quality of the code, a meetup in Italian, and a new and
improved home page coming soon.

If you have anything that you think your neighbors would find interesting,
we have created #blogs-articles and #event-decks channels on the Apache
Pulsar slack Workspace to capture them.

Thank you,
Aaron Williams
Resident of the Apache Pulsar Neighborhood

Re: [DISCUSS] The default value of maxPendingChunkedMessage

[Zike Yang]

Thanks for all your suggestions.

> For further improvement, I think we can deprecate `maxPendingChunkedMessage` 
> by extending the scope of  `ClientBuilder#memoryLimit` to consumers.

+1

I have created a PR to fix this inconsistency:
https://github.com/apache/pulsar/pull/14144 PTAL.

Thanks,
Zike Yang

On Sat, Feb 5, 2022 at 6:41 PM Haiting Jiang  wrote:
>
> > I agree with updating the Javadoc to align with the actual code. This
> > will lead to fewer surprises,
>
> +1.
>
> For further improvement, I think we can deprecate `maxPendingChunkedMessage` 
> by extending the scope of  `ClientBuilder#memoryLimit` to consumers.
>
> Thanks,
> Haiting
>
> On 2022/02/01 05:23:06 Michael Marshall wrote:
> > > We found that there are inconsistencies between the code and the
> > > documentation regarding the default value of maxPendingChunkedMessage.
> >
> > Great find!
> >
> > > A chunked buffer to contain all chunks could use much memory, for 
> > > example, if a
> > > message was split into N chunks, since each chunk is 5MB by default, then 
> > > 100
> > > buffers will use N*500 MB. It could reach 1GB if N > 2.
> >
> > This is a very good point.
> >
> > I agree with updating the Javadoc to align with the actual code. This
> > will lead to fewer surprises, and as Yunze Xu pointed out, a 10x
> > increase in the default could have dramatic effects on client memory
> > usage.
> >
> > Thanks,
> > Michael
> >
> >
> > On Sun, Jan 30, 2022 at 8:58 PM Zike Yang
> >  wrote:
> > >
> > > Hi, Yunze,
> > >
> > > Thanks for your opinion.
> > >
> > > > A chunked buffer to contain all chunks could use much memory, for 
> > > > example, if a
> > > > message was split into N chunks, since each chunk is 5MB by default, 
> > > > then 100
> > > > buffers will use N*500 MB. It could reach 1GB if N > 2.
> > > >
> > > > In addition, normally, only if at least 100 producers sent messages to a
> > > > partition would it be meaningful to configure maxPendingChunkedMessages 
> > > > to 100.
> > > > IMO, it's hard to see so many producers on a partition in production.
> > >
> > > +1. I agree with you. And keeping the current default value in the
> > > code (10) will not change the default behavior of the current client.
> > > If there are no other objections, I would like to fix this
> > > inconsistency in the java client.
> > >
> > > Thanks,
> > > Zike
> > >
> > >
> > > On Sun, Jan 30, 2022 at 7:26 PM Yunze Xu  
> > > wrote:
> > > >
> > > > After thinking for a while, I’d prefer 10 as the default value and I 
> > > > changed
> > > > the default value to 10 in C++ client, see
> > > > https://github.com/apache/pulsar/pull/14070.
> > > >
> > > > A chunked buffer to contain all chunks could use much memory, for 
> > > > example, if a
> > > > message was split into N chunks, since each chunk is 5MB by default, 
> > > > then 100
> > > > buffers will use N*500 MB. It could reach 1GB if N > 2.
> > > >
> > > > In addition, normally, only if at least 100 producers sent messages to a
> > > > partition would it be meaningful to configure maxPendingChunkedMessages 
> > > > to 100.
> > > > IMO, it's hard to see so many producers on a partition in production.
> > > >
> > > > Thanks,
> > > > Yunze Xu
> > > >
> > > > > 2022年1月30日 下午6:32，Zike Yang  写道：
> > > > >
> > > > > Hi, Pulsar community,
> > > > >
> > > > > We found that there are inconsistencies between the code and the
> > > > > documentation regarding the default value of maxPendingChunkedMessage.
> > > > >
> > > > > In the java client code, we use 10 as the default value. [1] But in
> > > > > the java doc, we use 100 as the default value. [2]
> > > > > We need to fix this inconsistency. But what should we take as the
> > > > > default value? From the code or the doc? I would like to hear your
> > > > > discussions.
> > > > >
> > > > > [1] 
> > > > > https://github.com/apache/pulsar/blob/d11147616aa6cc7888420f6325bb71cd7f7ab065/pulsar-client/src/main/java/org/apache/pulsar/client/impl/conf/ConsumerConfigurationData.java#L112-L113
> > > > > [2] 
> > > > > https://github.com/apache/pulsar/blob/1e2ff8a3941b7cc6d583f528ceedc393b7e607fb/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/ConsumerBuilder.java#L690
> > > > >
> > > > > Thanks,
> > > > > Zike Yang
> > > >
> >

Re: [VOTE] Pulsar Release 2.9.2 Candidate 2

[PengHui Li]

+1 (binding)

1. Checked the signature
2. Build from the source successfully
3. Start standalone
4. Publish and consume successfully
5. Cassandra connect works well
6. Checked state function

And passed our internal integration tests.

Regards,
Penghui

On Mon, Feb 7, 2022 at 4:37 PM PengHui Li  wrote:

> It's not a regression in 2.9.2, we should not block the 2.9.2 release.
> Instead, we can have the fix in 2.9.3.
>
> Penghui
>
> On Thu, Feb 3, 2022 at 8:42 PM Nicolò Boschi  wrote:
>
>> Hi Ran, thanks for driving the release.
>>
>> I haven't tested the rc yet but I firmly believe we should include this
>> pull [1] which fixes a regression introduced in Pulsar 2.9.0
>>
>> [1] https://github.com/apache/pulsar/pull/14097
>>
>>
>>
>> Il giorno mer 2 feb 2022 alle ore 08:34 Enrico Olivelli <
>> eolive...@gmail.com>
>> ha scritto:
>>
>> > (sorry for the late reply, I am still testing, I had some other
>> > priorities).
>> >
>> > I hope that the community will test this RC and report back
>> >
>> >
>> > Enrico
>> >
>> > Il giorno mar 25 gen 2022 alle ore 15:07 Ran Gao  ha
>> > scritto:
>> > >
>> > > Sorry, the 2.9.2 release candidate-1 has a wrong sign certificate.
>> > >
>> > > This is the second release candidate for Apache Pulsar, version 2.9.2.
>> > >
>> > > *** Please download, test, and vote on this release. This vote will
>> stay
>> > > open
>> > > for at least 72 hours ***
>> > >
>> > > Note that we are voting upon the source (tag), binaries are provided
>> for
>> > > convenience.
>> > >
>> > > Source and binary files:
>> > >
>> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.9.2-candidate-2/
>> > >
>> > > SHA-512 checksums:
>> > >
>> > >
>> >
>> 563f65582c5307b4ef1e0322958ed19d7c181fb8bb8d7b8cab06ab0a6adb5520f7d18b6f97960b93c3318815529a8b8721e00e9cc9484532a2e5ed3221450094
>> > >  ./apache-pulsar-2.9.2-bin.tar.gz
>> > >
>> >
>> 60d1049611b938b0ddc769132124d43820728afc8a06813a5ec9efc095c5497c59d9bbcaaf7df5b0c0e97e051d66f59c1f8ee08885d05ca2c635773e0283770a
>> > >  ./apache-pulsar-2.9.2-src.tar.gz
>> > >
>> > > Maven staging repo:
>> > >
>> https://repository.apache.org/content/repositories/orgapachepulsar-1136
>> > >
>> > > The tag to be voted upon:
>> > > v2.9.2-candidate-2 (8a5d2253b888b3b865a2aedf635d672821c7)
>> > > https://github.com/apache/pulsar/releases/tag/v2.9.2-candidate-2
>> > >
>> > > Pulsar's KEYS file containing PGP keys we use to sign the release:
>> > > https://dist.apache.org/repos/dist/dev/pulsar/KEYS
>> > >
>> > > Please download the source package, and follow the README to build
>> > > and run the Pulsar standalone service.
>> >
>>
>>
>> --
>> Nicolò Boschi
>>
>

[GitHub] [pulsar-helm-chart] shaileshjha061 commented on issue #222: Zookeeper Pod Restarts frequently

[GitBox]

shaileshjha061 commented on issue #222:
URL: 
https://github.com/apache/pulsar-helm-chart/issues/222#issuecomment-1031342601


   Yes. The Issue is Resolved. Thanks, @lhotari 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-helm-chart] shaileshjha061 closed issue #222: Zookeeper Pod Restarts frequently

[GitBox]

shaileshjha061 closed issue #222:
URL: https://github.com/apache/pulsar-helm-chart/issues/222


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-helm-chart] elangelo opened a new pull request #225: allow specifying the nodeSelector for the init jobs

[GitBox]

elangelo opened a new pull request #225:
URL: https://github.com/apache/pulsar-helm-chart/pull/225


   Fixes #
   
   ### Motivation
   
   When deploying pulsar to an AKS cluster with windows nodepools i was unable 
to specify that the Jobs of the initalize release had to run on linux nodes. 
With the change i can now specify a node selector for the init jobs.
   
   ### Modifications
   
   add nodeSelector on pulsar_init and bookie_init
   
   ### Verifying this change
   
   - [ ] Make sure that the change passes the CI checks.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[discuss] prometheus metrics doesn't satisfy with OpenMetrics format

[ZhangJian He]

Before, I am working on bumping Prometheus client to 0.12.0, but they
introduce a breaking change,
https://github.com/prometheus/client_java/pull/615, adopt the `OpenMetrics
format`, which acquired all counters have `_total` suffix,

but our metrics now have these metrics, there are not satisfied with the
OpenMetrics format, for example:

- pulsar_connection_closed_total_count

- pulsar_connection_created_total_count

- pulsar_source_received_total_1min

- system_exceptions_total_1min


I want to discuss, Should we adapt the `OpenMetrics format`?

If we want to be compatible with Open Metrics, I suggest adding metrics
named `_total` in a release version like 2.10.0, and removing the origin
metric in the next release like 2.11.0.

Re: [discuss] prometheus metrics doesn't satisfy with OpenMetrics format

[Enrico Olivelli]

What happens when you upgrade the Prometheus client ?

Can you share some examples of "before" and "after" ?
My understanding is that you posted how it looks like "after" the upgrade

Thanks for working on this

Enrico

Il giorno mar 8 feb 2022 alle ore 08:21 ZhangJian He
 ha scritto:
>
> Before, I am working on bumping Prometheus client to 0.12.0, but they
> introduce a breaking change,
> https://github.com/prometheus/client_java/pull/615, adopt the `OpenMetrics
> format`, which acquired all counters have `_total` suffix,
>
> but our metrics now have these metrics, there are not satisfied with the
> OpenMetrics format, for example:
>
> - pulsar_connection_closed_total_count
>
> - pulsar_connection_created_total_count
>
> - pulsar_source_received_total_1min
>
> - system_exceptions_total_1min
>
>
> I want to discuss, Should we adapt the `OpenMetrics format`?
>
> If we want to be compatible with Open Metrics, I suggest adding metrics
> named `_total` in a release version like 2.10.0, and removing the origin
> metric in the next release like 2.11.0.