Re: [PIP-78] Reduce redundant producers from partitioned producer

[Enrico Olivelli]

Yuri,
IIUC with this change the client will control which metrics are reported by
the broker ?

I am not sure it is a good idea, because metrics are usually managed by the
owners of the brokers, who sometimes are not the same who run the clients.

Also, I am not sure if this way it is possible for the client to force the
Broker to create many metrics and create some kind of damage.

Would it be better to add a Broker configuration flag to turn on this
feature ? I mean to allow the client to select the type of metrics ?


Enrico


Il giorno gio 2 dic 2021 alle ore 03:00 Yuri Mizushima <
yumiz...@yahoo-corp.jp> ha scritto:

> Do you have any comments?
> If there are no comments by Dec. 7, I will close the discussion and rebase
> the PR commit to current master.
>
> Regards,
>
> --
> Yuri Mizushima
> yumiz...@yahoo-corp.jp
>
>
> On 2021/11/16 15:46, "Yuri Mizushima"  wrote:
>
> Dear Pulsar community,
>
> I have created a new PR
> https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Fpull%2F12401&data=04%7C01%7Cyumizush%40yahoo-corp.jp%7Cd1cf3f01dc4b48994a0008d9a8cc7154%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637726419802266126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Sn57dCx1iItNEMArKWAXxE9frCbmQClw%2Fts0nmMkwyw%3D&reserved=0
> for stats aggregation,
> but I didn't discuss about the wire protocol change. I hope we will
> discuss it here.
>
> Currently, partitioned producer can't aggregate by any key such as
> cnx, producerId, producerName, and so on.
> I think we need to add any aggregation system.
> Therefore, added new aggregation policy as producerName (with client
> side implementation).
>
> New protocol field partial_producer_supported is not used for stats
> aggregation. It is used for backward compatibility.
>
> https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Fpull%2F12401%2Ffiles%23diff-f29399fed32e0916cf28452ba71078a3ae5ed77bbaef9f52a925165d8ee66cfdR489&data=04%7C01%7Cyumizush%40yahoo-corp.jp%7Cd1cf3f01dc4b48994a0008d9a8cc7154%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637726419802266126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=QswqgZHSeHJArmgAdBx1wPrm6rX4l4lTf39hs57g6NI%3D&reserved=0
>
> In my understanding, if introduce new stats aggregation key to client
> side,
> need a way to determine whether the feature is enabled at client side.
> For example, whether the producer has specific field or metadata,
> the version (e.g. protocol version) is greater than threshold, etc.
>
> Of course, if we can introduce aggregation feature without adding any
> new key or implementations from client side,
> we can support the feature not only new client but also old one.
>
> I'm looking forward to your opinions or suggestions to this PR.
>
> Regards,
> --
> Yuri Mizushima
> yumiz...@yahoo-corp.jp
>
>
> On 2021/05/11 14:26, "Yuri Mizushima"  wrote:
>
>
> Dear Pulsar Community,
>
> > I will submit the next PR about PartitionedTopicStats later.
> I submitted the next PR for this PIP. If you have any suggestions,
> please comment to this PR.
>
> https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Fpull%2F10534&data=04%7C01%7Cyumizush%40yahoo-corp.jp%7Cd1cf3f01dc4b48994a0008d9a8cc7154%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637726419802266126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=cHqzaY1H4N%2FaQEemy5ZRQSBAiXa3FDXDuFLyoJ4dRtA%3D&reserved=0
>
> Regards,
>
> --
> Yuri Mizushima
> yumiz...@yahoo-corp.jp
>
>
> "Yuri Mizushima"  wrote:
>
> Dear Pulsar Community,
>
> I submitted the PR for this PIP.
>
> https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Fpull%2F10279&data=04%7C01%7Cyumizush%40yahoo-corp.jp%7Cd1cf3f01dc4b48994a0008d9a8cc7154%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637726419802266126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=AzmPY98arKmn7UM1%2F6FJMwOKOavPVC5xVG%2Br5FBJlH4%3D&reserved=0
>
> This is a part of implementations.
> I will submit the next PR about PartitionedTopicStats later.
>
> Regards,
> --
> Yuri Mizushima
> yumiz...@yahoo-corp.jp
>
>
> "Yuri Mizushima"  wrote:
>
> Sijie,
>
> After sending previous mail, I watched meeting recording and
> understand about authn/authz issue.
> Therefore, I updated the PIP document.
>
> https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Fwiki%2FPIP-79%253A-Reduce-redundant-producers-from-partitioned-producer&data=04%7C01%7Cyumizush%40yahoo-corp.jp%7Cd1cf3f01dc4b48994a0

[GitHub] [pulsar-client-node] massakam commented on pull request #182: fix: use pause and resume api to ensure orderly init listener->consumer

[GitBox]

massakam commented on pull request #182:
URL: 
https://github.com/apache/pulsar-client-node/pull/182#issuecomment-984501591


   @lipeining Thank you for your contribution. The change looks good to me.
   
   But why is the destination branch `branch-1.3` instead of `master`? Do you 
want v1.3.x with this patch to be released? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-helm-chart] wangshu3000 opened a new pull request #182: Fixes #177 Fix indentation of component, as it should be under the la…

[GitBox]

wangshu3000 opened a new pull request #182:
URL: https://github.com/apache/pulsar-helm-chart/pull/182


   Fixes #177
   
   ### Motivation
   
   When deploying the pulsar helm chart, if i open the dashboard indicator, 
it'll be failed, and show message say the component is unknown field. Reason is 
the indentation of component line, it should be the child level of label tag.
   
   ### Modifications
   
   Changed indentation of the component line.
   
   ### Verifying this change
   
   - [ ] Make sure that the change passes the CI checks.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Creating Good Release notes

[Enrico Olivelli]

Hello community,

There is an open discussion on the Pulsar 2.9.0 release notes PR:
https://github.com/apache/pulsar/pull/12425

I have created the block of release notes by downloading the list of PR
using some GitHub API.
Then I have manually classified:
- News and Noteworthy: cool things in the Release
- Breaking Changes: things you MUST know when you upgrade
- Java Client, C++ Client, Python Client, Functions/Pulsar IO

The goal is to provide useful information for people who want to upgrade
Pulsar.

My problems are:
- PR titles are often badly written, but I don't want to fix all of them
(typos,  tenses of verbs, formatting)
- There are more than 300 PRs, I don't want to classify them manually, I
just highlighted the most important from my point of view

If for 2.9.0 we still keep a list of PR, then I believe that the current
status of the patch is good.

If we want to do it another way, then I am now asking if there is someone
who can volunteer in fixing and classifying the list of 300 PRs, it is a
huge task.

There is already much more work to do to get 2.9.0 completely released (and
also PulsarAdapters) and we have to cut 2.9.1 as soon as possible due to a
bad regression found in 2.9.0.

Thanks
Enrico

[GitHub] [pulsar-helm-chart] wangshu3000 opened a new pull request #183: Update ingress api version, extension/v1beta1 will not be supported i…

[GitBox]

wangshu3000 opened a new pull request #183:
URL: https://github.com/apache/pulsar-helm-chart/pull/183


   …n new k8s version, this change keep backward compatibility for lower 
kubernetes version 
   
   Fixes #157
   
   ### Motivation
   
   The ingress api extensions/v1beta1 version will not be supported in k8s 
version 1.22+, need to update to new networking.k8s.io/v1.
   This PR also keep support for old k8s version, to provide enough 
compatibility.
   
   ### Modifications
   
   Updated the api version for below 4 components:
   1. dashboard-ingress
   2. grafana-ingress
   3. proxy-ingress
   4. pulsar-manager-ingress
   
   ### Verifying this change
   
   - [ ] Make sure that the change passes the CI checks.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: Creating Good Release notes

[Jonathan Ellis]

I don't think a raw dump of PRs is right for release notes even if the
commit messages are perfectly written by God.  It's too low level for
almost everyone, and the people who really do want commit-level detail can
just use git, which is a better tool for exploring history than a static
text file.

On Thu, Dec 2, 2021 at 5:12 AM Enrico Olivelli  wrote:

> Hello community,
>
> There is an open discussion on the Pulsar 2.9.0 release notes PR:
> https://github.com/apache/pulsar/pull/12425
>
> I have created the block of release notes by downloading the list of PR
> using some GitHub API.
> Then I have manually classified:
> - News and Noteworthy: cool things in the Release
> - Breaking Changes: things you MUST know when you upgrade
> - Java Client, C++ Client, Python Client, Functions/Pulsar IO
>
> The goal is to provide useful information for people who want to upgrade
> Pulsar.
>
> My problems are:
> - PR titles are often badly written, but I don't want to fix all of them
> (typos,  tenses of verbs, formatting)
> - There are more than 300 PRs, I don't want to classify them manually, I
> just highlighted the most important from my point of view
>
> If for 2.9.0 we still keep a list of PR, then I believe that the current
> status of the patch is good.
>
> If we want to do it another way, then I am now asking if there is someone
> who can volunteer in fixing and classifying the list of 300 PRs, it is a
> huge task.
>
> There is already much more work to do to get 2.9.0 completely released (and
> also PulsarAdapters) and we have to cut 2.9.1 as soon as possible due to a
> bad regression found in 2.9.0.
>
> Thanks
> Enrico
>


-- 
Jonathan Ellis
co-founder, http://www.datastax.com
@spyced

Re: Creating Good Release notes

[Dave Fisher]

> On Dec 2, 2021, at 3:11 AM, Enrico Olivelli  wrote:
> 
> Hello community,
> 
> There is an open discussion on the Pulsar 2.9.0 release notes PR:
> https://github.com/apache/pulsar/pull/12425

In reading through the comments there I’m noticing these main themes.

(1) Wanting to standardize / rewrite PR titles. I don’t think that Developers 
from around the globe are going to do that. If this needs to be done then a 
volunteer who finds it important to them should review and update the PRs as 
they are being created.

(2) The other issue on consistently marking PR ids and PIPs should be possible 
with tooling.

I would suggest that the PMC needs to provide more support to whoever the RM 
for a release is. RMs are perhaps the most valuable committers we have and 
let’s try not to overburden them with extra tasks.

- POI uses automation and accepts whatever title are on the commits.
- For OpenOffice the RM doesn’t create the Release Notes, other members do into 
English and then various community members translate them. 

So, let’s not slow the process, but let’s go for eventual consistency.

Regards,
Dave

> 
> I have created the block of release notes by downloading the list of PR
> using some GitHub API.
> Then I have manually classified:
> - News and Noteworthy: cool things in the Release
> - Breaking Changes: things you MUST know when you upgrade
> - Java Client, C++ Client, Python Client, Functions/Pulsar IO
> 
> The goal is to provide useful information for people who want to upgrade
> Pulsar.
> 
> My problems are:
> - PR titles are often badly written, but I don't want to fix all of them
> (typos,  tenses of verbs, formatting)
> - There are more than 300 PRs, I don't want to classify them manually, I
> just highlighted the most important from my point of view
> 
> If for 2.9.0 we still keep a list of PR, then I believe that the current
> status of the patch is good.
> 
> If we want to do it another way, then I am now asking if there is someone
> who can volunteer in fixing and classifying the list of 300 PRs, it is a
> huge task.
> 
> There is already much more work to do to get 2.9.0 completely released (and
> also PulsarAdapters) and we have to cut 2.9.1 as soon as possible due to a
> bad regression found in 2.9.0.
> 
> Thanks
> Enrico

Re: [DISCUSS] Pulsar Protocol For Client Timeouts and Creating Producers

[Dave Fisher]

> On Dec 1, 2021, at 10:21 AM, Michael Marshall  wrote:
> 
> Following up here, I am still in need of reviews for PR [0]. It
> introduces an important clarification to the Pulsar Protocol Spec.
> Please take a look, if you are able.
> 
> [0] - https://github.com/apache/pulsar/pull/12948

I added some suggested reviewers to the PR.

Regards,
Dave

> 
> Thanks!
> Michael
> 
> On Tue, Nov 23, 2021 at 1:10 PM Michael Marshall  
> wrote:
>> 
>> I created a PR to update the protocol's documentation:
>> https://github.com/apache/pulsar/pull/12948. Please take a look, if
>> you're able.
>> 
>> Once this PR is accepted/merged, I will follow up with an update to
>> the Java client.
>> 
>> - Michael
>> 
>> On Thu, Nov 18, 2021 at 1:29 PM Michael Marshall  
>> wrote:
>>> 
>>> I view this as an edge case of the Pulsar Protocol that requires
>>> clarification. Once we clarify the spec, we can update the clients to
>>> conform to the spec. I don't think we need to give end users control
>>> over this small part of the protocol.
>>> 
>>> After reviewing the design a bit more, I think we should update the
>>> Java client to send the `CloseProducer` command, as well as update the
>>> spec to recommend this implementation. While the `ServerCnx` class
>>> "works" both ways, the current Java client implementation leads to
>>> unnecessary calls, unnecessary errors, and a longer time to recovery.
>>> 
>>> Specifically, if the client fails to send a `CloseProducer` command,
>>> it ends up getting into a sequence of retries where each new
>>> `Producer` command receives an immediate `ErrorResponse` because the
>>> `ServerCnx` already has a pending producer. By sending a
>>> `CloseProducer` command, the client gives the broker permission to
>>> stop keeping track of the original create producer request. It also
>>> means that if the topic eventually loads, the broker will respond to
>>> the right request id with a `ProducerSuccessResponse` command.
>>> 
>>> I will follow up with an update to the client and the protocol spec,
>>> unless there are any objections.
>>> 
>>> Thanks,
>>> Michael
>>> 
>>> On Thu, Nov 18, 2021 at 12:25 PM Neng Lu  wrote:
 
 How about making the behavior when timeout configurable? And by default, 
 it will send the `CloseProducer` cmd?
 
 On 2021/11/17 05:52:21 Michael Marshall wrote:
> Hi All,
> 
> I noticed that the `ServerCnxTest#testCreateProducerTimeout` test
> indicates that a producer will send a `CloserProducer` command when
> producer creation times out for the client.
> 
> The Java client does not follow this protocol. When the producer
> creation times out, it just schedules another attempt to create the
> producer.
> 
> The C++ client does follow this protocol and is annotated with the
> following comment:
> 
>> // Creating the producer has timed out. We need to ensure the broker 
>> closes the producer
>> // in case it was indeed created, otherwise it might prevent new create 
>> producer operation,
>> // since we are not closing the connection
> 
> I don't see anything in our official protocol spec indicating the
> "right" behavior. Given the test coverage, it seems like the initial
> design was to expect a `CloseProducer` command. However, I also see that
> the broker's `ServerCnx` class will reply to a redundant `Producer`
> command with a `ProducerSuccess` command, as long as the producer
> is already created.
> 
> Should I submit a PR to update the Java client to send a
> `CloseProducer` command when a `Producer` command times out?
> 
> Thanks,
> Michael
> 

Re: Creating Good Release notes

[Michael Marshall]

> I would suggest that the PMC needs to provide more support to
> whoever the RM for a release is. RMs are perhaps the most
> valuable committers we have and let’s try not to overburden them
> with extra tasks.

+1 - this is especially important if we want to have more
frequent releases.

I see two concepts here: release notes and changelog. In
my opinion, release notes highlight the important new features
and bug fixes, while changelogs are a raw list of changes. I think both
provide value, but as Jonathan pointed out, the changelog information
is already available on GitHub.

I propose that we update the PR template so that our GitHub bot can
automatically categorize PRs using GitHub labels. Important PRs that
should be highlighted in release notes can get a special label. Then, a
standard script can generate the release notes (and possibly a
changelog) based on PR labels. The release manager would just run
the script and commit the output. If someone would like to improve the
release notes, they can submit a PR.

Also, I think it'd be nice to add a table naming and thanking the
contributors for each release. For example, Apache Arrow generates
this list using a git command [0].

[0] - https://arrow.apache.org/release/6.0.0.html

Thanks,
Michael


On Thu, Dec 2, 2021 at 9:26 AM Dave Fisher  wrote:
>
>
>
> > On Dec 2, 2021, at 3:11 AM, Enrico Olivelli  wrote:
> >
> > Hello community,
> >
> > There is an open discussion on the Pulsar 2.9.0 release notes PR:
> > https://github.com/apache/pulsar/pull/12425
>
> In reading through the comments there I’m noticing these main themes.
>
> (1) Wanting to standardize / rewrite PR titles. I don’t think that Developers 
> from around the globe are going to do that. If this needs to be done then a 
> volunteer who finds it important to them should review and update the PRs as 
> they are being created.
>
> (2) The other issue on consistently marking PR ids and PIPs should be 
> possible with tooling.
>
> I would suggest that the PMC needs to provide more support to whoever the RM 
> for a release is. RMs are perhaps the most valuable committers we have and 
> let’s try not to overburden them with extra tasks.
>
> - POI uses automation and accepts whatever title are on the commits.
> - For OpenOffice the RM doesn’t create the Release Notes, other members do 
> into English and then various community members translate them.
>
> So, let’s not slow the process, but let’s go for eventual consistency.
>
> Regards,
> Dave
>
> >
> > I have created the block of release notes by downloading the list of PR
> > using some GitHub API.
> > Then I have manually classified:
> > - News and Noteworthy: cool things in the Release
> > - Breaking Changes: things you MUST know when you upgrade
> > - Java Client, C++ Client, Python Client, Functions/Pulsar IO
> >
> > The goal is to provide useful information for people who want to upgrade
> > Pulsar.
> >
> > My problems are:
> > - PR titles are often badly written, but I don't want to fix all of them
> > (typos,  tenses of verbs, formatting)
> > - There are more than 300 PRs, I don't want to classify them manually, I
> > just highlighted the most important from my point of view
> >
> > If for 2.9.0 we still keep a list of PR, then I believe that the current
> > status of the patch is good.
> >
> > If we want to do it another way, then I am now asking if there is someone
> > who can volunteer in fixing and classifying the list of 300 PRs, it is a
> > huge task.
> >
> > There is already much more work to do to get 2.9.0 completely released (and
> > also PulsarAdapters) and we have to cut 2.9.1 as soon as possible due to a
> > bad regression found in 2.9.0.
> >
> > Thanks
> > Enrico
>

Re: Creating Good Release notes

[Matteo Merli]

There is already a label `release/note-required` that is being used to
call out PR/issues when compiling the release notes.



--
Matteo Merli


On Thu, Dec 2, 2021 at 10:07 AM Michael Marshall  wrote:
>
> > I would suggest that the PMC needs to provide more support to
> > whoever the RM for a release is. RMs are perhaps the most
> > valuable committers we have and let’s try not to overburden them
> > with extra tasks.
>
> +1 - this is especially important if we want to have more
> frequent releases.
>
> I see two concepts here: release notes and changelog. In
> my opinion, release notes highlight the important new features
> and bug fixes, while changelogs are a raw list of changes. I think both
> provide value, but as Jonathan pointed out, the changelog information
> is already available on GitHub.
>
> I propose that we update the PR template so that our GitHub bot can
> automatically categorize PRs using GitHub labels. Important PRs that
> should be highlighted in release notes can get a special label. Then, a
> standard script can generate the release notes (and possibly a
> changelog) based on PR labels. The release manager would just run
> the script and commit the output. If someone would like to improve the
> release notes, they can submit a PR.
>
> Also, I think it'd be nice to add a table naming and thanking the
> contributors for each release. For example, Apache Arrow generates
> this list using a git command [0].
>
> [0] - https://arrow.apache.org/release/6.0.0.html
>
> Thanks,
> Michael
>
>
> On Thu, Dec 2, 2021 at 9:26 AM Dave Fisher  wrote:
> >
> >
> >
> > > On Dec 2, 2021, at 3:11 AM, Enrico Olivelli  wrote:
> > >
> > > Hello community,
> > >
> > > There is an open discussion on the Pulsar 2.9.0 release notes PR:
> > > https://github.com/apache/pulsar/pull/12425
> >
> > In reading through the comments there I’m noticing these main themes.
> >
> > (1) Wanting to standardize / rewrite PR titles. I don’t think that 
> > Developers from around the globe are going to do that. If this needs to be 
> > done then a volunteer who finds it important to them should review and 
> > update the PRs as they are being created.
> >
> > (2) The other issue on consistently marking PR ids and PIPs should be 
> > possible with tooling.
> >
> > I would suggest that the PMC needs to provide more support to whoever the 
> > RM for a release is. RMs are perhaps the most valuable committers we have 
> > and let’s try not to overburden them with extra tasks.
> >
> > - POI uses automation and accepts whatever title are on the commits.
> > - For OpenOffice the RM doesn’t create the Release Notes, other members do 
> > into English and then various community members translate them.
> >
> > So, let’s not slow the process, but let’s go for eventual consistency.
> >
> > Regards,
> > Dave
> >
> > >
> > > I have created the block of release notes by downloading the list of PR
> > > using some GitHub API.
> > > Then I have manually classified:
> > > - News and Noteworthy: cool things in the Release
> > > - Breaking Changes: things you MUST know when you upgrade
> > > - Java Client, C++ Client, Python Client, Functions/Pulsar IO
> > >
> > > The goal is to provide useful information for people who want to upgrade
> > > Pulsar.
> > >
> > > My problems are:
> > > - PR titles are often badly written, but I don't want to fix all of them
> > > (typos,  tenses of verbs, formatting)
> > > - There are more than 300 PRs, I don't want to classify them manually, I
> > > just highlighted the most important from my point of view
> > >
> > > If for 2.9.0 we still keep a list of PR, then I believe that the current
> > > status of the patch is good.
> > >
> > > If we want to do it another way, then I am now asking if there is someone
> > > who can volunteer in fixing and classifying the list of 300 PRs, it is a
> > > huge task.
> > >
> > > There is already much more work to do to get 2.9.0 completely released 
> > > (and
> > > also PulsarAdapters) and we have to cut 2.9.1 as soon as possible due to a
> > > bad regression found in 2.9.0.
> > >
> > > Thanks
> > > Enrico
> >

Re: Creating Good Release notes

[Michael Marshall]

> There is already a label `release/note-required` that is being used to
> call out PR/issues when compiling the release notes.

+1 I think we should use this label more--it has only been used 12 times [0].

[0] - 
https://github.com/apache/pulsar/pulls?q=is%3Apr+label%3Arelease%2Fnote-required+

- Michael

On Thu, Dec 2, 2021 at 12:10 PM Matteo Merli  wrote:
>
> There is already a label `release/note-required` that is being used to
> call out PR/issues when compiling the release notes.
>
>
>
> --
> Matteo Merli
> 
>
> On Thu, Dec 2, 2021 at 10:07 AM Michael Marshall  
> wrote:
> >
> > > I would suggest that the PMC needs to provide more support to
> > > whoever the RM for a release is. RMs are perhaps the most
> > > valuable committers we have and let’s try not to overburden them
> > > with extra tasks.
> >
> > +1 - this is especially important if we want to have more
> > frequent releases.
> >
> > I see two concepts here: release notes and changelog. In
> > my opinion, release notes highlight the important new features
> > and bug fixes, while changelogs are a raw list of changes. I think both
> > provide value, but as Jonathan pointed out, the changelog information
> > is already available on GitHub.
> >
> > I propose that we update the PR template so that our GitHub bot can
> > automatically categorize PRs using GitHub labels. Important PRs that
> > should be highlighted in release notes can get a special label. Then, a
> > standard script can generate the release notes (and possibly a
> > changelog) based on PR labels. The release manager would just run
> > the script and commit the output. If someone would like to improve the
> > release notes, they can submit a PR.
> >
> > Also, I think it'd be nice to add a table naming and thanking the
> > contributors for each release. For example, Apache Arrow generates
> > this list using a git command [0].
> >
> > [0] - https://arrow.apache.org/release/6.0.0.html
> >
> > Thanks,
> > Michael
> >
> >
> > On Thu, Dec 2, 2021 at 9:26 AM Dave Fisher  wrote:
> > >
> > >
> > >
> > > > On Dec 2, 2021, at 3:11 AM, Enrico Olivelli  wrote:
> > > >
> > > > Hello community,
> > > >
> > > > There is an open discussion on the Pulsar 2.9.0 release notes PR:
> > > > https://github.com/apache/pulsar/pull/12425
> > >
> > > In reading through the comments there I’m noticing these main themes.
> > >
> > > (1) Wanting to standardize / rewrite PR titles. I don’t think that 
> > > Developers from around the globe are going to do that. If this needs to 
> > > be done then a volunteer who finds it important to them should review and 
> > > update the PRs as they are being created.
> > >
> > > (2) The other issue on consistently marking PR ids and PIPs should be 
> > > possible with tooling.
> > >
> > > I would suggest that the PMC needs to provide more support to whoever the 
> > > RM for a release is. RMs are perhaps the most valuable committers we have 
> > > and let’s try not to overburden them with extra tasks.
> > >
> > > - POI uses automation and accepts whatever title are on the commits.
> > > - For OpenOffice the RM doesn’t create the Release Notes, other members 
> > > do into English and then various community members translate them.
> > >
> > > So, let’s not slow the process, but let’s go for eventual consistency.
> > >
> > > Regards,
> > > Dave
> > >
> > > >
> > > > I have created the block of release notes by downloading the list of PR
> > > > using some GitHub API.
> > > > Then I have manually classified:
> > > > - News and Noteworthy: cool things in the Release
> > > > - Breaking Changes: things you MUST know when you upgrade
> > > > - Java Client, C++ Client, Python Client, Functions/Pulsar IO
> > > >
> > > > The goal is to provide useful information for people who want to upgrade
> > > > Pulsar.
> > > >
> > > > My problems are:
> > > > - PR titles are often badly written, but I don't want to fix all of them
> > > > (typos,  tenses of verbs, formatting)
> > > > - There are more than 300 PRs, I don't want to classify them manually, I
> > > > just highlighted the most important from my point of view
> > > >
> > > > If for 2.9.0 we still keep a list of PR, then I believe that the current
> > > > status of the patch is good.
> > > >
> > > > If we want to do it another way, then I am now asking if there is 
> > > > someone
> > > > who can volunteer in fixing and classifying the list of 300 PRs, it is a
> > > > huge task.
> > > >
> > > > There is already much more work to do to get 2.9.0 completely released 
> > > > (and
> > > > also PulsarAdapters) and we have to cut 2.9.1 as soon as possible due 
> > > > to a
> > > > bad regression found in 2.9.0.
> > > >
> > > > Thanks
> > > > Enrico
> > >

[Website] CI - Pulsar Website Build takes a long time

[Dave Fisher]

Hi -

I noticed some things about the website build. 
https://github.com/apache/pulsar/actions/workflows/ci-pulsar-website-build.yaml

(1) It runs for nearly 2 hours.
(2) If it exceeds 2 hours then it is canceled.
(3) It runs every 6 hours.

We share runners with all of the ASF.

I think we need to change from 4 runs per day to either 2 or 3 runs.

We can also consider switching to the ASF’s Jenkins at 
https://builds.apache.org/job/Pulsar/ - it might give better performance.

Regards,
Dave

Re: [Website] CI - Pulsar Website Build takes a long time

[Li Li]

+1 Good idea, we will take a look and do some optimizations

Thanks,
LiLi

> On Dec 3, 2021, at 5:04 AM, Dave Fisher  wrote:
> 
> Hi -
> 
> I noticed some things about the website build. 
> https://github.com/apache/pulsar/actions/workflows/ci-pulsar-website-build.yaml
> 
> (1) It runs for nearly 2 hours.
> (2) If it exceeds 2 hours then it is canceled.
> (3) It runs every 6 hours.
> 
> We share runners with all of the ASF.
> 
> I think we need to change from 4 runs per day to either 2 or 3 runs.
> 
> We can also consider switching to the ASF’s Jenkins at 
> https://builds.apache.org/job/Pulsar/ - it might give better performance.
> 
> Regards,
> Dave

Re: [Discuss] Apache pulsar-manager 0.3.0 release

[Li Li]

+1

Thanks,
LiLi

> On Dec 1, 2021, at 2:26 PM, Hang Chen  wrote:
> 
> +1
> 
> Thanks,
> Hang
> 
> Lan Liang  于2021年12月1日周三 下午2:14写道：
>> 
>> +1, Thanks for your work.
>> 
>> 
>> 
>> 
>> 
>> 
>> Best Regards,
>> Lan Liang
>> On 11/29/2021 17:17，Enrico Olivelli wrote：
>> +1
>> 
>> 
>> 
>> 
>> 
>> Enrico
>> 
>> 
>> Il giorno lun 29 nov 2021 alle ore 07:50 陳智弘  ha 
>> scritto:
>> 
>> +1
>> Glad to hear that
>> 
>> 
>> Thomas
>> 
>> 
>> Guangning E  於 2021年11月25日 週四 13:59 寫道：
>> 
>> Hello everyone:
>> 
>> It has been a long time since the release of the previous version's major
>> release (0.2.0). I am planning to start the 0.3.0 release soon to fix some
>> issues with the pulsar manager .
>> 
>> 
>> 
>> Thanks,
>> Guangning

Re: Creating Good Release notes

[Yunze Xu]

First I agree with Jonathan that we should perform some changes with
the original PR descriptions.

Then, classifying these PRs is also necessary, otherwise the release notes
would be meaningless. There are a lot of PRs that should be classfied in
Misc part of https://github.com/apache/pulsar/pull/12425 
 and I also gave
some comments in the PR.

IMO, it’s okay to ignore the PRs that only fix some typos or fix some flaky 
tests.
But I found many PRs in Misc part should also be noted.

We should not sacrifice the release quality for a new release like 2.9.1.

> 2021年12月2日 下午7:11，Enrico Olivelli  写道：
> 
> Hello community,
> 
> There is an open discussion on the Pulsar 2.9.0 release notes PR:
> https://github.com/apache/pulsar/pull/12425
> 
> I have created the block of release notes by downloading the list of PR
> using some GitHub API.
> Then I have manually classified:
> - News and Noteworthy: cool things in the Release
> - Breaking Changes: things you MUST know when you upgrade
> - Java Client, C++ Client, Python Client, Functions/Pulsar IO
> 
> The goal is to provide useful information for people who want to upgrade
> Pulsar.
> 
> My problems are:
> - PR titles are often badly written, but I don't want to fix all of them
> (typos,  tenses of verbs, formatting)
> - There are more than 300 PRs, I don't want to classify them manually, I
> just highlighted the most important from my point of view
> 
> If for 2.9.0 we still keep a list of PR, then I believe that the current
> status of the patch is good.
> 
> If we want to do it another way, then I am now asking if there is someone
> who can volunteer in fixing and classifying the list of 300 PRs, it is a
> huge task.
> 
> There is already much more work to do to get 2.9.0 completely released (and
> also PulsarAdapters) and we have to cut 2.9.1 as soon as possible due to a
> bad regression found in 2.9.0.
> 
> Thanks
> Enrico

New committer: Michael Marshal

[Enrico Olivelli]

The Project Management Committee (PMC) for Apache  Pulsar
has invited Michael Marshal to become a committer and we are pleased to
announce that he has accepted.

Michael contributed a lot of interesting additions to the project and he is
very active in the community.

Being a committer enables easier contribution to the
project since there is no need to go via the patch
submission process. This should enable better productivity.

Congrats Micheal !


Enrico Olivelli

Re: [Website] CI - Pulsar Website Build takes a long time

[Enrico Olivelli]

Dave,

Il giorno ven 3 dic 2021 alle ore 04:16 Li Li 
ha scritto:

> +1 Good idea, we will take a look and do some optimizations
>
> Thanks,
> LiLi
>
> > On Dec 3, 2021, at 5:04 AM, Dave Fisher  wrote:
> >
> > Hi -
> >
> > I noticed some things about the website build.
> https://github.com/apache/pulsar/actions/workflows/ci-pulsar-website-build.yaml
> >
> > (1) It runs for nearly 2 hours.
>

why ?
is it so hard to build the website ?


> > (2) If it exceeds 2 hours then it is canceled.
> > (3) It runs every 6 hours.
> >
> > We share runners with all of the ASF.
> >
> > I think we need to change from 4 runs per day to either 2 or 3 runs.
>

+1


> >
> > We can also consider switching to the ASF’s Jenkins at
> https://builds.apache.org/job/Pulsar/ - it might give better performance.
>

We were on ASF Jenkins, but we used to use too many resources and then we
moved to GH.
I don't know the details but there are still many discussions on builds@
about the fact the Pulsar is consuming many ASF resources for CI.

We can move the WebSite stuff to Jenkins

IIUC recently the ASF started to assign dedicated Jenkins instances to
projects, we could try to go that way, and probably some company
may "donate" some resources to host the workers.

Enrico



> >
> > Regards,
> > Dave
>
>