Virtual Pulsar Community Meetings

[Enrico Olivelli]

Hello everyone,
in the BookKeeper community we used to have "Community Meetings" in order
to meet each other, discuss current works on the project, share knowledge
about current problems.

What about having Community meetings for Pulsar ?
I would be happy to organize and to host the first meeting.

In Apache we keep the decisions and the discussions on mailing lists, so
these meetings would be only to share information and we are not going to
make decisions.

The Pulsar community is distributed all over the world, we have people from
China, the US, Europe, it will be hard to find a good time for everyone.
I suggest for the first meeting to meet at 8:30 PST

For reference, this is the link to the "minutes" of the Community Meetings
in BK,
Sijie and Matteo will remember those days
https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings


Enrico

Re: Virtual Pulsar Community Meetings

[Lari Hotari]

+1 for starting regular Virtual Pulsar Community Meetings.

On Fri, Jan 29, 2021 at 10:05 AM Enrico Olivelli 
wrote:

> Hello everyone,
> in the BookKeeper community we used to have "Community Meetings" in order
> to meet each other, discuss current works on the project, share knowledge
> about current problems.
>
> What about having Community meetings for Pulsar ?
> I would be happy to organize and to host the first meeting.
>
> In Apache we keep the decisions and the discussions on mailing lists, so
> these meetings would be only to share information and we are not going to
> make decisions.
>
> The Pulsar community is distributed all over the world, we have people from
> China, the US, Europe, it will be hard to find a good time for everyone.
> I suggest for the first meeting to meet at 8:30 PST
>
> For reference, this is the link to the "minutes" of the Community Meetings
> in BK,
> Sijie and Matteo will remember those days
> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
>
>
> Enrico
>

Re: Virtual Pulsar Community Meetings

[Michael Marshall]

As a new contributor looking to get more involved, I’m very interested in 
regular community meetings.

Thanks,
Michael Marshall

> On Jan 29, 2021, at 1:13 AM, Lari Hotari  wrote:
> 
> +1 for starting regular Virtual Pulsar Community Meetings.
> 
>> On Fri, Jan 29, 2021 at 10:05 AM Enrico Olivelli 
>> wrote:
>> 
>> Hello everyone,
>> in the BookKeeper community we used to have "Community Meetings" in order
>> to meet each other, discuss current works on the project, share knowledge
>> about current problems.
>> 
>> What about having Community meetings for Pulsar ?
>> I would be happy to organize and to host the first meeting.
>> 
>> In Apache we keep the decisions and the discussions on mailing lists, so
>> these meetings would be only to share information and we are not going to
>> make decisions.
>> 
>> The Pulsar community is distributed all over the world, we have people from
>> China, the US, Europe, it will be hard to find a good time for everyone.
>> I suggest for the first meeting to meet at 8:30 PST
>> 
>> For reference, this is the link to the "minutes" of the Community Meetings
>> in BK,
>> Sijie and Matteo will remember those days
>> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
>> 
>> 
>> Enrico
>> 

Re: Virtual Pulsar Community Meetings

[Jia Zhai]

+1, we are doing some kind of similar thing in China for Pulsar Community
online & offline. Hope we could do this globally.

Best Regards.


Jia Zhai

Beijing, China

Mobile: +86 15810491983




On Fri, Jan 29, 2021 at 4:13 PM Lari Hotari  wrote:

> +1 for starting regular Virtual Pulsar Community Meetings.
>
> On Fri, Jan 29, 2021 at 10:05 AM Enrico Olivelli 
> wrote:
>
> > Hello everyone,
> > in the BookKeeper community we used to have "Community Meetings" in order
> > to meet each other, discuss current works on the project, share knowledge
> > about current problems.
> >
> > What about having Community meetings for Pulsar ?
> > I would be happy to organize and to host the first meeting.
> >
> > In Apache we keep the decisions and the discussions on mailing lists, so
> > these meetings would be only to share information and we are not going to
> > make decisions.
> >
> > The Pulsar community is distributed all over the world, we have people
> from
> > China, the US, Europe, it will be hard to find a good time for everyone.
> > I suggest for the first meeting to meet at 8:30 PST
> >
> > For reference, this is the link to the "minutes" of the Community
> Meetings
> > in BK,
> > Sijie and Matteo will remember those days
> >
> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
> >
> >
> > Enrico
> >
>

Re: Virtual Pulsar Community Meetings

[Jia Zhai]

One concern is related to the time slot,  8:30 PST seems to be a little
late for China(0:30 AM) and Japan, also a little early for the US.
Maybe 4:00-5:00 PM PST?

Best Regards.


Jia Zhai

Beijing, China

Mobile: +86 15810491983




On Fri, Jan 29, 2021 at 4:22 PM Michael Marshall 
wrote:

> As a new contributor looking to get more involved, I’m very interested in
> regular community meetings.
>
> Thanks,
> Michael Marshall
>
> > On Jan 29, 2021, at 1:13 AM, Lari Hotari  wrote:
> >
> > +1 for starting regular Virtual Pulsar Community Meetings.
> >
> >> On Fri, Jan 29, 2021 at 10:05 AM Enrico Olivelli 
> >> wrote:
> >>
> >> Hello everyone,
> >> in the BookKeeper community we used to have "Community Meetings" in
> order
> >> to meet each other, discuss current works on the project, share
> knowledge
> >> about current problems.
> >>
> >> What about having Community meetings for Pulsar ?
> >> I would be happy to organize and to host the first meeting.
> >>
> >> In Apache we keep the decisions and the discussions on mailing lists, so
> >> these meetings would be only to share information and we are not going
> to
> >> make decisions.
> >>
> >> The Pulsar community is distributed all over the world, we have people
> from
> >> China, the US, Europe, it will be hard to find a good time for everyone.
> >> I suggest for the first meeting to meet at 8:30 PST
> >>
> >> For reference, this is the link to the "minutes" of the Community
> Meetings
> >> in BK,
> >> Sijie and Matteo will remember those days
> >>
> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
> >>
> >>
> >> Enrico
> >>
>

Re: Virtual Pulsar Community Meetings

[Renkai Ge]

+1 Hope it can be a little earlier, too.

> 2021年1月29日 下午4:41，Jia Zhai  写道：
> 
> One concern is related to the time slot,  8:30 PST seems to be a little
> late for China(0:30 AM) and Japan, also a little early for the US.
> Maybe 4:00-5:00 PM PST?
> 
> Best Regards.
> 
> 
> Jia Zhai
> 
> Beijing, China
> 
> Mobile: +86 15810491983
> 
> 
> 
> 
> On Fri, Jan 29, 2021 at 4:22 PM Michael Marshall 
> wrote:
> 
>> As a new contributor looking to get more involved, I’m very interested in
>> regular community meetings.
>> 
>> Thanks,
>> Michael Marshall
>> 
>>> On Jan 29, 2021, at 1:13 AM, Lari Hotari  wrote:
>>> 
>>> +1 for starting regular Virtual Pulsar Community Meetings.
>>> 
 On Fri, Jan 29, 2021 at 10:05 AM Enrico Olivelli 
 wrote:
 
 Hello everyone,
 in the BookKeeper community we used to have "Community Meetings" in
>> order
 to meet each other, discuss current works on the project, share
>> knowledge
 about current problems.
 
 What about having Community meetings for Pulsar ?
 I would be happy to organize and to host the first meeting.
 
 In Apache we keep the decisions and the discussions on mailing lists, so
 these meetings would be only to share information and we are not going
>> to
 make decisions.
 
 The Pulsar community is distributed all over the world, we have people
>> from
 China, the US, Europe, it will be hard to find a good time for everyone.
 I suggest for the first meeting to meet at 8:30 PST
 
 For reference, this is the link to the "minutes" of the Community
>> Meetings
 in BK,
 Sijie and Matteo will remember those days
 
>> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
 
 
 Enrico
 
>> 

Re: Virtual Pulsar Community Meetings

[Aparnesh Gaurav]

+1 ! Great initiative .

Best regards,
Aparnesh

On Fri, Jan 29, 2021 at 12:45 AM Renkai Ge  wrote:

> +1 Hope it can be a little earlier, too.
>
> > 2021年1月29日 下午4:41，Jia Zhai  写道：
> >
> > One concern is related to the time slot,  8:30 PST seems to be a little
> > late for China(0:30 AM) and Japan, also a little early for the US.
> > Maybe 4:00-5:00 PM PST?
> >
> > Best Regards.
> >
> >
> > Jia Zhai
> >
> > Beijing, China
> >
> > Mobile: +86 15810491983
> >
> >
> >
> >
> > On Fri, Jan 29, 2021 at 4:22 PM Michael Marshall 
> > wrote:
> >
> >> As a new contributor looking to get more involved, I’m very interested
> in
> >> regular community meetings.
> >>
> >> Thanks,
> >> Michael Marshall
> >>
> >>> On Jan 29, 2021, at 1:13 AM, Lari Hotari  wrote:
> >>>
> >>> +1 for starting regular Virtual Pulsar Community Meetings.
> >>>
>  On Fri, Jan 29, 2021 at 10:05 AM Enrico Olivelli  >
>  wrote:
> 
>  Hello everyone,
>  in the BookKeeper community we used to have "Community Meetings" in
> >> order
>  to meet each other, discuss current works on the project, share
> >> knowledge
>  about current problems.
> 
>  What about having Community meetings for Pulsar ?
>  I would be happy to organize and to host the first meeting.
> 
>  In Apache we keep the decisions and the discussions on mailing lists,
> so
>  these meetings would be only to share information and we are not going
> >> to
>  make decisions.
> 
>  The Pulsar community is distributed all over the world, we have people
> >> from
>  China, the US, Europe, it will be hard to find a good time for
> everyone.
>  I suggest for the first meeting to meet at 8:30 PST
> 
>  For reference, this is the link to the "minutes" of the Community
> >> Meetings
>  in BK,
>  Sijie and Matteo will remember those days
> 
> >>
> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
> 
> 
>  Enrico
> 
> >>
>
>

-- 
Regards,
Aparnesh Gaurav

Re: Virtual Pulsar Community Meetings

[Fangbin Sun]

+1 Hope it can be a little earlier, too.

Enrico Olivelli  于2021年1月29日周五 下午4:05写道：

> Hello everyone,
> in the BookKeeper community we used to have "Community Meetings" in order
> to meet each other, discuss current works on the project, share knowledge
> about current problems.
>
> What about having Community meetings for Pulsar ?
> I would be happy to organize and to host the first meeting.
>
> In Apache we keep the decisions and the discussions on mailing lists, so
> these meetings would be only to share information and we are not going to
> make decisions.
>
> The Pulsar community is distributed all over the world, we have people from
> China, the US, Europe, it will be hard to find a good time for everyone.
> I suggest for the first meeting to meet at 8:30 PST
>
> For reference, this is the link to the "minutes" of the Community Meetings
> in BK,
> Sijie and Matteo will remember those days
> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
>
>
> Enrico
>

Re: Virtual Pulsar Community Meetings

[Lin Lin]

I look forward to it and hope to coordinate the time so that developers from 
China can also participate.

Re: Virtual Pulsar Community Meetings

[Sijie Guo]

That's a super great idea! Thank you for bringing this up!

Given there are a lot of committers/contributors are from North America and
Asia, I think we should pick up a better time that would be suitable for
people from Asia (Japan and China).
In the bookkeeper community, we used to run two events. One is to cover NA
and Asia, and the other one is to cover NA and EU.

I would suggest running biweekly meetings.

- Tuesday 4 PM - 5 PM PST for NA and Asia
- Thursday 8 AM - 9 AM PST for NA and EU

All the events can be recorded and uploaded to Youtube. So people are able
to watch the recordings as well.

If people are good with this proposal, I am happy to set up and coordinate
the meetings given I have run many meetings in the BookKeeper community
before.

We can get started next Tuesday and formalize the process as we go. Please
vote for your idea.

Thanks,
Sijie


On Fri, Jan 29, 2021 at 12:05 AM Enrico Olivelli 
wrote:

> Hello everyone,
> in the BookKeeper community we used to have "Community Meetings" in order
> to meet each other, discuss current works on the project, share knowledge
> about current problems.
>
> What about having Community meetings for Pulsar ?
> I would be happy to organize and to host the first meeting.
>
> In Apache we keep the decisions and the discussions on mailing lists, so
> these meetings would be only to share information and we are not going to
> make decisions.
>
> The Pulsar community is distributed all over the world, we have people from
> China, the US, Europe, it will be hard to find a good time for everyone.
> I suggest for the first meeting to meet at 8:30 PST
>
> For reference, this is the link to the "minutes" of the Community Meetings
> in BK,
> Sijie and Matteo will remember those days
> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
>
>
> Enrico
>

Re: Virtual Pulsar Community Meetings

[Yunze Xu]

+1

> 2021年1月29日 下午4:53，Sijie Guo  写道：
> 
> That's a super great idea! Thank you for bringing this up!
> 
> Given there are a lot of committers/contributors are from North America and
> Asia, I think we should pick up a better time that would be suitable for
> people from Asia (Japan and China).
> In the bookkeeper community, we used to run two events. One is to cover NA
> and Asia, and the other one is to cover NA and EU.
> 
> I would suggest running biweekly meetings.
> 
> - Tuesday 4 PM - 5 PM PST for NA and Asia
> - Thursday 8 AM - 9 AM PST for NA and EU
> 
> All the events can be recorded and uploaded to Youtube. So people are able
> to watch the recordings as well.
> 
> If people are good with this proposal, I am happy to set up and coordinate
> the meetings given I have run many meetings in the BookKeeper community
> before.
> 
> We can get started next Tuesday and formalize the process as we go. Please
> vote for your idea.
> 
> Thanks,
> Sijie
> 
> 
> On Fri, Jan 29, 2021 at 12:05 AM Enrico Olivelli 
> wrote:
> 
>> Hello everyone,
>> in the BookKeeper community we used to have "Community Meetings" in order
>> to meet each other, discuss current works on the project, share knowledge
>> about current problems.
>> 
>> What about having Community meetings for Pulsar ?
>> I would be happy to organize and to host the first meeting.
>> 
>> In Apache we keep the decisions and the discussions on mailing lists, so
>> these meetings would be only to share information and we are not going to
>> make decisions.
>> 
>> The Pulsar community is distributed all over the world, we have people from
>> China, the US, Europe, it will be hard to find a good time for everyone.
>> I suggest for the first meeting to meet at 8:30 PST
>> 
>> For reference, this is the link to the "minutes" of the Community Meetings
>> in BK,
>> Sijie and Matteo will remember those days
>> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
>> 
>> 
>> Enrico
>> 

Re: Virtual Pulsar Community Meetings

[Jinfeng Huang]

+1
That's a very good idea.
Taking the time zone into consideration is great.


Best Regards,
Jennifer


On Fri, Jan 29, 2021 at 5:02 PM Yunze Xu 
wrote:

> +1
>
> > 2021年1月29日 下午4:53，Sijie Guo  写道：
> >
> > That's a super great idea! Thank you for bringing this up!
> >
> > Given there are a lot of committers/contributors are from North America
> and
> > Asia, I think we should pick up a better time that would be suitable for
> > people from Asia (Japan and China).
> > In the bookkeeper community, we used to run two events. One is to cover
> NA
> > and Asia, and the other one is to cover NA and EU.
> >
> > I would suggest running biweekly meetings.
> >
> > - Tuesday 4 PM - 5 PM PST for NA and Asia
> > - Thursday 8 AM - 9 AM PST for NA and EU
> >
> > All the events can be recorded and uploaded to Youtube. So people are
> able
> > to watch the recordings as well.
> >
> > If people are good with this proposal, I am happy to set up and
> coordinate
> > the meetings given I have run many meetings in the BookKeeper community
> > before.
> >
> > We can get started next Tuesday and formalize the process as we go.
> Please
> > vote for your idea.
> >
> > Thanks,
> > Sijie
> >
> >
> > On Fri, Jan 29, 2021 at 12:05 AM Enrico Olivelli 
> > wrote:
> >
> >> Hello everyone,
> >> in the BookKeeper community we used to have "Community Meetings" in
> order
> >> to meet each other, discuss current works on the project, share
> knowledge
> >> about current problems.
> >>
> >> What about having Community meetings for Pulsar ?
> >> I would be happy to organize and to host the first meeting.
> >>
> >> In Apache we keep the decisions and the discussions on mailing lists, so
> >> these meetings would be only to share information and we are not going
> to
> >> make decisions.
> >>
> >> The Pulsar community is distributed all over the world, we have people
> from
> >> China, the US, Europe, it will be hard to find a good time for everyone.
> >> I suggest for the first meeting to meet at 8:30 PST
> >>
> >> For reference, this is the link to the "minutes" of the Community
> Meetings
> >> in BK,
> >> Sijie and Matteo will remember those days
> >>
> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
> >>
> >>
> >> Enrico
> >>
>
>

Re: Virtual Pulsar Community Meetings

[丛搏]

+1

On 2021/01/29 08:53:40, Sijie Guo  wrote: 
> That's a super great idea! Thank you for bringing this up!
> 
> Given there are a lot of committers/contributors are from North America and
> Asia, I think we should pick up a better time that would be suitable for
> people from Asia (Japan and China).
> In the bookkeeper community, we used to run two events. One is to cover NA
> and Asia, and the other one is to cover NA and EU.
> 
> I would suggest running biweekly meetings.
> 
> - Tuesday 4 PM - 5 PM PST for NA and Asia
> - Thursday 8 AM - 9 AM PST for NA and EU
> 
> All the events can be recorded and uploaded to Youtube. So people are able
> to watch the recordings as well.
> 
> If people are good with this proposal, I am happy to set up and coordinate
> the meetings given I have run many meetings in the BookKeeper community
> before.
> 
> We can get started next Tuesday and formalize the process as we go. Please
> vote for your idea.
> 
> Thanks,
> Sijie
> 
> 
> On Fri, Jan 29, 2021 at 12:05 AM Enrico Olivelli 
> wrote:
> 
> > Hello everyone,
> > in the BookKeeper community we used to have "Community Meetings" in order
> > to meet each other, discuss current works on the project, share knowledge
> > about current problems.
> >
> > What about having Community meetings for Pulsar ?
> > I would be happy to organize and to host the first meeting.
> >
> > In Apache we keep the decisions and the discussions on mailing lists, so
> > these meetings would be only to share information and we are not going to
> > make decisions.
> >
> > The Pulsar community is distributed all over the world, we have people from
> > China, the US, Europe, it will be hard to find a good time for everyone.
> > I suggest for the first meeting to meet at 8:30 PST
> >
> > For reference, this is the link to the "minutes" of the Community Meetings
> > in BK,
> > Sijie and Matteo will remember those days
> > https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
> >
> >
> > Enrico
> >
> 

Re: Virtual Pulsar Community Meetings

[Yong Zhang]

Awesome.

Agree with sijie, because of the timezone, it's better to run the meeting
separately.

Regards,
Yong

On Fri, 29 Jan 2021 at 16:53, Sijie Guo  wrote:

> That's a super great idea! Thank you for bringing this up!
>
> Given there are a lot of committers/contributors are from North America and
> Asia, I think we should pick up a better time that would be suitable for
> people from Asia (Japan and China).
> In the bookkeeper community, we used to run two events. One is to cover NA
> and Asia, and the other one is to cover NA and EU.
>
> I would suggest running biweekly meetings.
>
> - Tuesday 4 PM - 5 PM PST for NA and Asia
> - Thursday 8 AM - 9 AM PST for NA and EU
>
> All the events can be recorded and uploaded to Youtube. So people are able
> to watch the recordings as well.
>
> If people are good with this proposal, I am happy to set up and coordinate
> the meetings given I have run many meetings in the BookKeeper community
> before.
>
> We can get started next Tuesday and formalize the process as we go. Please
> vote for your idea.
>
> Thanks,
> Sijie
>
>
> On Fri, Jan 29, 2021 at 12:05 AM Enrico Olivelli 
> wrote:
>
> > Hello everyone,
> > in the BookKeeper community we used to have "Community Meetings" in order
> > to meet each other, discuss current works on the project, share knowledge
> > about current problems.
> >
> > What about having Community meetings for Pulsar ?
> > I would be happy to organize and to host the first meeting.
> >
> > In Apache we keep the decisions and the discussions on mailing lists, so
> > these meetings would be only to share information and we are not going to
> > make decisions.
> >
> > The Pulsar community is distributed all over the world, we have people
> from
> > China, the US, Europe, it will be hard to find a good time for everyone.
> > I suggest for the first meeting to meet at 8:30 PST
> >
> > For reference, this is the link to the "minutes" of the Community
> Meetings
> > in BK,
> > Sijie and Matteo will remember those days
> >
> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
> >
> >
> > Enrico
> >
>

Re: Virtual Pulsar Community Meetings

[xiaolong ran]

+1
I think we need to talk about the time zone into consideration is great.

Best Regards,
Xiaolong Ran

Jinfeng Huang  于2021年1月29日周五 下午5:06写道：

> +1
> That's a very good idea.
> Taking the time zone into consideration is great.
>
>
> Best Regards,
> Jennifer
>
>
> On Fri, Jan 29, 2021 at 5:02 PM Yunze Xu 
> wrote:
>
> > +1
> >
> > > 2021年1月29日 下午4:53，Sijie Guo  写道：
> > >
> > > That's a super great idea! Thank you for bringing this up!
> > >
> > > Given there are a lot of committers/contributors are from North America
> > and
> > > Asia, I think we should pick up a better time that would be suitable
> for
> > > people from Asia (Japan and China).
> > > In the bookkeeper community, we used to run two events. One is to cover
> > NA
> > > and Asia, and the other one is to cover NA and EU.
> > >
> > > I would suggest running biweekly meetings.
> > >
> > > - Tuesday 4 PM - 5 PM PST for NA and Asia
> > > - Thursday 8 AM - 9 AM PST for NA and EU
> > >
> > > All the events can be recorded and uploaded to Youtube. So people are
> > able
> > > to watch the recordings as well.
> > >
> > > If people are good with this proposal, I am happy to set up and
> > coordinate
> > > the meetings given I have run many meetings in the BookKeeper community
> > > before.
> > >
> > > We can get started next Tuesday and formalize the process as we go.
> > Please
> > > vote for your idea.
> > >
> > > Thanks,
> > > Sijie
> > >
> > >
> > > On Fri, Jan 29, 2021 at 12:05 AM Enrico Olivelli 
> > > wrote:
> > >
> > >> Hello everyone,
> > >> in the BookKeeper community we used to have "Community Meetings" in
> > order
> > >> to meet each other, discuss current works on the project, share
> > knowledge
> > >> about current problems.
> > >>
> > >> What about having Community meetings for Pulsar ?
> > >> I would be happy to organize and to host the first meeting.
> > >>
> > >> In Apache we keep the decisions and the discussions on mailing lists,
> so
> > >> these meetings would be only to share information and we are not going
> > to
> > >> make decisions.
> > >>
> > >> The Pulsar community is distributed all over the world, we have people
> > from
> > >> China, the US, Europe, it will be hard to find a good time for
> everyone.
> > >> I suggest for the first meeting to meet at 8:30 PST
> > >>
> > >> For reference, this is the link to the "minutes" of the Community
> > Meetings
> > >> in BK,
> > >> Sijie and Matteo will remember those days
> > >>
> >
> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
> > >>
> > >>
> > >> Enrico
> > >>
> >
> >
>

Re: Virtual Pulsar Community Meetings

[PengHui Li]

> I suggest for the first meeting to meet at 8:30 PST

@Enrico Seems the time would be not suitable for people from Asia.

> I would suggest running biweekly meetings.

- Tuesday 4 PM - 5 PM PST for NA and Asia
- Thursday 8 AM - 9 AM PST for NA and EU

the time is ok for me, +1

Thanks
- Penghui

Jinfeng Huang  于2021年1月29日周五 下午5:06写道：

> +1
> That's a very good idea.
> Taking the time zone into consideration is great.
>
>
> Best Regards,
> Jennifer
>
>
> On Fri, Jan 29, 2021 at 5:02 PM Yunze Xu 
> wrote:
>
> > +1
> >
> > > 2021年1月29日 下午4:53，Sijie Guo  写道：
> > >
> > > That's a super great idea! Thank you for bringing this up!
> > >
> > > Given there are a lot of committers/contributors are from North America
> > and
> > > Asia, I think we should pick up a better time that would be suitable
> for
> > > people from Asia (Japan and China).
> > > In the bookkeeper community, we used to run two events. One is to cover
> > NA
> > > and Asia, and the other one is to cover NA and EU.
> > >
> > > I would suggest running biweekly meetings.
> > >
> > > - Tuesday 4 PM - 5 PM PST for NA and Asia
> > > - Thursday 8 AM - 9 AM PST for NA and EU
> > >
> > > All the events can be recorded and uploaded to Youtube. So people are
> > able
> > > to watch the recordings as well.
> > >
> > > If people are good with this proposal, I am happy to set up and
> > coordinate
> > > the meetings given I have run many meetings in the BookKeeper community
> > > before.
> > >
> > > We can get started next Tuesday and formalize the process as we go.
> > Please
> > > vote for your idea.
> > >
> > > Thanks,
> > > Sijie
> > >
> > >
> > > On Fri, Jan 29, 2021 at 12:05 AM Enrico Olivelli 
> > > wrote:
> > >
> > >> Hello everyone,
> > >> in the BookKeeper community we used to have "Community Meetings" in
> > order
> > >> to meet each other, discuss current works on the project, share
> > knowledge
> > >> about current problems.
> > >>
> > >> What about having Community meetings for Pulsar ?
> > >> I would be happy to organize and to host the first meeting.
> > >>
> > >> In Apache we keep the decisions and the discussions on mailing lists,
> so
> > >> these meetings would be only to share information and we are not going
> > to
> > >> make decisions.
> > >>
> > >> The Pulsar community is distributed all over the world, we have people
> > from
> > >> China, the US, Europe, it will be hard to find a good time for
> everyone.
> > >> I suggest for the first meeting to meet at 8:30 PST
> > >>
> > >> For reference, this is the link to the "minutes" of the Community
> > Meetings
> > >> in BK,
> > >> Sijie and Matteo will remember those days
> > >>
> >
> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
> > >>
> > >>
> > >> Enrico
> > >>
> >
> >
>

Re: Virtual Pulsar Community Meetings

[Guangning E]

+1 @Sijie Guo   That looks a little more reasonable.


Thanks,
Guangning

PengHui Li  于2021年1月29日周五 下午5:20写道：

> > I suggest for the first meeting to meet at 8:30 PST
>
> @Enrico Seems the time would be not suitable for people from Asia.
>
> > I would suggest running biweekly meetings.
>
> - Tuesday 4 PM - 5 PM PST for NA and Asia
> - Thursday 8 AM - 9 AM PST for NA and EU
>
> the time is ok for me, +1
>
> Thanks
> - Penghui
>
> Jinfeng Huang  于2021年1月29日周五 下午5:06写道：
>
> > +1
> > That's a very good idea.
> > Taking the time zone into consideration is great.
> >
> >
> > Best Regards,
> > Jennifer
> >
> >
> > On Fri, Jan 29, 2021 at 5:02 PM Yunze Xu 
> > wrote:
> >
> > > +1
> > >
> > > > 2021年1月29日 下午4:53，Sijie Guo  写道：
> > > >
> > > > That's a super great idea! Thank you for bringing this up!
> > > >
> > > > Given there are a lot of committers/contributors are from North
> America
> > > and
> > > > Asia, I think we should pick up a better time that would be suitable
> > for
> > > > people from Asia (Japan and China).
> > > > In the bookkeeper community, we used to run two events. One is to
> cover
> > > NA
> > > > and Asia, and the other one is to cover NA and EU.
> > > >
> > > > I would suggest running biweekly meetings.
> > > >
> > > > - Tuesday 4 PM - 5 PM PST for NA and Asia
> > > > - Thursday 8 AM - 9 AM PST for NA and EU
> > > >
> > > > All the events can be recorded and uploaded to Youtube. So people are
> > > able
> > > > to watch the recordings as well.
> > > >
> > > > If people are good with this proposal, I am happy to set up and
> > > coordinate
> > > > the meetings given I have run many meetings in the BookKeeper
> community
> > > > before.
> > > >
> > > > We can get started next Tuesday and formalize the process as we go.
> > > Please
> > > > vote for your idea.
> > > >
> > > > Thanks,
> > > > Sijie
> > > >
> > > >
> > > > On Fri, Jan 29, 2021 at 12:05 AM Enrico Olivelli <
> eolive...@gmail.com>
> > > > wrote:
> > > >
> > > >> Hello everyone,
> > > >> in the BookKeeper community we used to have "Community Meetings" in
> > > order
> > > >> to meet each other, discuss current works on the project, share
> > > knowledge
> > > >> about current problems.
> > > >>
> > > >> What about having Community meetings for Pulsar ?
> > > >> I would be happy to organize and to host the first meeting.
> > > >>
> > > >> In Apache we keep the decisions and the discussions on mailing
> lists,
> > so
> > > >> these meetings would be only to share information and we are not
> going
> > > to
> > > >> make decisions.
> > > >>
> > > >> The Pulsar community is distributed all over the world, we have
> people
> > > from
> > > >> China, the US, Europe, it will be hard to find a good time for
> > everyone.
> > > >> I suggest for the first meeting to meet at 8:30 PST
> > > >>
> > > >> For reference, this is the link to the "minutes" of the Community
> > > Meetings
> > > >> in BK,
> > > >> Sijie and Matteo will remember those days
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
> > > >>
> > > >>
> > > >> Enrico
> > > >>
> > >
> > >
> >
>

[GitHub] [pulsar-client-node] zhengdai opened a new issue #139: fatal error: pulsar/c/message.h: No such file or directory

[GitBox]

zhengdai opened a new issue #139:
URL: https://github.com/apache/pulsar-client-node/issues/139


   I'm using Ubuntu 18.04.4，and I install Pulsar C++ client by apt install 
./apache-pulsar-client*.deb, and  the file libpulsar.so.2.7.0 and 
libpulsarnossl.so.2.7.0 are in the /usr/lib path. but when I use npm to install 
pulsar-client package, I still got the following error:
   `
   In file included from ../src/addon.cc:20:0:
   ../src/Message.h:24:10: fatal error: pulsar/c/message.h: No such file or 
directory
#include 
 ^~~~
   `
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [Discuss] draft PIP for "Changes to GitHub Actions based Pulsar CI"

[Sijie Guo]

Lari,

Thank you for bringing this proposal up! This is a great initiative.

However, I agreed with Yong. We have spent tons of effort splitting one
large workflow into multiple smaller workflows.

If this proposal is blocked by the other proposal, we should focus on
getting the changes for the other proposal before talking about merging
them.

Thanks,
Sijie

On Thu, Jan 28, 2021 at 9:55 PM Lari Hotari  wrote:

> Thank you for the comments Penghui.
>
> Exactly what you said, we should make the tests stable.
> The proposals in the other draft PIP "Changes to flaky test handling" deals
> with that.
> It's currently a draft and needs more eyes. Would you be able to take a
> closer look at that too?
>
> BR, Lari
>
> On Fri, Jan 29, 2021 at 6:41 AM PengHui Li 
> wrote:
>
> > Currently, especially for the integration tests, a lot time to build
> > pulsar distributions and docker images.
> > I think before merge tests we should to make the test stable, otherwise
> > rerun the test will become more expensive.
> >
> > Thanks,
> > Penghui
> > On Jan 29, 2021, 11:55 AM +0800, Yong Zhang  >,
> > wrote:
> > > I am not sure that merge all the workflows into one workflow is a good
> > > idea. As
> > > I know, Github Actions doesn't allow to rerun a single job in a
> workflow.
> > > That means
> > > if there has any failure in the workflow, we need to rerun all
> > > steps/stage. There has
> > > a worst-case is we failed in the different tests when rerunning it and
> > this
> > > would take
> > > more time to pass the CI.
> > >
> > > ---
> > > Yong
> > >
> > > On Fri, 29 Jan 2021 at 01:14, Lari Hotari 
> wrote:
> > >
> > > > Dear Pulsar community members,
> > > >
> > > > Currently, the Pulsar GitHub Actions workflows are consuming the
> > majority
> > > > of the shared pool of resources allocated for github.com/apache
> > projects.
> > > > Other Apache projects have been impacted and there is a demand to
> > improve
> > > > the Pulsar CI
> > > > 
> > asap.
> > > >
> > > > In GitHub Actions Runners, the unit of resources is the time that a
> > Runner
> > > > is occupied. I observed the workflow runs for handling a single Pull
> > > > Request (in my personal fork) and these were the running durations:
> > > > Workflow name Duration
> > > > CI - Build - MacOS 0:17:23
> > > > CI - Go Functions style check 0:02:38
> > > > CI - Unit - Brokers - Other 0:15:40
> > > > CI - Unit - Brokers - Client Impl 0:16:28
> > > > CI - Misc 0:16:51
> > > > CI - Unit - Proxy 0:14:23
> > > > CI - Go Functions Tests 0:22:08
> > > > CI - CPP, Python Tests 0:23:30
> > > > CI - Unit 0:42:11
> > > > CI - Integration - Sql 1:00:13
> > > > CI - Integration - Tiered JCloud 1:00:18
> > > > CI - Integration - Tiered FileSystem 1:00:13
> > > > CI - Integration - Function State 1:00:12
> > > > CI - Integration - Cli 1:10:22
> > > > CI - Integration - Transaction 1:16:34
> > > > CI - Integration - Process 1:11:23
> > > > CI - Shade - Test 1:15:45
> > > > CI - Unit - Brokers - Client Api 0:26:13
> > > > CI - Unit - Brokers - Broker Group 2 0:35:05
> > > > CI - Integration - Standalone 0:45:29
> > > > CI - Integration - Messaging 1:00:23
> > > > CI - Integration - Thread 1:00:19
> > > > CI - Integration - Backwards Compatibility 1:00:19
> > > > CI - Integration - Schema 1:00:19
> > > > CI - Unit - Brokers - Broker Group 1 2:02:31
> > > > TOTAL 19:36:50
> > > >
> > > > *In this case, the total resource consumption of GitHub Actions
> > Runners is
> > > > 19 hours 36 minutes 50 seconds for a single pull request to
> > apache/pulsar.*
> > > >
> > > > Since GitHub Actions Runner resource pool utilization is very high,
> > this
> > > > leads to the build queue to grow and take a long time to process.
> > > >
> > > > I have been looking for ways to improve the Pulsar CI for the last 3
> > > > months. During this period I worked on a few experiments. The
> learnings
> > > > from the past experiments are documented at a high level in the
> > following
> > > > draft PIP document.
> > > >
> > > > *The draft PIP "Changes to GitHub Actions based Pulsar CI" document
> is
> > a
> > > > Google doc:*
> > > >
> > > >
> >
> https://docs.google.com/document/d/1FNEWD3COdnNGMiryO9qBUW_83qtzAhqjDI5wwmPD-YE/edit?usp=sharing
> > > >
> > > > *Please participate* so that we get the plan adjusted based on the
> > feedback
> > > > asap. If there's already a similar effort ongoing, I hope we can join
> > > > efforts.
> > > >
> > > > *Let's fix Pulsar CI!*
> > > >
> > > > BR, Lari
> > > >
> >
>

Re: [E] Re: [PIP-78] Split the individual acknowledgments into multiple entries

[Sijie Guo]

Rajan - I understand your concern about memory. Lin, Penghui, and I also
acknowledged that none of the implementations solve every use case. Each
implementation has its limitation and concerns.

I am trying to find a way for both parties (You and Lin) can explore
different implementations. An abstraction/interface sounds like a
reasonable approach to take, no? We have a similar situation in the delayed
message scheduler. The current queue-based implementation works for certain
workloads but doesn't work for delayed messages spanning a long time span.
But the delayed message scheduler is an interface that allows people to
implement a better solution (i.e. a hashed-wheel-based implementation).
Does that make sense?

Thanks,
Sijie


On Thu, Jan 28, 2021 at 11:54 AM Rajan Dhabalia 
wrote:

> My only point was, if the broker tries to manage more than 10M unack
> message ranges (per subscription) then the broker has to face many negative
> consequences in terms of memory and CPU, and then it will require to build
> additional debt to solve such problems. Therefore, I consider 10M unack
> message ranges are more than enough for any application, if not then we
> should prevent those applications to not go beyond such thresholds to
> handle the back-pressure of unack messages and not allowing applications to
> build a cache at broker-side.
>
> We have seen multiple issues in the past due to large numbers of unack
> messages in memory and to handle such abuse, brokers have configurations to
> allow max unack messages per consumer (maxUnackedMessagesPerConsumer:
> default 50K) and subscription (maxUnackedMessagesPerSubscription: default
> 200K). So, brokers already have large enough limits to store unack messages
> and going beyond that limit causes scaling issues in brokers, so better to
> keep the system stable and safe.
>
> Thanks,
> Rajan
>
> On Thu, Jan 28, 2021 at 9:50 AM Sijie Guo  wrote:
>
> > Agreed with Lin. I think we should try to abstract this into an interface
> > and allow different implementations.
> >
> > Rajan - what is your real concern making it abstract?
> >
> > - Sijie
> >
> > On Wed, Jan 27, 2021 at 7:37 PM Lin Lin  wrote:
> >
> > > Hi Rajan,
> > > Thank you for your PR.
> > > The main difference lies in whether 10MB is enough and memory doubling
> > > problem, which is caused by different business scenarios.
> > > In some business scenario, the QPS of 20k/s is considered to be very
> low,
> > > and requests exceeding this order of magnitude are common.
> > > If it is only increased to 10MB, the time exceeding the threshold only
> > > changes from 30 seconds to 60 seconds, and the problems in PIP are
> still
> > > not solved.
> > > "large enough" may be base on your scenario, and in some scenario, it
> is
> > > not enough in most cases...
> > > Because the problem has not been solved, I suggest to abstract, so that
> > > different people can choose.
> > > Your PR is an improvement to the current performance, there is no
> > conflict
> > > between them.
> > >
> > > Thanks
> > >
> > > On 2021/01/27 03:50:07, Rajan Dhabalia  wrote:
> > > > I have created a PR which should allow brokers to store up to 10M
> > > > unack-message ranges. I think it should be large enough for any
> > usecases
> > > > and probably now, we might not need to introduce abstraction for ack
> > > > management to avoid any further complexity in message acknowledgement
> > > path
> > > > as well.
> > > > https://github.com/apache/pulsar/pull/9292
> > > >
> > > > Thanks,
> > > > Rajan
> > >
> > >
> >
>

Re: [PIP-78] Reduce redundant producers from partitioned producer

[Sijie Guo]

Yuri,

Thank you for bringing this up! This is a super helpful proposal!

The problem is very similar to what an RPC framework (like Finagle) with
client-side load balancing has.

An RPC framework with a client-side load-balancing mechanism needs to send
requests across multiple nodes. If you have an RPC service that has
thousands of nodes, there are thousands of clients connecting to that RPC
service. How to reduce the connections and how to effectively load balance
requests across thousands of nodes are the problems that a client-side
loading technology needs to solve. If you think about "partition" as "node"
and "partitioned producer" as "RPC client", the problem is exactly the
same. Finagle (the Twitter RPC framework) has implemented a lot of client-side
load-balancing algorithms
 and
there are some great articles that you can reference

.

I agree with the direction of introducing a mechanism to reduce the number
of producers in a partitioned topic producer. However, I have a concern
about introducing `.numPartitionsLimit(10)` directly to the producer
builder. It limits the possibility to implement different algorithms on
selecting partitions.

So instead of directly implementing the logic within the partitioned topic
producer, I think the proposal can be broken into two parts:

1) Introduce some kind of lazy-loading mechanism in the partitioned
producer to initialize the producers for partitions lazily. I.e., only
initialize a producer when the message router selects a partition.
2) Implement a message router that only selects one or N partitions.

In this way, the partitioned producer is only responsible for managing a
collection of producers, and the message router is responsible for
selecting the partitions. This allows people to be able to implement
different message routers. We can even adopt the client-side load balancing
algorithms from Finagle.

Thanks,
Sijie

On Wed, Jan 27, 2021 at 7:18 PM Yuri Mizushima 
wrote:

> I notice that PIP-78 has already assigned to another issue.
>
> https://mail-archives.apache.org/mod_mbox/pulsar-dev/202101.mbox/%3CCAG%3DTQOrPH49v9ToDE_aeQzEiDC%2BEgSR61ERoqanpWfQGvEB_Vw%40mail.gmail.com%3E
>
> So, I'll change the PIP number to 79.
>
> https://github.com/apache/pulsar/wiki/PIP-79%3A-Reduce-redundant-producers-from-partitioned-producer
>
> Regards,
> --
> Yuri Mizushima
> yumiz...@yahoo-corp.jp
>
> "Yuri Mizushima"  wrote:
>
> Dear Pulsar community,
>
> When partitioned producer connects to partitioned topic,
> sometimes doesn't need to connect to all of partitions depending on
> rate, routing mode, etc.
> So, I drafted a PIP about reducing redundant producers from
> partitioned producer.
> I'd like to use system resources (e.g. connections between Client and
> Broker, memory usage of both Client and Broker)
> more efficiently by this feature.
>
> https://github.com/apache/pulsar/wiki/PIP-78%3A-Reduce-redundant-producers-from-partitioned-producer
>
> Feel free to ask me any questions or suggestions, etc.
>
> Best regards,
> --
> Yuri Mizushima
> yumiz...@yahoo-corp.jp
>
>
>

Re: Virtual Pulsar Community Meetings

[Sijie Guo]

FYI - I duplicated the BookKeeper community meeting doc and created one for
Pulsar.
https://docs.google.com/document/d/19dXkVXeU2q_nHmkG8zURjKnYlvD96TbKf5KjYyASsOE/edit#

Really excited to look forward to the first community meeting!

- Sijie

On Fri, Jan 29, 2021 at 12:53 AM Sijie Guo  wrote:

> That's a super great idea! Thank you for bringing this up!
>
> Given there are a lot of committers/contributors are from North America
> and Asia, I think we should pick up a better time that would be suitable
> for people from Asia (Japan and China).
> In the bookkeeper community, we used to run two events. One is to cover NA
> and Asia, and the other one is to cover NA and EU.
>
> I would suggest running biweekly meetings.
>
> - Tuesday 4 PM - 5 PM PST for NA and Asia
> - Thursday 8 AM - 9 AM PST for NA and EU
>
> All the events can be recorded and uploaded to Youtube. So people are able
> to watch the recordings as well.
>
> If people are good with this proposal, I am happy to set up and coordinate
> the meetings given I have run many meetings in the BookKeeper community
> before.
>
> We can get started next Tuesday and formalize the process as we go. Please
> vote for your idea.
>
> Thanks,
> Sijie
>
>
> On Fri, Jan 29, 2021 at 12:05 AM Enrico Olivelli 
> wrote:
>
>> Hello everyone,
>> in the BookKeeper community we used to have "Community Meetings" in order
>> to meet each other, discuss current works on the project, share knowledge
>> about current problems.
>>
>> What about having Community meetings for Pulsar ?
>> I would be happy to organize and to host the first meeting.
>>
>> In Apache we keep the decisions and the discussions on mailing lists, so
>> these meetings would be only to share information and we are not going to
>> make decisions.
>>
>> The Pulsar community is distributed all over the world, we have people
>> from
>> China, the US, Europe, it will be hard to find a good time for everyone.
>> I suggest for the first meeting to meet at 8:30 PST
>>
>> For reference, this is the link to the "minutes" of the Community Meetings
>> in BK,
>> Sijie and Matteo will remember those days
>> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
>>
>>
>> Enrico
>>
>

Re: Virtual Pulsar Community Meetings

[Anonymitaet _]

+1 

Exchange of ideals offers us more opportunities to learn about Pulsar and the 
community.

On 2021/1/29, 16:53, "Sijie Guo"  wrote:

That's a super great idea! Thank you for bringing this up!

Given there are a lot of committers/contributors are from North America and
Asia, I think we should pick up a better time that would be suitable for
people from Asia (Japan and China).
In the bookkeeper community, we used to run two events. One is to cover NA
and Asia, and the other one is to cover NA and EU.

I would suggest running biweekly meetings.

- Tuesday 4 PM - 5 PM PST for NA and Asia
- Thursday 8 AM - 9 AM PST for NA and EU

All the events can be recorded and uploaded to Youtube. So people are able
to watch the recordings as well.

If people are good with this proposal, I am happy to set up and coordinate
the meetings given I have run many meetings in the BookKeeper community
before.

We can get started next Tuesday and formalize the process as we go. Please
vote for your idea.

Thanks,
Sijie


On Fri, Jan 29, 2021 at 12:05 AM Enrico Olivelli 
wrote:

> Hello everyone,
> in the BookKeeper community we used to have "Community Meetings" in order
> to meet each other, discuss current works on the project, share knowledge
> about current problems.
>
> What about having Community meetings for Pulsar ?
> I would be happy to organize and to host the first meeting.
>
> In Apache we keep the decisions and the discussions on mailing lists, so
> these meetings would be only to share information and we are not going to
> make decisions.
>
> The Pulsar community is distributed all over the world, we have people 
from
> China, the US, Europe, it will be hard to find a good time for everyone.
> I suggest for the first meeting to meet at 8:30 PST
>
> For reference, this is the link to the "minutes" of the Community Meetings
> in BK,
> Sijie and Matteo will remember those days
> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
>
>
> Enrico
>

Re: Virtual Pulsar Community Meetings

[ran leon]

+1 Sijie Guo 

The virtual Pulsar community meeting is so cool and I think Sijie's proposal is 
more reasonable, thanks.

Best Regards,
Ran

On 2021/01/29 08:53:40, Sijie Guo  wrote: 
> That's a super great idea! Thank you for bringing this up!
> 
> Given there are a lot of committers/contributors are from North America and
> Asia, I think we should pick up a better time that would be suitable for
> people from Asia (Japan and China).
> In the bookkeeper community, we used to run two events. One is to cover NA
> and Asia, and the other one is to cover NA and EU.
> 
> I would suggest running biweekly meetings.
> 
> - Tuesday 4 PM - 5 PM PST for NA and Asia
> - Thursday 8 AM - 9 AM PST for NA and EU
> 
> All the events can be recorded and uploaded to Youtube. So people are able
> to watch the recordings as well.
> 
> If people are good with this proposal, I am happy to set up and coordinate
> the meetings given I have run many meetings in the BookKeeper community
> before.
> 
> We can get started next Tuesday and formalize the process as we go. Please
> vote for your idea.
> 
> Thanks,
> Sijie
> 
> 
> On Fri, Jan 29, 2021 at 12:05 AM Enrico Olivelli 
> wrote:
> 
> > Hello everyone,
> > in the BookKeeper community we used to have "Community Meetings" in order
> > to meet each other, discuss current works on the project, share knowledge
> > about current problems.
> >
> > What about having Community meetings for Pulsar ?
> > I would be happy to organize and to host the first meeting.
> >
> > In Apache we keep the decisions and the discussions on mailing lists, so
> > these meetings would be only to share information and we are not going to
> > make decisions.
> >
> > The Pulsar community is distributed all over the world, we have people from
> > China, the US, Europe, it will be hard to find a good time for everyone.
> > I suggest for the first meeting to meet at 8:30 PST
> >
> > For reference, this is the link to the "minutes" of the Community Meetings
> > in BK,
> > Sijie and Matteo will remember those days
> > https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
> >
> >
> > Enrico
> >
> 

Re: [Discuss] draft PIP for "Changes to GitHub Actions based Pulsar CI"

[Lari Hotari]

Thanks for the feedback, Sijie.

> If this proposal is blocked by the other proposal, we should focus on
getting the changes for the other proposal before talking about merging
them.

Yes, the current proposal depends on the draft PIP for "Changes to flaky
test handling". I'll follow up on fixing the flaky test in a new email
thread.

I hope we could get the discussions going on both draft PIPs and find
consensus together as a community.
During the discussions, more solution options will come up. Each solution
has trade offs.
It would be useful to document the options when the community doesn't
immediately agree on a single choice.
I was thinking that these options could be documented in the same draft PIP
documents.

I can give multiple authors editing access to the Google Docs so that we
can keep on editing a single document for both draft PIPs.
Anyone who would want to add more solution options to the documents, please
let me know so that I'll add editing access.

Sijie, would you like to document the option around keeping the workflow as
multiple smaller workflows?
I have understood that the problems that have come up with the Pulsar CI
regarding resource consumption would have to be resolved in that
alternative as well.

I believe that everyone is open to any set of solution alternatives which
solves the problems that we have with Pulsar CI.
We all know that it's urgent to fix Pulsar CI asap. We can do it together.

BR, Lari


On Fri, Jan 29, 2021 at 11:51 AM Sijie Guo  wrote:

> Lari,
>
> Thank you for bringing this proposal up! This is a great initiative.
>
> However, I agreed with Yong. We have spent tons of effort splitting one
> large workflow into multiple smaller workflows.
>
> If this proposal is blocked by the other proposal, we should focus on
> getting the changes for the other proposal before talking about merging
> them.
>
> Thanks,
> Sijie
>
> On Thu, Jan 28, 2021 at 9:55 PM Lari Hotari  wrote:
>
> > Thank you for the comments Penghui.
> >
> > Exactly what you said, we should make the tests stable.
> > The proposals in the other draft PIP "Changes to flaky test handling"
> deals
> > with that.
> > It's currently a draft and needs more eyes. Would you be able to take a
> > closer look at that too?
> >
> > BR, Lari
> >
> > On Fri, Jan 29, 2021 at 6:41 AM PengHui Li 
> > wrote:
> >
> > > Currently, especially for the integration tests, a lot time to build
> > > pulsar distributions and docker images.
> > > I think before merge tests we should to make the test stable, otherwise
> > > rerun the test will become more expensive.
> > >
> > > Thanks,
> > > Penghui
> > > On Jan 29, 2021, 11:55 AM +0800, Yong Zhang <
> zhangyong1025...@gmail.com
> > >,
> > > wrote:
> > > > I am not sure that merge all the workflows into one workflow is a
> good
> > > > idea. As
> > > > I know, Github Actions doesn't allow to rerun a single job in a
> > workflow.
> > > > That means
> > > > if there has any failure in the workflow, we need to rerun all
> > > > steps/stage. There has
> > > > a worst-case is we failed in the different tests when rerunning it
> and
> > > this
> > > > would take
> > > > more time to pass the CI.
> > > >
> > > > ---
> > > > Yong
> > > >
> > > > On Fri, 29 Jan 2021 at 01:14, Lari Hotari 
> > wrote:
> > > >
> > > > > Dear Pulsar community members,
> > > > >
> > > > > Currently, the Pulsar GitHub Actions workflows are consuming the
> > > majority
> > > > > of the shared pool of resources allocated for github.com/apache
> > > projects.
> > > > > Other Apache projects have been impacted and there is a demand to
> > > improve
> > > > > the Pulsar CI
> > > > >  >
> > > asap.
> > > > >
> > > > > In GitHub Actions Runners, the unit of resources is the time that a
> > > Runner
> > > > > is occupied. I observed the workflow runs for handling a single
> Pull
> > > > > Request (in my personal fork) and these were the running durations:
> > > > > Workflow name Duration
> > > > > CI - Build - MacOS 0:17:23
> > > > > CI - Go Functions style check 0:02:38
> > > > > CI - Unit - Brokers - Other 0:15:40
> > > > > CI - Unit - Brokers - Client Impl 0:16:28
> > > > > CI - Misc 0:16:51
> > > > > CI - Unit - Proxy 0:14:23
> > > > > CI - Go Functions Tests 0:22:08
> > > > > CI - CPP, Python Tests 0:23:30
> > > > > CI - Unit 0:42:11
> > > > > CI - Integration - Sql 1:00:13
> > > > > CI - Integration - Tiered JCloud 1:00:18
> > > > > CI - Integration - Tiered FileSystem 1:00:13
> > > > > CI - Integration - Function State 1:00:12
> > > > > CI - Integration - Cli 1:10:22
> > > > > CI - Integration - Transaction 1:16:34
> > > > > CI - Integration - Process 1:11:23
> > > > > CI - Shade - Test 1:15:45
> > > > > CI - Unit - Brokers - Client Api 0:26:13
> > > > > CI - Unit - Brokers - Broker Group 2 0:35:05
> > > > > CI - Integration - Standalone 0:45:29
> > > > > CI - Integration - Messaging 1:00:23
> > > > > CI - Integration - Thread 1:00:19
> > > > > CI

Re: Virtual Pulsar Community Meetings

[Enrico Olivelli]

Thank you all,
and thank you Sijie for the document.

I would prefer to have initially only one meeting, I am fine with 4 PM PST,
as most of the contributors are from the US and from Asia, it is not so
late for Europe (1 AM).
If one meeting works then we can think about splitting it when there is
demand from people from Europe

So I would like to start on 2nd February 2021 at 4 PM PST as written in the
Google Doc

@Sijie did you mention that we should "VOTE" before starting this activity
?

Enrico


Il giorno ven 29 gen 2021 alle ore 13:03 ran leon 
ha scritto:

> +1 Sijie Guo 
>
> The virtual Pulsar community meeting is so cool and I think Sijie's
> proposal is more reasonable, thanks.
>
> Best Regards,
> Ran
>
> On 2021/01/29 08:53:40, Sijie Guo  wrote:
> > That's a super great idea! Thank you for bringing this up!
> >
> > Given there are a lot of committers/contributors are from North America
> and
> > Asia, I think we should pick up a better time that would be suitable for
> > people from Asia (Japan and China).
> > In the bookkeeper community, we used to run two events. One is to cover
> NA
> > and Asia, and the other one is to cover NA and EU.
> >
> > I would suggest running biweekly meetings.
> >
> > - Tuesday 4 PM - 5 PM PST for NA and Asia
> > - Thursday 8 AM - 9 AM PST for NA and EU
> >
> > All the events can be recorded and uploaded to Youtube. So people are
> able
> > to watch the recordings as well.
> >
> > If people are good with this proposal, I am happy to set up and
> coordinate
> > the meetings given I have run many meetings in the BookKeeper community
> > before.
> >
> > We can get started next Tuesday and formalize the process as we go.
> Please
> > vote for your idea.
> >
> > Thanks,
> > Sijie
> >
> >
> > On Fri, Jan 29, 2021 at 12:05 AM Enrico Olivelli 
> > wrote:
> >
> > > Hello everyone,
> > > in the BookKeeper community we used to have "Community Meetings" in
> order
> > > to meet each other, discuss current works on the project, share
> knowledge
> > > about current problems.
> > >
> > > What about having Community meetings for Pulsar ?
> > > I would be happy to organize and to host the first meeting.
> > >
> > > In Apache we keep the decisions and the discussions on mailing lists,
> so
> > > these meetings would be only to share information and we are not going
> to
> > > make decisions.
> > >
> > > The Pulsar community is distributed all over the world, we have people
> from
> > > China, the US, Europe, it will be hard to find a good time for
> everyone.
> > > I suggest for the first meeting to meet at 8:30 PST
> > >
> > > For reference, this is the link to the "minutes" of the Community
> Meetings
> > > in BK,
> > > Sijie and Matteo will remember those days
> > >
> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
> > >
> > >
> > > Enrico
> > >
> >
>

Re: Virtual Pulsar Community Meetings

[Lari Hotari]

Thank you Sijie for taking this forward.

I'd love to have a single global meeting. 4 PM PST is just too late for me
since it's 2AM.

I'd like to suggest 2 hours earlier, Tuesdays at 2PM PST.  Would that work
for all?

-Lari

On Fri, Jan 29, 2021 at 12:28 PM Sijie Guo  wrote:

> FYI - I duplicated the BookKeeper community meeting doc and created one for
> Pulsar.
>
> https://docs.google.com/document/d/19dXkVXeU2q_nHmkG8zURjKnYlvD96TbKf5KjYyASsOE/edit#
>
> Really excited to look forward to the first community meeting!
>
> - Sijie
>
> On Fri, Jan 29, 2021 at 12:53 AM Sijie Guo  wrote:
>
> > That's a super great idea! Thank you for bringing this up!
> >
> > Given there are a lot of committers/contributors are from North America
> > and Asia, I think we should pick up a better time that would be suitable
> > for people from Asia (Japan and China).
> > In the bookkeeper community, we used to run two events. One is to cover
> NA
> > and Asia, and the other one is to cover NA and EU.
> >
> > I would suggest running biweekly meetings.
> >
> > - Tuesday 4 PM - 5 PM PST for NA and Asia
> > - Thursday 8 AM - 9 AM PST for NA and EU
> >
> > All the events can be recorded and uploaded to Youtube. So people are
> able
> > to watch the recordings as well.
> >
> > If people are good with this proposal, I am happy to set up and
> coordinate
> > the meetings given I have run many meetings in the BookKeeper community
> > before.
> >
> > We can get started next Tuesday and formalize the process as we go.
> Please
> > vote for your idea.
> >
> > Thanks,
> > Sijie
> >
> >
> > On Fri, Jan 29, 2021 at 12:05 AM Enrico Olivelli 
> > wrote:
> >
> >> Hello everyone,
> >> in the BookKeeper community we used to have "Community Meetings" in
> order
> >> to meet each other, discuss current works on the project, share
> knowledge
> >> about current problems.
> >>
> >> What about having Community meetings for Pulsar ?
> >> I would be happy to organize and to host the first meeting.
> >>
> >> In Apache we keep the decisions and the discussions on mailing lists, so
> >> these meetings would be only to share information and we are not going
> to
> >> make decisions.
> >>
> >> The Pulsar community is distributed all over the world, we have people
> >> from
> >> China, the US, Europe, it will be hard to find a good time for everyone.
> >> I suggest for the first meeting to meet at 8:30 PST
> >>
> >> For reference, this is the link to the "minutes" of the Community
> Meetings
> >> in BK,
> >> Sijie and Matteo will remember those days
> >>
> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
> >>
> >>
> >> Enrico
> >>
> >
>

Re: Virtual Pulsar Community Meetings

[Ran Gao]

+1 @sijie

The virtual Pulsar community meeting is so cool and I think Sijie's proposal is 
more reasonable, thanks.

Best Regards,
Ran

On 2021/01/29 08:53:40, Sijie Guo  wrote: 
> That's a super great idea! Thank you for bringing this up!
> 
> Given there are a lot of committers/contributors are from North America and
> Asia, I think we should pick up a better time that would be suitable for
> people from Asia (Japan and China).
> In the bookkeeper community, we used to run two events. One is to cover NA
> and Asia, and the other one is to cover NA and EU.
> 
> I would suggest running biweekly meetings.
> 
> - Tuesday 4 PM - 5 PM PST for NA and Asia
> - Thursday 8 AM - 9 AM PST for NA and EU
> 
> All the events can be recorded and uploaded to Youtube. So people are able
> to watch the recordings as well.
> 
> If people are good with this proposal, I am happy to set up and coordinate
> the meetings given I have run many meetings in the BookKeeper community
> before.
> 
> We can get started next Tuesday and formalize the process as we go. Please
> vote for your idea.
> 
> Thanks,
> Sijie
> 
> 
> On Fri, Jan 29, 2021 at 12:05 AM Enrico Olivelli 
> wrote:
> 
> > Hello everyone,
> > in the BookKeeper community we used to have "Community Meetings" in order
> > to meet each other, discuss current works on the project, share knowledge
> > about current problems.
> >
> > What about having Community meetings for Pulsar ?
> > I would be happy to organize and to host the first meeting.
> >
> > In Apache we keep the decisions and the discussions on mailing lists, so
> > these meetings would be only to share information and we are not going to
> > make decisions.
> >
> > The Pulsar community is distributed all over the world, we have people from
> > China, the US, Europe, it will be hard to find a good time for everyone.
> > I suggest for the first meeting to meet at 8:30 PST
> >
> > For reference, this is the link to the "minutes" of the Community Meetings
> > in BK,
> > Sijie and Matteo will remember those days
> > https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
> >
> >
> > Enrico
> >
> 

Re: [Discuss] draft PIP for "Changes to GitHub Actions based Pulsar CI"

[Sijie Guo]

Lari,

Yes, we can keep this proposal open for discussion. That's for sure.

I just don't have any good solution at this moment with a multiple-workflow
approach using Github Actions.

An alternative is to look into Azure Pipeline, which the Flink community is
using.
We are still learning there. Will post thoughts here once we have a better
idea.

Thanks,
Sijie

On Fri, Jan 29, 2021 at 5:07 AM Lari Hotari  wrote:

> Thanks for the feedback, Sijie.
>
> > If this proposal is blocked by the other proposal, we should focus on
> getting the changes for the other proposal before talking about merging
> them.
>
> Yes, the current proposal depends on the draft PIP for "Changes to flaky
> test handling". I'll follow up on fixing the flaky test in a new email
> thread.
>
> I hope we could get the discussions going on both draft PIPs and find
> consensus together as a community.
> During the discussions, more solution options will come up. Each solution
> has trade offs.
> It would be useful to document the options when the community doesn't
> immediately agree on a single choice.
> I was thinking that these options could be documented in the same draft PIP
> documents.
>
> I can give multiple authors editing access to the Google Docs so that we
> can keep on editing a single document for both draft PIPs.
> Anyone who would want to add more solution options to the documents, please
> let me know so that I'll add editing access.
>
> Sijie, would you like to document the option around keeping the workflow as
> multiple smaller workflows?
> I have understood that the problems that have come up with the Pulsar CI
> regarding resource consumption would have to be resolved in that
> alternative as well.
>
> I believe that everyone is open to any set of solution alternatives which
> solves the problems that we have with Pulsar CI.
> We all know that it's urgent to fix Pulsar CI asap. We can do it together.
>
> BR, Lari
>
>
> On Fri, Jan 29, 2021 at 11:51 AM Sijie Guo  wrote:
>
> > Lari,
> >
> > Thank you for bringing this proposal up! This is a great initiative.
> >
> > However, I agreed with Yong. We have spent tons of effort splitting one
> > large workflow into multiple smaller workflows.
> >
> > If this proposal is blocked by the other proposal, we should focus on
> > getting the changes for the other proposal before talking about merging
> > them.
> >
> > Thanks,
> > Sijie
> >
> > On Thu, Jan 28, 2021 at 9:55 PM Lari Hotari  wrote:
> >
> > > Thank you for the comments Penghui.
> > >
> > > Exactly what you said, we should make the tests stable.
> > > The proposals in the other draft PIP "Changes to flaky test handling"
> > deals
> > > with that.
> > > It's currently a draft and needs more eyes. Would you be able to take a
> > > closer look at that too?
> > >
> > > BR, Lari
> > >
> > > On Fri, Jan 29, 2021 at 6:41 AM PengHui Li 
> > > wrote:
> > >
> > > > Currently, especially for the integration tests, a lot time to build
> > > > pulsar distributions and docker images.
> > > > I think before merge tests we should to make the test stable,
> otherwise
> > > > rerun the test will become more expensive.
> > > >
> > > > Thanks,
> > > > Penghui
> > > > On Jan 29, 2021, 11:55 AM +0800, Yong Zhang <
> > zhangyong1025...@gmail.com
> > > >,
> > > > wrote:
> > > > > I am not sure that merge all the workflows into one workflow is a
> > good
> > > > > idea. As
> > > > > I know, Github Actions doesn't allow to rerun a single job in a
> > > workflow.
> > > > > That means
> > > > > if there has any failure in the workflow, we need to rerun all
> > > > > steps/stage. There has
> > > > > a worst-case is we failed in the different tests when rerunning it
> > and
> > > > this
> > > > > would take
> > > > > more time to pass the CI.
> > > > >
> > > > > ---
> > > > > Yong
> > > > >
> > > > > On Fri, 29 Jan 2021 at 01:14, Lari Hotari 
> > > wrote:
> > > > >
> > > > > > Dear Pulsar community members,
> > > > > >
> > > > > > Currently, the Pulsar GitHub Actions workflows are consuming the
> > > > majority
> > > > > > of the shared pool of resources allocated for github.com/apache
> > > > projects.
> > > > > > Other Apache projects have been impacted and there is a demand to
> > > > improve
> > > > > > the Pulsar CI
> > > > > > <
> https://github.com/apache/pulsar/pull/9159#issuecomment-766915396
> > >
> > > > asap.
> > > > > >
> > > > > > In GitHub Actions Runners, the unit of resources is the time
> that a
> > > > Runner
> > > > > > is occupied. I observed the workflow runs for handling a single
> > Pull
> > > > > > Request (in my personal fork) and these were the running
> durations:
> > > > > > Workflow name Duration
> > > > > > CI - Build - MacOS 0:17:23
> > > > > > CI - Go Functions style check 0:02:38
> > > > > > CI - Unit - Brokers - Other 0:15:40
> > > > > > CI - Unit - Brokers - Client Impl 0:16:28
> > > > > > CI - Misc 0:16:51
> > > > > > CI - Unit - Proxy 0:14:23
> > > > > > CI - Go Functions Tests 0:22:08
> > > > > > CI - CPP, Pytho

Re: Virtual Pulsar Community Meetings

[Sijie Guo]

The main reason that I propose two meetings is the timezone issue. I don't
think we are going to find a perfect time for contributors from different
regions.
The OSS collaboration emphasizes asynchronous communication. The purpose of
the meetings is not to turn asynchronous communication into synchronous
communication.
Those meetings are only used for alignments and to make some of the
communications more efficient. So whether it is one meeting or two
meetings, it doesn't really matter.

We can keep two meetings available. People can choose which one to attend
and discuss. If there is a meeting that fewer people attend, we can always
cancel it.

Community meetings are informal events for gathering people together to
discuss topics. I don't think we need a vote to do that.
We can get the first meeting started. As we go, we can make it better.

- Sijie

On Fri, Jan 29, 2021 at 5:15 AM Enrico Olivelli  wrote:

> Thank you all,
> and thank you Sijie for the document.
>
> I would prefer to have initially only one meeting, I am fine with 4 PM PST,
> as most of the contributors are from the US and from Asia, it is not so
> late for Europe (1 AM).
> If one meeting works then we can think about splitting it when there is
> demand from people from Europe
>
> So I would like to start on 2nd February 2021 at 4 PM PST as written in the
> Google Doc
>
> @Sijie did you mention that we should "VOTE" before starting this activity
> ?
>
> Enrico
>
>
> Il giorno ven 29 gen 2021 alle ore 13:03 ran leon 
> ha scritto:
>
> > +1 Sijie Guo 
> >
> > The virtual Pulsar community meeting is so cool and I think Sijie's
> > proposal is more reasonable, thanks.
> >
> > Best Regards,
> > Ran
> >
> > On 2021/01/29 08:53:40, Sijie Guo  wrote:
> > > That's a super great idea! Thank you for bringing this up!
> > >
> > > Given there are a lot of committers/contributors are from North America
> > and
> > > Asia, I think we should pick up a better time that would be suitable
> for
> > > people from Asia (Japan and China).
> > > In the bookkeeper community, we used to run two events. One is to cover
> > NA
> > > and Asia, and the other one is to cover NA and EU.
> > >
> > > I would suggest running biweekly meetings.
> > >
> > > - Tuesday 4 PM - 5 PM PST for NA and Asia
> > > - Thursday 8 AM - 9 AM PST for NA and EU
> > >
> > > All the events can be recorded and uploaded to Youtube. So people are
> > able
> > > to watch the recordings as well.
> > >
> > > If people are good with this proposal, I am happy to set up and
> > coordinate
> > > the meetings given I have run many meetings in the BookKeeper community
> > > before.
> > >
> > > We can get started next Tuesday and formalize the process as we go.
> > Please
> > > vote for your idea.
> > >
> > > Thanks,
> > > Sijie
> > >
> > >
> > > On Fri, Jan 29, 2021 at 12:05 AM Enrico Olivelli 
> > > wrote:
> > >
> > > > Hello everyone,
> > > > in the BookKeeper community we used to have "Community Meetings" in
> > order
> > > > to meet each other, discuss current works on the project, share
> > knowledge
> > > > about current problems.
> > > >
> > > > What about having Community meetings for Pulsar ?
> > > > I would be happy to organize and to host the first meeting.
> > > >
> > > > In Apache we keep the decisions and the discussions on mailing lists,
> > so
> > > > these meetings would be only to share information and we are not
> going
> > to
> > > > make decisions.
> > > >
> > > > The Pulsar community is distributed all over the world, we have
> people
> > from
> > > > China, the US, Europe, it will be hard to find a good time for
> > everyone.
> > > > I suggest for the first meeting to meet at 8:30 PST
> > > >
> > > > For reference, this is the link to the "minutes" of the Community
> > Meetings
> > > > in BK,
> > > > Sijie and Matteo will remember those days
> > > >
> >
> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
> > > >
> > > >
> > > > Enrico
> > > >
> > >
> >
>

Re: Virtual Pulsar Community Meetings

[Sijie Guo]

2 PM PT doesn't work well for Asia people. I have been dealing with a
global team for a while.
There is also on way to cover people in all regions. That's why OSS
communities use asynchronous communication.

The meetings are just for alignments and discussion. It doesn't change how
collaboration work in the ASF.
As I comment in Enrico's reply, we can keep two meetings available. People
can choose which one to attend and discuss.

- Sijie

On Fri, Jan 29, 2021 at 5:58 AM Lari Hotari  wrote:

> Thank you Sijie for taking this forward.
>
> I'd love to have a single global meeting. 4 PM PST is just too late for me
> since it's 2AM.
>
> I'd like to suggest 2 hours earlier, Tuesdays at 2PM PST.  Would that work
> for all?
>
> -Lari
>
> On Fri, Jan 29, 2021 at 12:28 PM Sijie Guo  wrote:
>
> > FYI - I duplicated the BookKeeper community meeting doc and created one
> for
> > Pulsar.
> >
> >
> https://docs.google.com/document/d/19dXkVXeU2q_nHmkG8zURjKnYlvD96TbKf5KjYyASsOE/edit#
> >
> > Really excited to look forward to the first community meeting!
> >
> > - Sijie
> >
> > On Fri, Jan 29, 2021 at 12:53 AM Sijie Guo  wrote:
> >
> > > That's a super great idea! Thank you for bringing this up!
> > >
> > > Given there are a lot of committers/contributors are from North America
> > > and Asia, I think we should pick up a better time that would be
> suitable
> > > for people from Asia (Japan and China).
> > > In the bookkeeper community, we used to run two events. One is to cover
> > NA
> > > and Asia, and the other one is to cover NA and EU.
> > >
> > > I would suggest running biweekly meetings.
> > >
> > > - Tuesday 4 PM - 5 PM PST for NA and Asia
> > > - Thursday 8 AM - 9 AM PST for NA and EU
> > >
> > > All the events can be recorded and uploaded to Youtube. So people are
> > able
> > > to watch the recordings as well.
> > >
> > > If people are good with this proposal, I am happy to set up and
> > coordinate
> > > the meetings given I have run many meetings in the BookKeeper community
> > > before.
> > >
> > > We can get started next Tuesday and formalize the process as we go.
> > Please
> > > vote for your idea.
> > >
> > > Thanks,
> > > Sijie
> > >
> > >
> > > On Fri, Jan 29, 2021 at 12:05 AM Enrico Olivelli 
> > > wrote:
> > >
> > >> Hello everyone,
> > >> in the BookKeeper community we used to have "Community Meetings" in
> > order
> > >> to meet each other, discuss current works on the project, share
> > knowledge
> > >> about current problems.
> > >>
> > >> What about having Community meetings for Pulsar ?
> > >> I would be happy to organize and to host the first meeting.
> > >>
> > >> In Apache we keep the decisions and the discussions on mailing lists,
> so
> > >> these meetings would be only to share information and we are not going
> > to
> > >> make decisions.
> > >>
> > >> The Pulsar community is distributed all over the world, we have people
> > >> from
> > >> China, the US, Europe, it will be hard to find a good time for
> everyone.
> > >> I suggest for the first meeting to meet at 8:30 PST
> > >>
> > >> For reference, this is the link to the "minutes" of the Community
> > Meetings
> > >> in BK,
> > >> Sijie and Matteo will remember those days
> > >>
> >
> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
> > >>
> > >>
> > >> Enrico
> > >>
> > >
> >
>

Re: [Discuss] draft PIP for "Changes to GitHub Actions based Pulsar CI"

[Lari Hotari]

Hi Sijie,

Let's keep this work going since resolving the problems with Pulsar CI are
urgent.

I took a quick glance on the Azure Pipelines solution in Flink. By Googling
I found https://cwiki.apache.org/confluence/display/FLINK/Azure+Pipelines .
In the repository I found
https://github.com/apache/flink/blob/master/azure-pipelines.yml which
references
https://github.com/apache/flink/blob/master/tools/azure-pipelines/jobs-template.yml

It uses the build matrix feature to parallelize the execution:
https://github.com/apache/flink/blob/dd0ee24e55dab4ae76201103c76495bc4fa0f73b/tools/azure-pipelines/jobs-template.yml#L88-L107

What would be the key benefit for Pulsar CI of using Azure Pipelines over
GitHub Actions?

-Lari

On Fri, Jan 29, 2021 at 6:03 PM Sijie Guo  wrote:

> Lari,
>
> Yes, we can keep this proposal open for discussion. That's for sure.
>
> I just don't have any good solution at this moment with a multiple-workflow
> approach using Github Actions.
>
> An alternative is to look into Azure Pipeline, which the Flink community is
> using.
> We are still learning there. Will post thoughts here once we have a better
> idea.
>
> Thanks,
> Sijie
>
> On Fri, Jan 29, 2021 at 5:07 AM Lari Hotari  wrote:
>
> > Thanks for the feedback, Sijie.
> >
> > > If this proposal is blocked by the other proposal, we should focus on
> > getting the changes for the other proposal before talking about merging
> > them.
> >
> > Yes, the current proposal depends on the draft PIP for "Changes to flaky
> > test handling". I'll follow up on fixing the flaky test in a new email
> > thread.
> >
> > I hope we could get the discussions going on both draft PIPs and find
> > consensus together as a community.
> > During the discussions, more solution options will come up. Each solution
> > has trade offs.
> > It would be useful to document the options when the community doesn't
> > immediately agree on a single choice.
> > I was thinking that these options could be documented in the same draft
> PIP
> > documents.
> >
> > I can give multiple authors editing access to the Google Docs so that we
> > can keep on editing a single document for both draft PIPs.
> > Anyone who would want to add more solution options to the documents,
> please
> > let me know so that I'll add editing access.
> >
> > Sijie, would you like to document the option around keeping the workflow
> as
> > multiple smaller workflows?
> > I have understood that the problems that have come up with the Pulsar CI
> > regarding resource consumption would have to be resolved in that
> > alternative as well.
> >
> > I believe that everyone is open to any set of solution alternatives which
> > solves the problems that we have with Pulsar CI.
> > We all know that it's urgent to fix Pulsar CI asap. We can do it
> together.
> >
> > BR, Lari
> >
> >
> > On Fri, Jan 29, 2021 at 11:51 AM Sijie Guo  wrote:
> >
> > > Lari,
> > >
> > > Thank you for bringing this proposal up! This is a great initiative.
> > >
> > > However, I agreed with Yong. We have spent tons of effort splitting one
> > > large workflow into multiple smaller workflows.
> > >
> > > If this proposal is blocked by the other proposal, we should focus on
> > > getting the changes for the other proposal before talking about merging
> > > them.
> > >
> > > Thanks,
> > > Sijie
> > >
> > > On Thu, Jan 28, 2021 at 9:55 PM Lari Hotari  wrote:
> > >
> > > > Thank you for the comments Penghui.
> > > >
> > > > Exactly what you said, we should make the tests stable.
> > > > The proposals in the other draft PIP "Changes to flaky test handling"
> > > deals
> > > > with that.
> > > > It's currently a draft and needs more eyes. Would you be able to
> take a
> > > > closer look at that too?
> > > >
> > > > BR, Lari
> > > >
> > > > On Fri, Jan 29, 2021 at 6:41 AM PengHui Li 
> > > > wrote:
> > > >
> > > > > Currently, especially for the integration tests, a lot time to
> build
> > > > > pulsar distributions and docker images.
> > > > > I think before merge tests we should to make the test stable,
> > otherwise
> > > > > rerun the test will become more expensive.
> > > > >
> > > > > Thanks,
> > > > > Penghui
> > > > > On Jan 29, 2021, 11:55 AM +0800, Yong Zhang <
> > > zhangyong1025...@gmail.com
> > > > >,
> > > > > wrote:
> > > > > > I am not sure that merge all the workflows into one workflow is a
> > > good
> > > > > > idea. As
> > > > > > I know, Github Actions doesn't allow to rerun a single job in a
> > > > workflow.
> > > > > > That means
> > > > > > if there has any failure in the workflow, we need to rerun all
> > > > > > steps/stage. There has
> > > > > > a worst-case is we failed in the different tests when rerunning
> it
> > > and
> > > > > this
> > > > > > would take
> > > > > > more time to pass the CI.
> > > > > >
> > > > > > ---
> > > > > > Yong
> > > > > >
> > > > > > On Fri, 29 Jan 2021 at 01:14, Lari Hotari  >
> > > > wrote:
> > > > > >
> > > > > > > Dear Pulsar community members,
> > > > > > >
> > > > > > > Currentl

Re: Virtual Pulsar Community Meetings

[Lari Hotari]

Gotcha. Yes, it's impossible to have just one time that works for all. I'm
fine with your original proposal.

-Lari

On Fri, Jan 29, 2021 at 6:30 PM Sijie Guo  wrote:

> 2 PM PT doesn't work well for Asia people. I have been dealing with a
> global team for a while.
> There is also on way to cover people in all regions. That's why OSS
> communities use asynchronous communication.
>
> The meetings are just for alignments and discussion. It doesn't change how
> collaboration work in the ASF.
> As I comment in Enrico's reply, we can keep two meetings available. People
> can choose which one to attend and discuss.
>
> - Sijie
>
> On Fri, Jan 29, 2021 at 5:58 AM Lari Hotari  wrote:
>
> > Thank you Sijie for taking this forward.
> >
> > I'd love to have a single global meeting. 4 PM PST is just too late for
> me
> > since it's 2AM.
> >
> > I'd like to suggest 2 hours earlier, Tuesdays at 2PM PST.  Would that
> work
> > for all?
> >
> > -Lari
> >
> > On Fri, Jan 29, 2021 at 12:28 PM Sijie Guo  wrote:
> >
> > > FYI - I duplicated the BookKeeper community meeting doc and created one
> > for
> > > Pulsar.
> > >
> > >
> >
> https://docs.google.com/document/d/19dXkVXeU2q_nHmkG8zURjKnYlvD96TbKf5KjYyASsOE/edit#
> > >
> > > Really excited to look forward to the first community meeting!
> > >
> > > - Sijie
> > >
> > > On Fri, Jan 29, 2021 at 12:53 AM Sijie Guo  wrote:
> > >
> > > > That's a super great idea! Thank you for bringing this up!
> > > >
> > > > Given there are a lot of committers/contributors are from North
> America
> > > > and Asia, I think we should pick up a better time that would be
> > suitable
> > > > for people from Asia (Japan and China).
> > > > In the bookkeeper community, we used to run two events. One is to
> cover
> > > NA
> > > > and Asia, and the other one is to cover NA and EU.
> > > >
> > > > I would suggest running biweekly meetings.
> > > >
> > > > - Tuesday 4 PM - 5 PM PST for NA and Asia
> > > > - Thursday 8 AM - 9 AM PST for NA and EU
> > > >
> > > > All the events can be recorded and uploaded to Youtube. So people are
> > > able
> > > > to watch the recordings as well.
> > > >
> > > > If people are good with this proposal, I am happy to set up and
> > > coordinate
> > > > the meetings given I have run many meetings in the BookKeeper
> community
> > > > before.
> > > >
> > > > We can get started next Tuesday and formalize the process as we go.
> > > Please
> > > > vote for your idea.
> > > >
> > > > Thanks,
> > > > Sijie
> > > >
> > > >
> > > > On Fri, Jan 29, 2021 at 12:05 AM Enrico Olivelli <
> eolive...@gmail.com>
> > > > wrote:
> > > >
> > > >> Hello everyone,
> > > >> in the BookKeeper community we used to have "Community Meetings" in
> > > order
> > > >> to meet each other, discuss current works on the project, share
> > > knowledge
> > > >> about current problems.
> > > >>
> > > >> What about having Community meetings for Pulsar ?
> > > >> I would be happy to organize and to host the first meeting.
> > > >>
> > > >> In Apache we keep the decisions and the discussions on mailing
> lists,
> > so
> > > >> these meetings would be only to share information and we are not
> going
> > > to
> > > >> make decisions.
> > > >>
> > > >> The Pulsar community is distributed all over the world, we have
> people
> > > >> from
> > > >> China, the US, Europe, it will be hard to find a good time for
> > everyone.
> > > >> I suggest for the first meeting to meet at 8:30 PST
> > > >>
> > > >> For reference, this is the link to the "minutes" of the Community
> > > Meetings
> > > >> in BK,
> > > >> Sijie and Matteo will remember those days
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
> > > >>
> > > >>
> > > >> Enrico
> > > >>
> > > >
> > >
> >
>

Re: [Discuss] draft PIP for "Changes to GitHub Actions based Pulsar CI"

[Sijie Guo]

Currently, Github Actions are shared across one large `apache`
organization. It is the main problem for GA-based CI besides flaky tests.

If we use Azure Pipeline, we can have a dedicated project for the pulsar.
So we will have more resources to run.
It will solve the problem that this proposal tries to solve. The approach
has been used by Flink. We have started some experiments. We will share
some of results here next week.

Thanks,
Sijie

On Fri, Jan 29, 2021 at 8:34 AM Lari Hotari  wrote:

> Hi Sijie,
>
> Let's keep this work going since resolving the problems with Pulsar CI are
> urgent.
>
> I took a quick glance on the Azure Pipelines solution in Flink. By Googling
> I found https://cwiki.apache.org/confluence/display/FLINK/Azure+Pipelines
> .
> In the repository I found
> https://github.com/apache/flink/blob/master/azure-pipelines.yml which
> references
>
> https://github.com/apache/flink/blob/master/tools/azure-pipelines/jobs-template.yml
>
> It uses the build matrix feature to parallelize the execution:
>
> https://github.com/apache/flink/blob/dd0ee24e55dab4ae76201103c76495bc4fa0f73b/tools/azure-pipelines/jobs-template.yml#L88-L107
>
> What would be the key benefit for Pulsar CI of using Azure Pipelines over
> GitHub Actions?
>
> -Lari
>
> On Fri, Jan 29, 2021 at 6:03 PM Sijie Guo  wrote:
>
> > Lari,
> >
> > Yes, we can keep this proposal open for discussion. That's for sure.
> >
> > I just don't have any good solution at this moment with a
> multiple-workflow
> > approach using Github Actions.
> >
> > An alternative is to look into Azure Pipeline, which the Flink community
> is
> > using.
> > We are still learning there. Will post thoughts here once we have a
> better
> > idea.
> >
> > Thanks,
> > Sijie
> >
> > On Fri, Jan 29, 2021 at 5:07 AM Lari Hotari  wrote:
> >
> > > Thanks for the feedback, Sijie.
> > >
> > > > If this proposal is blocked by the other proposal, we should focus on
> > > getting the changes for the other proposal before talking about merging
> > > them.
> > >
> > > Yes, the current proposal depends on the draft PIP for "Changes to
> flaky
> > > test handling". I'll follow up on fixing the flaky test in a new email
> > > thread.
> > >
> > > I hope we could get the discussions going on both draft PIPs and find
> > > consensus together as a community.
> > > During the discussions, more solution options will come up. Each
> solution
> > > has trade offs.
> > > It would be useful to document the options when the community doesn't
> > > immediately agree on a single choice.
> > > I was thinking that these options could be documented in the same draft
> > PIP
> > > documents.
> > >
> > > I can give multiple authors editing access to the Google Docs so that
> we
> > > can keep on editing a single document for both draft PIPs.
> > > Anyone who would want to add more solution options to the documents,
> > please
> > > let me know so that I'll add editing access.
> > >
> > > Sijie, would you like to document the option around keeping the
> workflow
> > as
> > > multiple smaller workflows?
> > > I have understood that the problems that have come up with the Pulsar
> CI
> > > regarding resource consumption would have to be resolved in that
> > > alternative as well.
> > >
> > > I believe that everyone is open to any set of solution alternatives
> which
> > > solves the problems that we have with Pulsar CI.
> > > We all know that it's urgent to fix Pulsar CI asap. We can do it
> > together.
> > >
> > > BR, Lari
> > >
> > >
> > > On Fri, Jan 29, 2021 at 11:51 AM Sijie Guo  wrote:
> > >
> > > > Lari,
> > > >
> > > > Thank you for bringing this proposal up! This is a great initiative.
> > > >
> > > > However, I agreed with Yong. We have spent tons of effort splitting
> one
> > > > large workflow into multiple smaller workflows.
> > > >
> > > > If this proposal is blocked by the other proposal, we should focus on
> > > > getting the changes for the other proposal before talking about
> merging
> > > > them.
> > > >
> > > > Thanks,
> > > > Sijie
> > > >
> > > > On Thu, Jan 28, 2021 at 9:55 PM Lari Hotari  wrote:
> > > >
> > > > > Thank you for the comments Penghui.
> > > > >
> > > > > Exactly what you said, we should make the tests stable.
> > > > > The proposals in the other draft PIP "Changes to flaky test
> handling"
> > > > deals
> > > > > with that.
> > > > > It's currently a draft and needs more eyes. Would you be able to
> > take a
> > > > > closer look at that too?
> > > > >
> > > > > BR, Lari
> > > > >
> > > > > On Fri, Jan 29, 2021 at 6:41 AM PengHui Li <
> codelipeng...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Currently, especially for the integration tests, a lot time to
> > build
> > > > > > pulsar distributions and docker images.
> > > > > > I think before merge tests we should to make the test stable,
> > > otherwise
> > > > > > rerun the test will become more expensive.
> > > > > >
> > > > > > Thanks,
> > > > > > Penghui
> > > > > > On Jan 29, 2021, 11:55 AM +0800, Yong 

Fixing flaky tests: help needed

[Lari Hotari]

Dear Pulsar community members,

In order to improve our CI, we will have to fix the flaky tests. In some
cases it might be necessary to replace an existing test with a redesigned
test.

The draft PIP "Changes to flaky test handling" document

lists
the top 10 flaky tests. A lot of them have already been address by pull
requests in the past week or so.

This is the list of recent PRs that fix flaky tests from the top 10 flaky
tests list:
https://github.com/apache/pulsar/pull/9286
https://github.com/apache/pulsar/pull/9243
https://github.com/apache/pulsar/pull/9258
https://github.com/apache/pulsar/pull/9356

These are the GH issues for the remaining ones in the top 10 flaky tests
list:
https://github.com/apache/pulsar/issues/6368
https://github.com/apache/pulsar/issues/9369
https://github.com/apache/pulsar/issues/9368

If you would like to help to fix flaky tests you can pick one of the open
issues above. Just add a comment on the issue when you start working on it
so that we can coordinate activities.

It is also helpful to report a flaky test when you encounter one. I've been
using this type of template for reporting a flaky test:
https://gist.github.com/lhotari/a5c67359b362b4f3d8729330d65a2298 . The
issues #9368 and #9369 have been reported using this template.
Search for the test name before reporting so that we don't end up with
duplicates.

The issues #6368, #9369 and #9368 are the 3 next important issues to fix.
I'm planning to create a more extensive list of the flaky failures so that
we can target the most flaky ones when we continue fixing the flaky tests.
I have some scripts in development to assist in mining the Pulsar Github
Action workflow run logs.

This is a search to find flaky issues in Pulsar GH issues:
https://github.com/apache/pulsar/issues?q=flaky+sort%3Aupdated-desc+is%3Aopen

Looking forward to the contributions for fixing flaky tests,

BR,

Lari

Re: Virtual Pulsar Community Meetings

[Rui Fu]

Agreed with @sijie’s proposal. Double events are more reasonable and friendly, 
also +1 for the recordings @YouTube. Look forward to attend the community 
meeting, cheers.


Best,

Rui Fu
在 2021年1月29日 +0800 PM4:53，Sijie Guo ，写道：
> That's a super great idea! Thank you for bringing this up!
>
> Given there are a lot of committers/contributors are from North America and
> Asia, I think we should pick up a better time that would be suitable for
> people from Asia (Japan and China).
> In the bookkeeper community, we used to run two events. One is to cover NA
> and Asia, and the other one is to cover NA and EU.
>
> I would suggest running biweekly meetings.
>
> - Tuesday 4 PM - 5 PM PST for NA and Asia
> - Thursday 8 AM - 9 AM PST for NA and EU
>
> All the events can be recorded and uploaded to Youtube. So people are able
> to watch the recordings as well.
>
> If people are good with this proposal, I am happy to set up and coordinate
> the meetings given I have run many meetings in the BookKeeper community
> before.
>
> We can get started next Tuesday and formalize the process as we go. Please
> vote for your idea.
>
> Thanks,
> Sijie
>
>
> On Fri, Jan 29, 2021 at 12:05 AM Enrico Olivelli 
> wrote:
>
> > Hello everyone,
> > in the BookKeeper community we used to have "Community Meetings" in order
> > to meet each other, discuss current works on the project, share knowledge
> > about current problems.
> >
> > What about having Community meetings for Pulsar ?
> > I would be happy to organize and to host the first meeting.
> >
> > In Apache we keep the decisions and the discussions on mailing lists, so
> > these meetings would be only to share information and we are not going to
> > make decisions.
> >
> > The Pulsar community is distributed all over the world, we have people from
> > China, the US, Europe, it will be hard to find a good time for everyone.
> > I suggest for the first meeting to meet at 8:30 PST
> >
> > For reference, this is the link to the "minutes" of the Community Meetings
> > in BK,
> > Sijie and Matteo will remember those days
> > https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
> >
> >
> > Enrico
> >

[DISCUSS] Apache pulsar-client-go v0.4.0 release

[xiaolong ran]

Hello all:

I hope you've all been doing well. Since the last v0.3.0 release,
almost two months and a half have passed. So I will start to release
v0.4.0 next week.

You can find the whole change list of the v0.4.0 release from:

https://github.com/apache/pulsar-client-go/pulls?q=is%3Aopen+is%3Apr+milestone%3A0.4.0

If I missed some features, please let me know. And if
some PRs can't complete in a few days, the owner can
help check if we should include them in v0.4.0 or push to
the next release version(v0.5.0).

--
Thanks
Xiaolong Ran

Re: Virtual Pulsar Community Meetings

[Enrico Olivelli]

Tuesday 2nd February 4pm PST works for me as first meeting date.

I am fine with having two meetings as well if this will help participation.

Regarding the YouTube recordings, I am afraid that this would scare people
that maybe would like to attend or tell their opinion but they don't feel
good in being recorded and published on social.

I would skip YouTube at least for the first times

Enrico




Il Ven 29 Gen 2021, 20:27 Rui Fu  ha scritto:

> Agreed with @sijie’s proposal. Double events are more reasonable and
> friendly, also +1 for the recordings @YouTube. Look forward to attend the
> community meeting, cheers.
>
>
> Best,
>
> Rui Fu
> 在 2021年1月29日 +0800 PM4:53，Sijie Guo ，写道：
> > That's a super great idea! Thank you for bringing this up!
> >
> > Given there are a lot of committers/contributors are from North America
> and
> > Asia, I think we should pick up a better time that would be suitable for
> > people from Asia (Japan and China).
> > In the bookkeeper community, we used to run two events. One is to cover
> NA
> > and Asia, and the other one is to cover NA and EU.
> >
> > I would suggest running biweekly meetings.
> >
> > - Tuesday 4 PM - 5 PM PST for NA and Asia
> > - Thursday 8 AM - 9 AM PST for NA and EU
> >
> > All the events can be recorded and uploaded to Youtube. So people are
> able
> > to watch the recordings as well.
> >
> > If people are good with this proposal, I am happy to set up and
> coordinate
> > the meetings given I have run many meetings in the BookKeeper community
> > before.
> >
> > We can get started next Tuesday and formalize the process as we go.
> Please
> > vote for your idea.
> >
> > Thanks,
> > Sijie
> >
> >
> > On Fri, Jan 29, 2021 at 12:05 AM Enrico Olivelli 
> > wrote:
> >
> > > Hello everyone,
> > > in the BookKeeper community we used to have "Community Meetings" in
> order
> > > to meet each other, discuss current works on the project, share
> knowledge
> > > about current problems.
> > >
> > > What about having Community meetings for Pulsar ?
> > > I would be happy to organize and to host the first meeting.
> > >
> > > In Apache we keep the decisions and the discussions on mailing lists,
> so
> > > these meetings would be only to share information and we are not going
> to
> > > make decisions.
> > >
> > > The Pulsar community is distributed all over the world, we have people
> from
> > > China, the US, Europe, it will be hard to find a good time for
> everyone.
> > > I suggest for the first meeting to meet at 8:30 PST
> > >
> > > For reference, this is the link to the "minutes" of the Community
> Meetings
> > > in BK,
> > > Sijie and Matteo will remember those days
> > >
> https://cwiki.apache.org/confluence/display/BOOKKEEPER/Community+Meetings
> > >
> > >
> > > Enrico
> > >
>