[DISCUSS] Apache Pulsar 2.8.2 Release

[linlin]

Hi all,

I hope you've all been doing well. It has been more than one month since we
released the Apache Pulsar 2.8.1. We have a lot of fixes already merged. So
I would like to start to prepare our next patch release in the next few
days.

You can find the whole change list of the 2.8.2 release from:
https://github.com/apache/pulsar/pulls?q=is%3Aopen+is%3Apr+label%3Arelease%2F2.8.2

If I missed some features, please let me know. And if some PRs can't be
completed in a few days, the owner can help check if we should include them
in 2.8.2 or push them to the next release version.

Thanks,
Lin Lin

Re: [DISCUSS] Apache Pulsar 2.8.2 Release

[PengHui Li]

+1

On Wed, Oct 27, 2021 at 4:22 PM linlin  wrote:

> Hi all,
>
> I hope you've all been doing well. It has been more than one month since we
> released the Apache Pulsar 2.8.1. We have a lot of fixes already merged. So
> I would like to start to prepare our next patch release in the next few
> days.
>
> You can find the whole change list of the 2.8.2 release from:
>
> https://github.com/apache/pulsar/pulls?q=is%3Aopen+is%3Apr+label%3Arelease%2F2.8.2
>
> If I missed some features, please let me know. And if some PRs can't be
> completed in a few days, the owner can help check if we should include them
> in 2.8.2 or push them to the next release version.
>
> Thanks,
> Lin Lin
>

Re: [DISCUSSION] PIP-108: Add method to help user judge if consumer queue has message

[PengHui Li]

@ZhangJian He, as Matteo mentioned, Use `consumer.receive(0,
TimeUnit.SECONDS)` can achieve the same purpose for checking if there are
messages in the local cache.

Thanks
Penghui

On Tue, Oct 26, 2021 at 2:35 PM ZhangJian He  wrote:

> If some users need the message content to do user-defined actions, we need
> to ensure the user can't use the `peekMessage` to do things like ack
> because the message are still in the blockingQueue, return just a
> content-copy?
>
> introduced `localBuffer` might be good ? `hasMessagesInLocalBuffer`
>
> JiangHaiting  于2021年10月26日周二 下午2:20写道：
>
> > I'm totally +1 for the feature to check if we can get
> > message immediately from consumer, this is to say we have message
> > locally.
> >
> >
> > In my understanding, it's useful to implement some user-defined order to
> > consume messages among different topics, in your case, the "distributed
> > flow control ability".
> > But in the past few years, I've met some users have defined the consume
> > order of different topics by part of the message content, like
> > some critical property value. 
> > In these situations, a `peek` method is more suitable.
> >
> >
> > Further more, peek is not effectively equals to `consumer.receive(0,
> > TimeUnit.SECONDS)`. As you will have to store the message somewhere else
> if
> > you find that it's not the most priority message to process.
> >
> >
> > One last thing, put the concept of "receiverQueue" in the api of
> consumer,
> > seems a little bit strange, IMHO.
> >
> >
> >
> >
> > -- Original --
> > From:
> >   "dev"
> > <
> > shoot...@gmail.com>;
> > Date: Tue, Oct 26, 2021 12:54 PM
> > To: "dev" >
> > Subject: Re: [DISCUSSION] PIP-108: Add method to help user judge if
> > consumer queue has message
> >
> >
> >
> > 3. Our solution implements the distributed flow control ability at client
> > side, so we don't use the listener way.
> > 2. Per customer per consumer in different tenants and namespace, and the
> > `flow-control` need(Some of our customer's machines can't work on high
> > traffic), So `Multi-topic` can't use.
> > 1. We want to use this api to judge if there's messages to receive, like
> > that pseudo code
> > if (consumer.hasMessage()) {
> >  .submit(() -> {
> >  consumer.pollMessagesAccordingToTheDistributedFlowControl()
> >  })
> > }
> >
> > Matteo Merli  >
> > > I'm a bit hesitant about this because I think there are already at
> > > least 3 different ways to handle similar scenarios.
> > >
> > >  1. Using listener and avoid calling receive directly
> > >  2. Use multi-topic consumer, so there's a single `Consumer`
> > instance
> > > exposed
> > >  3. Use `consumer.receive(0, TimeUnit.SECONDS)` to probe for
> > message
> > >
> > >
> > > --
> > > Matteo Merli
> > >  > >
> > > On Mon, Oct 25, 2021 at 7:34 PM ZhangJian He  >
> > wrote:
> > > >
> > > > I think it's better to add the method to Consumer interface
> > instead of
> > > let
> > > > user casting it to `ConsumerBase`.
> > > > `peek` is most complexly,  for the reason, I can use the
> > `peek` object to
> > > > ack、negative ack, but when to remove from the `BlockingQueue`?
> > > > IMHO, people use this api are just to judge if has the message,
> > > otherwise,
> > > > they can just use `receive(0,TimeUnit)
> > > >
> > > > JiangHaiting  > 上午10:19写道：
> > > >
> > > > > Can this method
> > > > >
> > "org.apache.pulsar.client.impl.ConsumerBase#getTotalIncomingMessages"
> > > do
> > > > > the trick? Though you have to change the type to
> > ConsumerBase.
> > > > >
> > > > >
> > > > > And maybe `peek` is more suitable and useful to add to the
> > Consumer
> > > > > interface？
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > -- Original --
> > > > > From:
> > > >
> >
> >  
> > "dev"
> > > >
> >
> >
> > <
> > > > > shoot...@gmail.com>;
> > > > > Date: Mon, Oct 25, 2021 07:24 PM
> > > > > To: "dev" > > > >
> > > > > Subject: [DISCUSSION] PIP-108: Add method to help
> > user judge if
> > > > > consumer queue has message
> > > > >
> > > > >
> > > > >
> > > > > https://github.com/apache/pulsar/issues/12479
> > > > >
> > > > > --- Pasted here for quoting convenience ---
> > > > >
> > > > > ## Motivation
> > > > > Currently, I have an application that manages ten thousand
> > of
> > > consumers,
> > > > > and a logic to schedule consumers's receive. It would be
> > helpful to
> > > know if
> > > > > one of the consumers have message to recive.
> > > > >
> > > > > ## Goal
> > > > > To make `Consumer` can judge if there are unreceiving
> > messages
> > > > >
> > > > > ## API Changes
> > > > >
> > > > > Add `hasMessageInReceiverQueue` on the `Consumer`
> interface.
> > > > >
> > > > > ## Implementati

Re: [DISCUSSION] PIP-108: Add method to help user judge if consumer queue has message

[ZhangJian He]

Some users want to use this api to judge if there's messages to receive,
like that pseudo code:
if (consumer.hasMessage()) {
   .submit(() -> {
  consumer.pollMessagesAccordingToTheDistributedFlowControl()
   })
}

don't want to remove the message from queue.

PengHui Li  于2021年10月27日周三 下午7:43写道：

> @ZhangJian He, as Matteo mentioned, Use `consumer.receive(0,
> TimeUnit.SECONDS)` can achieve the same purpose for checking if there are
> messages in the local cache.
>
> Thanks
> Penghui
>
> On Tue, Oct 26, 2021 at 2:35 PM ZhangJian He  wrote:
>
> > If some users need the message content to do user-defined actions, we
> need
> > to ensure the user can't use the `peekMessage` to do things like ack
> > because the message are still in the blockingQueue, return just a
> > content-copy?
> >
> > introduced `localBuffer` might be good ? `hasMessagesInLocalBuffer`
> >
> > JiangHaiting  于2021年10月26日周二 下午2:20写道：
> >
> > > I'm totally +1 for the feature to check if we can get
> > > message immediately from consumer, this is to say we have message
> > > locally.
> > >
> > >
> > > In my understanding, it's useful to implement some user-defined order
> to
> > > consume messages among different topics, in your case, the "distributed
> > > flow control ability".
> > > But in the past few years, I've met some users have defined the consume
> > > order of different topics by part of the message content, like
> > > some critical property value. 
> > > In these situations, a `peek` method is more suitable.
> > >
> > >
> > > Further more, peek is not effectively equals to `consumer.receive(0,
> > > TimeUnit.SECONDS)`. As you will have to store the message somewhere
> else
> > if
> > > you find that it's not the most priority message to process.
> > >
> > >
> > > One last thing, put the concept of "receiverQueue" in the api of
> > consumer,
> > > seems a little bit strange, IMHO.
> > >
> > >
> > >
> > >
> > > -- Original --
> > > From:
> > >   "dev"
> > > <
> > > shoot...@gmail.com>;
> > > Date: Tue, Oct 26, 2021 12:54 PM
> > > To: "dev" > >
> > > Subject: Re: [DISCUSSION] PIP-108: Add method to help user judge
> if
> > > consumer queue has message
> > >
> > >
> > >
> > > 3. Our solution implements the distributed flow control ability at
> client
> > > side, so we don't use the listener way.
> > > 2. Per customer per consumer in different tenants and namespace, and
> the
> > > `flow-control` need(Some of our customer's machines can't work on high
> > > traffic), So `Multi-topic` can't use.
> > > 1. We want to use this api to judge if there's messages to receive,
> like
> > > that pseudo code
> > > if (consumer.hasMessage()) {
> > >  .submit(() -> {
> > >  consumer.pollMessagesAccordingToTheDistributedFlowControl()
> > >  })
> > > }
> > >
> > > Matteo Merli  > >
> > > > I'm a bit hesitant about this because I think there are already at
> > > > least 3 different ways to handle similar scenarios.
> > > >
> > > >  1. Using listener and avoid calling receive directly
> > > >  2. Use multi-topic consumer, so there's a single `Consumer`
> > > instance
> > > > exposed
> > > >  3. Use `consumer.receive(0, TimeUnit.SECONDS)` to probe for
> > > message
> > > >
> > > >
> > > > --
> > > > Matteo Merli
> > > >  > > >
> > > > On Mon, Oct 25, 2021 at 7:34 PM ZhangJian He  > >
> > > wrote:
> > > > >
> > > > > I think it's better to add the method to Consumer interface
> > > instead of
> > > > let
> > > > > user casting it to `ConsumerBase`.
> > > > > `peek` is most complexly,  for the reason, I can use the
> > > `peek` object to
> > > > > ack、negative ack, but when to remove from the
> `BlockingQueue`?
> > > > > IMHO, people use this api are just to judge if has the
> message,
> > > > otherwise,
> > > > > they can just use `receive(0,TimeUnit)
> > > > >
> > > > > JiangHaiting  > > 上午10:19写道：
> > > > >
> > > > > > Can this method
> > > > > >
> > > "org.apache.pulsar.client.impl.ConsumerBase#getTotalIncomingMessages"
> > > > do
> > > > > > the trick? Though you have to change the type to
> > > ConsumerBase.
> > > > > >
> > > > > >
> > > > > > And maybe `peek` is more suitable and useful to add to
> the
> > > Consumer
> > > > > > interface？
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > -- Original --
> > > > > > From:
> > > > >
> > >
> >
> >  
> > > "dev"
> > > > >
> > >
> >
> >
> > > <
> > > > > > shoot...@gmail.com>;
> > > > > > Date: Mon, Oct 25, 2021 07:24 PM
> > > > > > To: "dev" > > > > >
> > > > > > Subject: [DISCUSSION] PIP-108: Add method to
> help
> > > user judge if
> > > > > > consumer queue has message
> > > > > >
> > > > > >
> > > > > >
> > > > > > https://github.com/apache/puls

Re: [DISCUSS] Apache Pulsar 2.8.2 Release

[Enrico Olivelli]

+1

Enrico

Il giorno mer 27 ott 2021 alle ore 13:39 PengHui Li  ha
scritto:

> +1
>
> On Wed, Oct 27, 2021 at 4:22 PM linlin  wrote:
>
> > Hi all,
> >
> > I hope you've all been doing well. It has been more than one month since
> we
> > released the Apache Pulsar 2.8.1. We have a lot of fixes already merged.
> So
> > I would like to start to prepare our next patch release in the next few
> > days.
> >
> > You can find the whole change list of the 2.8.2 release from:
> >
> >
> https://github.com/apache/pulsar/pulls?q=is%3Aopen+is%3Apr+label%3Arelease%2F2.8.2
> >
> > If I missed some features, please let me know. And if some PRs can't be
> > completed in a few days, the owner can help check if we should include
> them
> > in 2.8.2 or push them to the next release version.
> >
> > Thanks,
> > Lin Lin
> >
>

Re: [DISCUSS] Apache Pulsar 2.8.2 Release

[r...@apache.org]

LGTM +1

--
Thanks
Xiaolong Ran

Enrico Olivelli  于2021年10月27日周三 下午8:34写道：

> +1
>
> Enrico
>
> Il giorno mer 27 ott 2021 alle ore 13:39 PengHui Li 
> ha
> scritto:
>
> > +1
> >
> > On Wed, Oct 27, 2021 at 4:22 PM linlin  wrote:
> >
> > > Hi all,
> > >
> > > I hope you've all been doing well. It has been more than one month
> since
> > we
> > > released the Apache Pulsar 2.8.1. We have a lot of fixes already
> merged.
> > So
> > > I would like to start to prepare our next patch release in the next few
> > > days.
> > >
> > > You can find the whole change list of the 2.8.2 release from:
> > >
> > >
> >
> https://github.com/apache/pulsar/pulls?q=is%3Aopen+is%3Apr+label%3Arelease%2F2.8.2
> > >
> > > If I missed some features, please let me know. And if some PRs can't be
> > > completed in a few days, the owner can help check if we should include
> > them
> > > in 2.8.2 or push them to the next release version.
> > >
> > > Thanks,
> > > Lin Lin
> > >
> >
>

Re: [DISCUSS] Apache Pulsar 2.8.2 Release

[Michael Marshall]

+1

Michael

> On Oct 27, 2021, at 7:40 AM, ranxiaolong...@gmail.com wrote:
> 
> LGTM +1
> 
> --
> Thanks
> Xiaolong Ran
> 
> Enrico Olivelli  于2021年10月27日周三 下午8:34写道：
> 
>> +1
>> 
>> Enrico
>> 
>>> Il giorno mer 27 ott 2021 alle ore 13:39 PengHui Li 
>>> ha
>>> scritto:
>>> 
>>> +1
>>> 
 On Wed, Oct 27, 2021 at 4:22 PM linlin  wrote:
>>> 
 Hi all,
 
 I hope you've all been doing well. It has been more than one month
>> since
>>> we
 released the Apache Pulsar 2.8.1. We have a lot of fixes already
>> merged.
>>> So
 I would like to start to prepare our next patch release in the next few
 days.
 
 You can find the whole change list of the 2.8.2 release from:
 
 
>>> 
>> https://github.com/apache/pulsar/pulls?q=is%3Aopen+is%3Apr+label%3Arelease%2F2.8.2
 
 If I missed some features, please let me know. And if some PRs can't be
 completed in a few days, the owner can help check if we should include
>>> them
 in 2.8.2 or push them to the next release version.
 
 Thanks,
 Lin Lin
 
>>> 
>> 

Re: [DISCUSS] Apache Pulsar 2.8.2 Release

[Hang Chen]

+1

Thanks,
Hang

Michael Marshall  于2021年10月27日周三 下午9:41写道：
>
> +1
>
> Michael
>
> > On Oct 27, 2021, at 7:40 AM, ranxiaolong...@gmail.com wrote:
> >
> > LGTM +1
> >
> > --
> > Thanks
> > Xiaolong Ran
> >
> > Enrico Olivelli  于2021年10月27日周三 下午8:34写道：
> >
> >> +1
> >>
> >> Enrico
> >>
> >>> Il giorno mer 27 ott 2021 alle ore 13:39 PengHui Li 
> >>> ha
> >>> scritto:
> >>>
> >>> +1
> >>>
>  On Wed, Oct 27, 2021 at 4:22 PM linlin  wrote:
> >>>
>  Hi all,
> 
>  I hope you've all been doing well. It has been more than one month
> >> since
> >>> we
>  released the Apache Pulsar 2.8.1. We have a lot of fixes already
> >> merged.
> >>> So
>  I would like to start to prepare our next patch release in the next few
>  days.
> 
>  You can find the whole change list of the 2.8.2 release from:
> 
> 
> >>>
> >> https://github.com/apache/pulsar/pulls?q=is%3Aopen+is%3Apr+label%3Arelease%2F2.8.2
> 
>  If I missed some features, please let me know. And if some PRs can't be
>  completed in a few days, the owner can help check if we should include
> >>> them
>  in 2.8.2 or push them to the next release version.
> 
>  Thanks,
>  Lin Lin
> 
> >>>
> >>

[Workflow] Doc Bot Runs Successfully!

[Anonymitaet _]

Hi Pulsar enthusiasts,



Here comes exciting news!



As you may notice, a doc Bot has been working in the Pulsar community for a 
while. It improves the efficiency of developing docs by automatically labeling 
PRs with doc info or reminding you to provide doc-related info, which reduces 
much manual triage work.



These days, the doc Bot still has some issues, but we're trying our best to 
solve them. Generally, the doc Bot works smoothly. We're so glad to see that!



Besides, we've documented the doc Bot info here: [PIP 106] Introduce Bot to 
Improve Efficiency of Developing 
Docs.
 It contains every detail about the doc Bot, including the implementation 
process, discussion, etc. You can take it for reference if you need to achieve 
similar features. And feel free to comment if you have any suggestions, thanks!



P.S. Special thanks to Tao Feng and Enrico for making this happen!

Thanks to Yong Zhang, Ting Yuan, Peng Hui, for your valuable suggestions!



Anonymitaet

Re: Revote: Pulsar website concepts

[Melissa Logan]

Thanks, Mattison, noted.

Guangning, agreed, the intent is to continue using Docusaurus for docs
with their Theming functionality:
https://docusaurus.io/docs/using-themes#themes-design.



On Sun, Oct 24, 2021 at 6:13 AM Guangning E  wrote:
>
> The design looks good, the current design seems to have only a few home
> pages, how will it be implemented for documentation and how do you plan to
> integrate these pages with the current documentation pages?
> The current pulsar site is based on the docusaurus https://docusaurus.io/
> framework and we are currently upgrading the site to a new version.
> docusaurus is an excellent framework for presenting documentation, so my
> recommendation is to continue using docusaurus as a framework that
> continues to reuse the previous build process and reduces the learning cost
> for users.
>
> mattison chao  于2021年10月23日周六 下午2:02写道：
>
> > +1 for Option 1, but I think we need to increase the background colour
> > contrast.
> >



--
Melissa Logan (she/her)
Principal, Constantia.io
meli...@constantia.io
Cell: 503-317-8498
LinkedIn | Twitter

[GitHub] [pulsar-client-node] kimula edited a comment on issue #131: more than 3 topics blocks async operations

[GitBox]

kimula edited a comment on issue #131:
URL: 
https://github.com/apache/pulsar-client-node/issues/131#issuecomment-720263581


   @steelone 
   i succeeded to run your example and reproduce the problem (it was due to not 
your example but my environment).
   i think that using `listener` instead of `while` loop makes it work.
   like:
   ```javascript
   const consumer = await client.subscribe({
   topic,
   subscription: 'sub1',
   subscriptionType,
   listener: (msg, consumer) => {
   try {
   consumer.acknowledge(msg);
   onMessage(msg.getData().toString());
   } catch (e) {
   consumer.close();
   onError(e);
   }
   }
   });
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org