Re: [VOTE] Pulsar Node.js Client Release 1.8.2 Candidate 4

2023-04-18 Thread Yunze Xu 
+1 (binding)

* Checked signatures and checksums
* Build from source and run e2e examples (Ubuntu 20.04 and Node.js v16.19.0)
*  Install from NPM and run e2e examples with OAuth2 authentication
through StreamNative cloud, including the `node:19-bullseye` and
`node:19-alpine3.16` containers

Thanks,
Yunze

On Mon, Apr 17, 2023 at 11:30 PM Yuto Furuta  wrote:
>
> +1 (non-binding)
>
> * verify checksum and signatures
> * build the source
> * run producer, consumer, consumer_listener (source/npm)
>
> Regards,
>
> Yuto Furuta
> 
> 差出人: Baodi Shi 
> 送信日時: 2023年4月14日 18:12
> 宛先: dev@pulsar.apache.org 
> 件名: [VOTE] Pulsar Node.js Client Release 1.8.2 Candidate 4
>
> Hi everyone,
>
> This is the first release candidate for Apache Pulsar Node.js client,
> version 1.8.2.
>
> It fixes the following issues:
> https://github.com/apache/pulsar-client-node/pulls?q=is%3Apr+label%3Arelease%2Fv1.8.2+is%3Aclosed
>
> Please download the source files and review this release candidate:
> - Download the source package, verify shasum and asc
> - Follow the README.md to build and run the Pulsar Node.js client.
>
> The release candidate package has been published to the npm registry:
>
> https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.npmjs.com%2Fpackage%2Fpulsar-client%2Fv%2F1.8.2-rc.4&data=05%7C01%7Cyfuruta%40yahoo-corp.jp%7Ca14b3d60007641e8d58f08db3cc86e9d%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C638170603764950233%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=bKG26QUm5gWo1CuJc3AHOcHgMb1KWPkjiAkYeM1VHWA%3D&reserved=0
>
> You can install it by `npm i pulsar-client@1.8.2-rc.4
> --pulsar_binary_host_mirror=
> https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fpulsar%2Fpulsar-client-node%2F%2560&data=05%7C01%7Cyfuruta%40yahoo-corp.jp%7Ca14b3d60007641e8d58f08db3cc86e9d%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C638170603764950233%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=m%2Fhe8LW3RSogO%2FnPl%2FAzik217g%2FPAH0cBneu%2FXFTuxw%3D&reserved=0
> and verify the package.
>
> You can refer to this repository to verify tls related features:
>
> https://github.com/shibd/pulsar-client-tls-test
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Source files:
> https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fpulsar%2Fpulsar-client-node%2Fpulsar-client-node-1.8.2-rc.4%2F&data=05%7C01%7Cyfuruta%40yahoo-corp.jp%7Ca14b3d60007641e8d58f08db3cc86e9d%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C638170603764950233%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=TUQ7OMPmRmeXJWD%2Fty53ForqI9Y6JMUF6VogHew7mHo%3D&reserved=0
>
> Pulsar's KEYS file containing PGP keys we use to sign the release:
> https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2Fpulsar%2FKEYS&data=05%7C01%7Cyfuruta%40yahoo-corp.jp%7Ca14b3d60007641e8d58f08db3cc86e9d%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C638170603764950233%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=K8gGLEtq%2BLYSYC6kYdseBEMmGzJgSdEzNBUmx5C%2BM%2Fg%3D&reserved=0
>
> SHA-512 checksum:
> 2bcfd0155abe653924d974839307c2bc980c982252e83c6442b1a368262057d4401eef186ec2b962ec973deedd81a899469ecee6afc61bd5718fa548c7c510ad
>  ./apache-pulsar-client-node-1.8.2.tar.gz
>
> The tag to be voted upon:
> v1.8.2-rc.4(7b3de1f)
> https://github.com/apache/pulsar-client-node/releases/tag/v1.8.2-rc.4
>
> Please review and vote on the release candidate #1 for the version
> 1.8.2, as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
>
> Thanks,
> Baodi Shi

Re: [DISCUSS] Sorting out pulsar's internal thread pools

2023-04-18 Thread Zike Yang 
Sounds great. +1

Thanks,
Zike Yang

On Tue, Apr 18, 2023 at 2:37 PM Xiangying Meng  wrote:
>
> Thank you for bringing up this important topic. I completely agree with
> this initiative.
> This would be a great starting point for revisiting and improving the
> Pulsar codebase.
>
> Thanks,
> Xiangying
>
> On Tue, Apr 18, 2023 at 2:18 PM Lin Lin  wrote:
>
> > This is a good idea.
> >
> > Thanks,
> > Lin Lin
> >
> > On 2023/04/18 02:07:55 mattisonc...@gmail.com wrote:
> > >
> > > Hello, folks.
> > >
> > > I would like to start discussing the pulsar internal thread pool sorting
> > out.
> > >
> > > How did I get this idea?
> > >
> > > Recently, we met some problems with the BK operation timeout. After
> > investigating, we found an issue that is we share the IO
> > executor(workgroup) with the Bookkeeper client and internal client and do
> > some other async task in the dispatcher or somewhere to avoid deadlock.
> > >
> > > But the problem over here. If we use this executor to do some kind of
> > `blocking`(or spend much time computing. e.g. reply to many delayed
> > messages) operation, it will block BK clients from sending requests if they
> > are using the same thread.
> > >
> > > And then, I checked all the usage of the thread pool. We need the rule
> > to constrain what thread pool we should use.
> > >
> > > What am I expecting?
> > >
> > > I want to collect all the thread pools and define a clear usage guide to
> > avoid wrong use and improve the fault tolerance(the component problem
> > shouldn't affect the whole broker)
> > >
> > >
> > >
> > > I need to hear your guy's opinions. Please feel free to leave any
> > questions. Thanks!
> > >
> > >
> > > Best,
> > > Mattison
> > >
> > >
> > >
> >

Re: [DISCUSS] Sorting out pulsar's internal thread pools

2023-04-18 Thread Cong Zhao 
I think this is a good idea for the reasonable use of pulsar threads.

Thanks,
Cong Zhao

On 2023/04/18 02:07:55 mattisonc...@gmail.com wrote:
> 
> Hello, folks.
> 
> I would like to start discussing the pulsar internal thread pool sorting out.
> 
> How did I get this idea?
> 
> Recently, we met some problems with the BK operation timeout. After 
> investigating, we found an issue that is we share the IO executor(workgroup) 
> with the Bookkeeper client and internal client and do some other async task 
> in the dispatcher or somewhere to avoid deadlock.
> 
> But the problem over here. If we use this executor to do some kind of 
> `blocking`(or spend much time computing. e.g. reply to many delayed messages) 
> operation, it will block BK clients from sending requests if they are using 
> the same thread.
> 
> And then, I checked all the usage of the thread pool. We need the rule to 
> constrain what thread pool we should use.
> 
> What am I expecting?
> 
> I want to collect all the thread pools and define a clear usage guide to 
> avoid wrong use and improve the fault tolerance(the component problem 
> shouldn't affect the whole broker)
> 
> 
> 
> I need to hear your guy's opinions. Please feel free to leave any questions. 
> Thanks!
> 
> 
> Best,
> Mattison
> 
> 
>

Re: [DISCUSS] Add checklist for PMC binding vote of PIP

2023-04-18 Thread Asaf Mesika 
The problem I'm trying to solve is: lack of ability to understand PIPs.
PIPs I had the chance of reading lack:
* Background information: It should contain all background information
necessary to understand the problem and the solution
* Clarity: It should be written in a coherent and easy to understand way.

I thought this could improve using 2 ways:
1. Define a clear template for PIPs - this should solve all the missing
information. This is in progress.
2. Provide a checklist to verify the +1 voter check those 3 things:
background information, clarity, solid technical solution.

Both Enrico and Yunze say, if I understand correctly, that the +1 voter
checks those 3 things implicitly.
Yet when I try to learn Pulsar by reading historical PIPs, I find some
lacking on those things (clarity, background information) making it super
hard for me to get onboard into Pulsar.

Another aspect worth noting is: community increase. In my own opinion,
documents with clarity and enough background information produce a feeling
of quality - high quality. Making Pulsar PIPs clear and have all
information to understand them will help grow Pulsar adoption.

Maybe incremental improvements are better.. If I understand correctly, both
Enrico and Yunze - you are ok with having a summary template, but have it
non-required?

Enrico - Regarding previous suggestions. Root cause - help make Pulsar
better from my own perspective. Some suggestions may be super bad
suggestions and hopefully some will be good :)
This specific one - I validated with the PMC members in the weekly zoom
meeting roughly 3 weeks ago, and got +1 across the board (we had 5 people).
I did it since I felt it was a touchy subject.

Thanks,

Asaf






On Tue, Apr 18, 2023 at 9:15 AM Yunze Xu 
wrote:

> Basically I think describing how much work the reviewer did to give
> his +1 is good. Just like the vote for a release, each +1 follows with
> the verifications he did, e.g. here [1] is a vote for Pulsar 2.11.1
> candidate 1:
>
> > • Built from the source package (maven 3.8.6 OpenJDK 17.0)
> > • Ran binary package standalone with pub/sub
> > ...
>
> But I don't think forcing the rule is good. The proposal could
> sometimes be not so complicated. From my personal experience,
> sometimes I vote my +1 just because I think it's good and there is no
> serious problem. If you want me to vote again with the checklist, I
> might still not have an idea of what I should write, unless there is a
> template and I filled the template. Only if the proposal is somehow
> complicated will the checklist be meaningful, like the PIP-192, which
> is a very complicated proposal.
>
> > Moreover, this checklist can ensure that all participants have
> thoroughly reviewed the PIP,
>
> Regarding this point from Xiangying, I want to repeat a similar
> thought [2] for the previous discussion.
>
> IF ANYONE WANT, HE CAN STILL COPY A CHECKLIST FROM OTHERS AND JUST
> PERFORM SOME SLIGHTLY CHANGES.
>
> Forcing a checklist won't change anything if there is a PMC that gave
> his vote without any careful review. It just makes the rule more
> complicated. If you don't trust a PMC, no rule could restrict him.
> Rules only make him a better game player.
>
> In addition, when a reviewer approves a PR, should he add a checklist
> as well, instead of a simple LGTM or +1? Huge PRs appear more often
> than complicated proposals.
>
> In conclusion, I am +0 to this suggestion. If this suggestion is
> passed, I will follow it well. But if I cannot think of a checklist
> with a proposal, I will try to be a good vote game player.
>
> [1] https://lists.apache.org/thread/13xmt4jdwmlo1mo5dhkxlg9pnkfdwjjj
> [2] https://lists.apache.org/thread/o0vw1dfoo84pscfd46gdm3sm9mvovmr2
>
> Thanks,
> Yunze
>
> On Mon, Apr 17, 2023 at 3:48 PM PengHui Li  wrote:
> >
> > I don't think it will bring more burden on reviewers.
> > It will only provide a checklist for reviewers before
> > you vote +1 or -1. It could be done in 1 minute if you
> > did a great proposal review. Of course, if you are
> > missing some aspects that should be reviewed,
> > This will make the reviewer spend more time reviewing
> > the missing items, but it is valuable.
> >
> > I don't think this proposal is accusing PMCs, but PMCs
> > might also miss some items. The checklist can help PMCs
> > to avoid missing items. Actually, I think every PMC has
> > checklist for a proposal review. It might be recorded in
> > a tiny notebook, or in his brain. Now, the proposal provides
> > a way to share your experience of proposal review.
> >
> > And we are actually doing the same thing in the voting of
> > release. Everyone will provide a list of what they have
> > verified with +1 or -1.
> >
> > Regards,
> > Penghui
> >
> >
> > On Mon, Apr 17, 2023 at 10:37 AM Xiangying Meng 
> > wrote:
> >
> > > Hi, Asaf
> > > This is a great suggestion. I believe one significant advantage is that
> > > it can help newcomers better understand the voting process and how
> > > decisions are ma

Re: [VOTE] Pulsar Release 2.11.1 Candidate-2

2023-04-18 Thread Hang Chen 
+1 (binding)

- Checksum and signatures
- Built from sources on MacOS (JDK 17 and Maven 3.8.6)
- Run rat check and check-binary-license on the source.
- Setup Pulsar cluster with one zookeeper node, one bookie node, and
one broker node
- Checked the Grafana dashboard metrics
- Run Pulsar perf produce and consume

Thanks,
Hang

PengHui Li  于2023年4月18日周二 11:27写道：
>
> Ah, yes, it works after setting `PULSAR_STANDALONE_USE_ZOOKEEPER=1`
> We need to push a PR to fix the validation document.
>
> +1 (binding)
>
> Thanks,
> Penghui
>
> On Mon, Apr 17, 2023 at 11:14 PM Haiting Jiang 
> wrote:
>
> > Hi Penghui
> >
> > Did you set `PULSAR_STANDALONE_USE_ZOOKEEPER=1` when verifying
> > Stateful Functions ?
> >
> > Thanks,
> > Haiting
> >
> > On Mon, Apr 17, 2023 at 7:48 PM PengHui Li  wrote:
> > >
> > > The RC version failed to verify the state function
> > >
> > > The output:
> > >
> > > ```
> > > ~/Downloads/release_2.11.1/apache-pulsar-2.11.1 » bin/pulsar-admin
> > > functions status --tenant test --namespace test-namespace --name
> > word_count
> > > {
> > >   "numInstances" : 1,
> > >   "numRunning" : 1,
> > >   "instances" : [ {
> > > "instanceId" : 0,
> > > "status" : {
> > >   "running" : true,
> > >   "error" : "",
> > >   "numRestarts" : 0,
> > >   "numReceived" : 15,
> > >   "numSuccessfullyProcessed" : 0,
> > >   "numUserExceptions" : 15,
> > >   "latestUserExceptions" : [ {
> > > "exceptionString" : "State test/test-namespace/word_count is not
> > > enabled.",
> > > "timestampMs" : 1681731596735
> > >   }, {
> > > "exceptionString" : "State test/test-namespace/word_count is not
> > > enabled.",
> > > "timestampMs" : 1681731596736
> > >   }, {
> > > "exceptionString" : "State test/test-namespace/word_count is not
> > > enabled.",
> > > "timestampMs" : 1681731596737
> > >   }, {
> > > "exceptionString" : "State test/test-namespace/word_count is not
> > > enabled.",
> > > "timestampMs" : 1681731596738
> > >   }, {
> > > "exceptionString" : "State test/test-namespace/word_count is not
> > > enabled.",
> > > "timestampMs" : 1681731596738
> > >   }, {
> > > "exceptionString" : "State test/test-namespace/word_count is not
> > > enabled.",
> > > "timestampMs" : 1681731656845
> > >   }, {
> > > "exceptionString" : "State test/test-namespace/word_count is not
> > > enabled.",
> > > "timestampMs" : 1681731656845
> > >   }, {
> > > "exceptionString" : "State test/test-namespace/word_count is not
> > > enabled.",
> > > "timestampMs" : 1681731656846
> > >   }, {
> > > "exceptionString" : "State test/test-namespace/word_count is not
> > > enabled.",
> > > "timestampMs" : 1681731656847
> > >   }, {
> > > "exceptionString" : "State test/test-namespace/word_count is not
> > > enabled.",
> > > "timestampMs" : 1681731656847
> > >   } ],
> > >   "numSystemExceptions" : 0,
> > >   "latestSystemExceptions" : [ ],
> > >   "averageLatency" : 0.146536533,
> > >   "lastInvocationTime" : 1681731656847,
> > >   "workerId" : "c-standalone-fw-localhost-8080"
> > > }
> > >   } ]
> > > }
> > >
> > -
> > > ~/Downloads/release_2.11.1/apache-pulsar-2.11.1 » bin/pulsar-admin
> > > functions querystate --tenant test --namespace test-namespace --name
> > > word_count -k hello -w
> > > # key 'hello' doesn't exist.
> > > # key 'hello' doesn't exist.
> > > # key 'hello' doesn't exist
> > > State storage client is not done initializing. Please try again in a
> > little
> > > while.
> > >
> > > Reason: State storage client is not done initializing. Please try again
> > in
> > > a little while.
> > > ```
> > >
> > > The logs from broker:
> > >
> > > ```
> > > 2023-04-17T19:42:55,447+0800 [pulsar-web-48-15] INFO
> > >  org.eclipse.jetty.server.RequestLog - 127.0.0.1 - -
> > [17/Apr/2023:19:42:55
> > > +0800] "GET /admin/v2/namespaces/test HTTP/1.1" 200 23 "-"
> > > "Pulsar-Java-v2.11.1" 5
> > > 2023-04-17T19:42:55,450+0800 [pulsar-web-48-4] INFO
> > >  org.apache.pulsar.common.nar.NarUnpacker - Created directory
> > >
> > /var/folders/t4/w6thygwj0y596w0j5h_pzlw4gn/T/pulsar-nar/functions4142072620718955215.tmp-unpacked
> > > 2023-04-17T19:42:55,451+0800 [pulsar-web-48-4] INFO
> > >  org.apache.pulsar.common.nar.NarUnpacker - Extracting
> > >
> > /var/folders/t4/w6thygwj0y596w0j5h_pzlw4gn/T/functions4142072620718955215.tmp
> > > to
> > >
> > /var/folders/t4/w6thygwj0y596w0j5h_pzlw4gn/T/pulsar-nar/functions4142072620718955215.tmp-unpacked/3E3BMXidvkyl6VKTjb2b4Q
> > > 2023-04-17T19:42:55,452+0800 [pulsar-web-48-4] ERROR
> > > org.apache.pulsar.common.nar.NarUnpacker - There was a problem extracting
> > > the na

Re: [VOTE] Pulsar Release 2.10.4 Candidate 4

2023-04-18 Thread Hang Chen 
+1(binding)

- Checksum and signatures
- Built from sources on MacOS (JDK 11 and Maven 3.8.6)
- Run rat check and check-binary-license on the source.
- Setup Pulsar cluster with one zookeeper node, one bookie node, and
one broker node
- Checked the Grafana dashboard metrics
- Run Pulsar perf produce and consume
- Run HDFS-based tiered storage offload and consume
- Triggered the Pulsar CI [1], and all the tests passed


Regards,
Hang

[1] https://github.com/hangc0276/pulsar/pull/19

guo jiwei  于2023年4月18日周二 11:15写道：
>
> +1 (binding)
>
> - Check the signature
> - Build from the source package
> - Start the standalone
> - Validate Pub/Sub and Java Functions
> - Validate Stateful Functions
>
> Regards
> Jiwei Guo (Tboy)
>
> On Mon, Apr 17, 2023 at 8:52 PM PengHui Li  wrote:
> >
> > +1 (binding)
> >
> > - Checked the signature
> > - Build from the source package
> > - Start standalone
> > - Checked cassandra connector
> > - Checked state function
> >
> > Regards,
> > Penghui
> >
> > On Sat, Apr 15, 2023 at 5:37 PM  wrote:
> >
> > > +1 (Binding)
> > >
> > >  • Built from the source package (maven 3.8.6 OpenJDK 11)
> > >  • Ran binary package standalone with pub/sub
> > >  • Ran docker image(pulsar-all) standalone with pub/sub
> > >  • Ran License check
> > >
> > > Best,
> > > Mattison
> > > On Apr 12, 2023, 17:10 +0800, Xiangying Meng ,
> > > wrote:
> > > > This is the fourth release candidate for Apache Pulsar, version 2.10.4.
> > > >
> > > > This release contains 126 commits by 37 contributors.
> > > > https://github.com/apache/pulsar/compare/v2.10.3...v2.10.4-candidate-4
> > > >
> > > > *** Please download, test and vote on this release. This vote will stay
> > > open
> > > > for at least 72 hours ***
> > > >
> > > > Note that we are voting upon the source (tag), binaries are provided for
> > > > convenience.
> > > >
> > > > Source and binary files:
> > > > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.10.4-candidate-4/
> > > >
> > > > SHA-512 checksums:
> > > >
> > > 63343005235be32e970574c9733f06cb472adfdd6511d53b91902d66c805b21cee4039b51b69013bf0f9cbcde82f4cd944c069a7d119d1c908a40716ff82eca3
> > > > apache-pulsar-2.10.4-bin.tar.gz
> > > >
> > > 2d3398a758917bccefa8550f3f69ec8a72a29f541bcd45963e6fddaec024cc690b33f1d49392dc2437e332e90a89e47334925a50960c5f8960e34c1ac8ed2543
> > > > apache-pulsar-2.10.4-src.tar.gz
> > > >
> > > > Maven staging repo:
> > > > https://repository.apache.org/content/repositories/orgapachepulsar-1226
> > > >
> > > > The tag to be voted upon:
> > > > v2.10.4-candidate-4
> > > > (1fe05d5cd3ec9f70cd7179efa4b69eac72fd88bd)
> > > > https://github.com/apache/pulsar/releases/tag/v2.10.4-candidate-4
> > > >
> > > > Pulsar's KEYS file containing PGP keys you use to sign the release:
> > > > https://downloads.apache.org/pulsar/KEYS
> > > >
> > > > Docker images:
> > > >
> > > > 
> > > >
> > > https://hub.docker.com/layers/xiangyingmeng/pulsar/2.10.4/images/sha256-8b76d49401d3fe398be3cde395fb164ad8722b64691e31c44991f32746ca8119?context=repo
> > > >
> > > > 
> > > >
> > > https://hub.docker.com/layers/xiangyingmeng/pulsar-all/2.10.4/images/sha256-c20a13ed215e4837f95a99cf84914d03f557204d44ed610dfc41d2e23a77a92c?context=repo
> > > >
> > > > Please download the source package, and follow the README to build
> > > > and run the Pulsar standalone service.
> > >

[VOTE] Reactive Java client for Apache Pulsar 0.3.0 Candidate 1

2023-04-18 Thread Christophe Bornet 
This is release candidate 1 for the Reactive Java client for Apache
Pulsar, version 0.3.0.

*** Please download, test and vote on this release. This vote will
stay open for at least 72 hours ***

Note that we are voting upon the source (tag). Binaries in the Maven
repository are provided for convenience.

Source package:
https://dist.apache.org/repos/dist/dev/pulsar/pulsar-client-reactive-0.3.0-candidate-1/

SHA-512 checksums:
fbc927bdaaba2873f7e2fff7fa4f778c79294b09b8098ca290bf7f095a42770a2fabece9c5d2ca5212a0b14c30e6f5d02561333804252cff85a0216214c25ff4
 pulsar-client-reactive-0.3.0-src.tar.gz

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachepulsar-1228/

The tag to be voted upon:
v0.3.0-candidate-1 (277d004cfb16379ccbfea8da87a933c7670475b7)
https://github.com/apache/pulsar-client-reactive/releases/tag/v0.3.0-candidate-1

Please download the source package, and follow detailed instructions
for pulsar-client-reactive release validation at
https://github.com/apache/pulsar-client-reactive/wiki/Release-process#release-validation
.

Best regards

Christophe Bornet

Re: [VOTE] Pulsar Release 2.10.4 Candidate 4

2023-04-18 Thread Xiangying Meng 
Close this vote with 4 +1 (binding)

- Mattison
- Penghui
- Jiwei
- Hang

On Tue, Apr 18, 2023 at 10:16 PM Hang Chen  wrote:

> +1(binding)
>
> - Checksum and signatures
> - Built from sources on MacOS (JDK 11 and Maven 3.8.6)
> - Run rat check and check-binary-license on the source.
> - Setup Pulsar cluster with one zookeeper node, one bookie node, and
> one broker node
> - Checked the Grafana dashboard metrics
> - Run Pulsar perf produce and consume
> - Run HDFS-based tiered storage offload and consume
> - Triggered the Pulsar CI [1], and all the tests passed
>
>
> Regards,
> Hang
>
> [1] https://github.com/hangc0276/pulsar/pull/19
>
> guo jiwei  于2023年4月18日周二 11:15写道：
> >
> > +1 (binding)
> >
> > - Check the signature
> > - Build from the source package
> > - Start the standalone
> > - Validate Pub/Sub and Java Functions
> > - Validate Stateful Functions
> >
> > Regards
> > Jiwei Guo (Tboy)
> >
> > On Mon, Apr 17, 2023 at 8:52 PM PengHui Li  wrote:
> > >
> > > +1 (binding)
> > >
> > > - Checked the signature
> > > - Build from the source package
> > > - Start standalone
> > > - Checked cassandra connector
> > > - Checked state function
> > >
> > > Regards,
> > > Penghui
> > >
> > > On Sat, Apr 15, 2023 at 5:37 PM  wrote:
> > >
> > > > +1 (Binding)
> > > >
> > > >  • Built from the source package (maven 3.8.6 OpenJDK 11)
> > > >  • Ran binary package standalone with pub/sub
> > > >  • Ran docker image(pulsar-all) standalone with pub/sub
> > > >  • Ran License check
> > > >
> > > > Best,
> > > > Mattison
> > > > On Apr 12, 2023, 17:10 +0800, Xiangying Meng ,
> > > > wrote:
> > > > > This is the fourth release candidate for Apache Pulsar, version
> 2.10.4.
> > > > >
> > > > > This release contains 126 commits by 37 contributors.
> > > > >
> https://github.com/apache/pulsar/compare/v2.10.3...v2.10.4-candidate-4
> > > > >
> > > > > *** Please download, test and vote on this release. This vote will
> stay
> > > > open
> > > > > for at least 72 hours ***
> > > > >
> > > > > Note that we are voting upon the source (tag), binaries are
> provided for
> > > > > convenience.
> > > > >
> > > > > Source and binary files:
> > > > >
> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.10.4-candidate-4/
> > > > >
> > > > > SHA-512 checksums:
> > > > >
> > > >
> 63343005235be32e970574c9733f06cb472adfdd6511d53b91902d66c805b21cee4039b51b69013bf0f9cbcde82f4cd944c069a7d119d1c908a40716ff82eca3
> > > > > apache-pulsar-2.10.4-bin.tar.gz
> > > > >
> > > >
> 2d3398a758917bccefa8550f3f69ec8a72a29f541bcd45963e6fddaec024cc690b33f1d49392dc2437e332e90a89e47334925a50960c5f8960e34c1ac8ed2543
> > > > > apache-pulsar-2.10.4-src.tar.gz
> > > > >
> > > > > Maven staging repo:
> > > > >
> https://repository.apache.org/content/repositories/orgapachepulsar-1226
> > > > >
> > > > > The tag to be voted upon:
> > > > > v2.10.4-candidate-4
> > > > > (1fe05d5cd3ec9f70cd7179efa4b69eac72fd88bd)
> > > > > https://github.com/apache/pulsar/releases/tag/v2.10.4-candidate-4
> > > > >
> > > > > Pulsar's KEYS file containing PGP keys you use to sign the release:
> > > > > https://downloads.apache.org/pulsar/KEYS
> > > > >
> > > > > Docker images:
> > > > >
> > > > > 
> > > > >
> > > >
> https://hub.docker.com/layers/xiangyingmeng/pulsar/2.10.4/images/sha256-8b76d49401d3fe398be3cde395fb164ad8722b64691e31c44991f32746ca8119?context=repo
> > > > >
> > > > > 
> > > > >
> > > >
> https://hub.docker.com/layers/xiangyingmeng/pulsar-all/2.10.4/images/sha256-c20a13ed215e4837f95a99cf84914d03f557204d44ed610dfc41d2e23a77a92c?context=repo
> > > > >
> > > > > Please download the source package, and follow the README to build
> > > > > and run the Pulsar standalone service.
> > > >
>

Re: [VOTE] Pulsar Release 2.11.1 Candidate-2

2023-04-18 Thread guo jiwei 
Close this vote with 4 +1 (binding)

- Mattison
- Penghui
- Haiting
- Hang


Regards
Jiwei Guo (Tboy)

On Tue, Apr 18, 2023 at 9:47 PM Hang Chen  wrote:
>
> +1 (binding)
>
> - Checksum and signatures
> - Built from sources on MacOS (JDK 17 and Maven 3.8.6)
> - Run rat check and check-binary-license on the source.
> - Setup Pulsar cluster with one zookeeper node, one bookie node, and
> one broker node
> - Checked the Grafana dashboard metrics
> - Run Pulsar perf produce and consume
>
> Thanks,
> Hang
>
> PengHui Li  于2023年4月18日周二 11:27写道：
> >
> > Ah, yes, it works after setting `PULSAR_STANDALONE_USE_ZOOKEEPER=1`
> > We need to push a PR to fix the validation document.
> >
> > +1 (binding)
> >
> > Thanks,
> > Penghui
> >
> > On Mon, Apr 17, 2023 at 11:14 PM Haiting Jiang 
> > wrote:
> >
> > > Hi Penghui
> > >
> > > Did you set `PULSAR_STANDALONE_USE_ZOOKEEPER=1` when verifying
> > > Stateful Functions ?
> > >
> > > Thanks,
> > > Haiting
> > >
> > > On Mon, Apr 17, 2023 at 7:48 PM PengHui Li  wrote:
> > > >
> > > > The RC version failed to verify the state function
> > > >
> > > > The output:
> > > >
> > > > ```
> > > > ~/Downloads/release_2.11.1/apache-pulsar-2.11.1 » bin/pulsar-admin
> > > > functions status --tenant test --namespace test-namespace --name
> > > word_count
> > > > {
> > > >   "numInstances" : 1,
> > > >   "numRunning" : 1,
> > > >   "instances" : [ {
> > > > "instanceId" : 0,
> > > > "status" : {
> > > >   "running" : true,
> > > >   "error" : "",
> > > >   "numRestarts" : 0,
> > > >   "numReceived" : 15,
> > > >   "numSuccessfullyProcessed" : 0,
> > > >   "numUserExceptions" : 15,
> > > >   "latestUserExceptions" : [ {
> > > > "exceptionString" : "State test/test-namespace/word_count is not
> > > > enabled.",
> > > > "timestampMs" : 1681731596735
> > > >   }, {
> > > > "exceptionString" : "State test/test-namespace/word_count is not
> > > > enabled.",
> > > > "timestampMs" : 1681731596736
> > > >   }, {
> > > > "exceptionString" : "State test/test-namespace/word_count is not
> > > > enabled.",
> > > > "timestampMs" : 1681731596737
> > > >   }, {
> > > > "exceptionString" : "State test/test-namespace/word_count is not
> > > > enabled.",
> > > > "timestampMs" : 1681731596738
> > > >   }, {
> > > > "exceptionString" : "State test/test-namespace/word_count is not
> > > > enabled.",
> > > > "timestampMs" : 1681731596738
> > > >   }, {
> > > > "exceptionString" : "State test/test-namespace/word_count is not
> > > > enabled.",
> > > > "timestampMs" : 1681731656845
> > > >   }, {
> > > > "exceptionString" : "State test/test-namespace/word_count is not
> > > > enabled.",
> > > > "timestampMs" : 1681731656845
> > > >   }, {
> > > > "exceptionString" : "State test/test-namespace/word_count is not
> > > > enabled.",
> > > > "timestampMs" : 1681731656846
> > > >   }, {
> > > > "exceptionString" : "State test/test-namespace/word_count is not
> > > > enabled.",
> > > > "timestampMs" : 1681731656847
> > > >   }, {
> > > > "exceptionString" : "State test/test-namespace/word_count is not
> > > > enabled.",
> > > > "timestampMs" : 1681731656847
> > > >   } ],
> > > >   "numSystemExceptions" : 0,
> > > >   "latestSystemExceptions" : [ ],
> > > >   "averageLatency" : 0.146536533,
> > > >   "lastInvocationTime" : 1681731656847,
> > > >   "workerId" : "c-standalone-fw-localhost-8080"
> > > > }
> > > >   } ]
> > > > }
> > > >
> > > -
> > > > ~/Downloads/release_2.11.1/apache-pulsar-2.11.1 » bin/pulsar-admin
> > > > functions querystate --tenant test --namespace test-namespace --name
> > > > word_count -k hello -w
> > > > # key 'hello' doesn't exist.
> > > > # key 'hello' doesn't exist.
> > > > # key 'hello' doesn't exist
> > > > State storage client is not done initializing. Please try again in a
> > > little
> > > > while.
> > > >
> > > > Reason: State storage client is not done initializing. Please try again
> > > in
> > > > a little while.
> > > > ```
> > > >
> > > > The logs from broker:
> > > >
> > > > ```
> > > > 2023-04-17T19:42:55,447+0800 [pulsar-web-48-15] INFO
> > > >  org.eclipse.jetty.server.RequestLog - 127.0.0.1 - -
> > > [17/Apr/2023:19:42:55
> > > > +0800] "GET /admin/v2/namespaces/test HTTP/1.1" 200 23 "-"
> > > > "Pulsar-Java-v2.11.1" 5
> > > > 2023-04-17T19:42:55,450+0800 [pulsar-web-48-4] INFO
> > > >  org.apache.pulsar.common.nar.NarUnpacker - Created directory
> > > >
> > > /var/folders/t4/w6thygwj0y596w0j5h_pzlw4gn/T/pulsar-nar/functions4142072620718955215.tmp-unpacked
> > > > 2023-04-17T19:42:55,451+0800 [pulsar-web-48-4] INFO
> > > >  org.apache.pulsar

Re: Re:Re: [DISCUSS] PIP-255: Assign topic partitions to bundle by round robin

2023-04-18 Thread Lin Lin 
We  make this configuration item to be a dynamic configuration. 
We change change it on broker level.
If we can change it on namespace level, even load of bundle in some namespace 
is balanced, it is still difficult to make broker  balance

On 2023/04/16 16:07:45 lifepuzzlefun wrote:
> I think this feature is very helpful on heavy traffic topic which have 
> continuous stable load on each partition.
> 
> 
> Is there a way we can set some kind of namespace policy to set the plugin 
> PartitionAssigner. Hope this can be set on namespace level, 
> if this can be achieved, it is more adoptable to try this feature in  
> production environment. : - )
> 
> At 2023-04-12 11:24:11, "Lin Lin"  wrote:
> >As I mentioned in the implementation of PIP, we will plug-in the partition 
> >assignment strategy. 
> >
> >However, in the same cluster, it is impossible for some Brokers to use 
> >consistent hashing and some Brokers to use round robin.
> >
> >On 2023/04/11 07:37:19 Xiangying Meng wrote:
> >> Hi Linlin,
> >> > This is an incompatible modification, so the entire cluster needs to be
> >> upgraded, not just a part of the nodes
> >> 
> >> Appreciate your contribution to the new feature in PIP-255.
> >>  I have a question regarding the load-balancing aspect of this feature.
> >> 
> >> You mentioned that this is an incompatible modification,
> >> and the entire cluster needs to be upgraded, not just a part of the nodes.
> >>  I was wondering why we can only have one load-balancing strategy.
> >> Would it be possible to abstract the logic here and make it an optional
> >> choice?
> >> This way, we could have multiple load-balancing strategies,
> >> such as hash-based, round-robin, etc., available for users to choose from.
> >> 
> >> I'd love to hear your thoughts on this.
> >> 
> >> Best regards,
> >> Xiangying
> >> 
> >> On Mon, Apr 10, 2023 at 8:23 PM PengHui Li  wrote:
> >> 
> >> > Hi Lin,
> >> >
> >> > > The load managed by each Bundle is not even. Even if the number of
> >> > partitions managed
> >> >by each bundle is the same, there is no guarantee that the sum of the
> >> > loads of these partitions
> >> >will be the same.
> >> >
> >> > Do we expect that the bundles should have the same loads? The bundle is 
> >> > the
> >> > base unit of the
> >> > load balancer, we can set the high watermark of the bundle, e.g., the
> >> > maximum topics and throughput.
> >> > But the bundle can have different real loads, and if one bundle runs out 
> >> > of
> >> > the high watermark, the bundle
> >> > will be split. Users can tune the high watermark to distribute the loads
> >> > evenly across brokers.
> >> >
> >> > For example, there are 4 bundles with loads 1, 3, 2, 4, the maximum load 
> >> > of
> >> > a bundle is 5 and 2 brokers.
> >> > We can assign bundle 0 and bundle 3 to broker-0 and bundle 1 and bundle 2
> >> > to broker-2.
> >> >
> >> > Of course, this is the ideal situation. If bundle 0 has been assigned to
> >> > broker-0 and bundle 1 has been
> >> > assigned to broker-1. Now, bundle 2 will go to broker 1, and bundle 3 
> >> > will
> >> > go to broker 1. The loads for each
> >> > broker are 3 and 7. Dynamic programming can help to find an optimized
> >> > solution with more bundle unloads.
> >> >
> >> > So, should we design the bundle to have even loads? It is difficult to
> >> > achieve in reality. And the proposal
> >> > said, "Let each bundle carry the same load as possible". Is it the 
> >> > correct
> >> > direction for the load balancer?
> >> >
> >> > > Doesn't shed loads very well. The existing default policy
> >> > ThresholdShedder has a relatively high usage
> >> >threshold, and various traffic thresholds need to be set. Many 
> >> > clusters
> >> > with high TPS and small message
> >> >bodies may have high CPU but low traffic; And for many small-scale
> >> > clusters, the threshold needs to be
> >> >modified according to the actual business.
> >> >
> >> > Can it be resolved by introducing the entry write/read rate to the bundle
> >> > stats?
> >> >
> >> > > The removed Bundle cannot be well distributed to other Brokers. The 
> >> > > load
> >> > information of each Broker
> >> >will be reported at regular intervals, so the judgment of the Leader
> >> > Broker when allocating Bundles cannot
> >> >be guaranteed to be completely correct. Secondly, if there are a large
> >> > number of Bundles to be redistributed,
> >> >the Leader may make the low-load Broker a new high-load node when the
> >> > load information is not up-to-date.
> >> >
> >> > Can we try to force-sync the load data of the brokers before performing 
> >> > the
> >> > distribution of a large number of
> >> > bundles?
> >> >
> >> > For the Goal section in the proposal. It looks like it doesn't map to the
> >> > issues mentioned in the Motivation section.
> >> > IMO, the proposal should clearly describe the Goal, like which problem 
> >> > will
> >> > be resolved with this proposal.
> >> > Both of the above 3 issu

[VOTE] Pulsar Manager Release 0.4.0 Candidate 1

2023-04-18 Thread tison 
Hi everyone,

Please review and vote on the release candidate #1 for the version 0.4.0,
as follows:

[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

The complete staging area is available for your review, which includes:
* Release notes
https://github.com/apache/pulsar-manager/releases/tag/v0.4.0-candidate-1
* The official Apache source and binary distributions to be deployed to
dist.apache.org
* Source code tag "v0.4.0"

PulsarManager's KEYS file contains PGP keys we used to sign this release:
https://downloads.apache.org/pulsar/KEYS

Please download these packages and review this release candidate:

- Review release notes
- Download the source package (verify shasum, and asc) and follow the
instructions to build and run the pulsar-manager front end and back-end
service.
- Download the binary package (verify shasum, and asc) and follow the
instructions to run the pulsar-manager front-end and back-end service.

The vote will be open for at least 72 hours. It is adopted by majority
approval, with at least 3 PMC affirmative votes.

Source and binary files:
https://dist.apache.org/repos/dist/dev/pulsar/pulsar-manager/apache-pulsar-manager-0.4.0/apache-pulsar-manager-0.4.0-bin.tar.gz
https://dist.apache.org/repos/dist/dev/pulsar/pulsar-manager/apache-pulsar-manager-0.4.0/apache-pulsar-manager-0.4.0-src.tar.gz

Best,
tison.

Re: [VOTE] Pulsar Release 2.9.5 Candidate 3

2023-04-18 Thread guo jiwei 
+1 (binding)

- Checked the signature
- Build from the source package
- Start standalone
- Validate Functions
- Validate Connectors
- Validate Stateful Functions


Regards
Jiwei Guo (Tboy)

On Mon, Apr 17, 2023 at 9:10 PM PengHui Li  wrote:
>
> +1 (binding)
>
> - Checked the signature
> - Build from the source package
> - Start standalone
> - Checked cassandra connector
> - Checked state function
>
> Regards,
> Penghui
>
> On Sat, Apr 15, 2023 at 6:08 PM  wrote:
>
> > +1 (Binding)
> >
> >  • Built from the source package (maven 3.8.6 OpenJDK 11.0)
> >  • Ran binary package standalone with pub/sub
> >  • Ran docker image(pulsar-all) standalone with pub/sub
> >  • Ran License check
> >
> > Best,
> > Mattison
> > On Apr 10, 2023, 15:38 +0800, Cong Zhao , wrote:
> > > This is the third release candidate for Apache Pulsar, version 2.9.5.
> > >
> > > This release contains 105 commits by 32 contributors.
> > > https://github.com/apache/pulsar/compare/v2.9.4...v2.9.5-candidate-3
> > >
> > > *** Please download, test, and vote on this release. This vote will stay
> > > open
> > > for at least 72 hours ***
> > >
> > > Note that we are voting upon the source (tag), binaries are provided for
> > > convenience.
> > >
> > > Source and binary files:
> > > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.9.5-candidate-3/
> > >
> > > SHA-512 checksums:
> > >
> > c5f1b06f2f7249616a07b732daa036546591c1aa9e9f37e88ddce6d3eaf3d78b2c6da548ef42946d20d70f9397cd5b55ea4e01fe5cb4aa96fb4608bd62635f67
> > > apache-pulsar-2.9.5-bin.tar.gz
> > >
> > >
> > 72dee0fb642a269c5d0aedfa3e56aee503f56af319b67e4628f39da8cdd44f0bbdcd019e121eafca32b7b7665959770a3d91f0618d157837808da9511e9011a40f
> > > apache-pulsar-2.9.5-src.tar.gz
> > >
> > >
> > 738478bbcf323080487f5645b4c1edb9f45d126a8a2cd7e8c5678c8569ec7b21042ecb6fdd4796b65c6a24bd1e5ad082f5c0d15eb1a64408be4f4c53a06d3ef660
> > > apache-pulsar-offloaders-2.9.5-bin.tar.gz
> > >
> > > Maven staging repo:
> > > https://repository.apache.org/content/repositories/orgapachepulsar-1223/
> > >
> > > The tag to be voted upon:
> > > v2.9.5-candidate-3 (7337470f33586aa639854266efe95c2fa32f48db)
> > > https://github.com/apache/pulsar/releases/tag/v2.9.5-candidate-3
> > >
> > > Pulsar's KEYS file containing PGP keys you use to sign the release:
> > > https://downloads.apache.org/pulsar/KEYS
> > >
> > > Docker images:
> > >
> > > 
> > >
> > https://hub.docker.com/layers/czcoder/pulsar/2.9.5/images/sha256-9bcbad78a65a09b3fe5d3acd158574ad0eda4531cc8c49db16760a2ed1364070?context=explore
> > >
> > > 
> > >
> > https://hub.docker.com/layers/czcoder/pulsar-all/2.9.5/images/sha256-94e50aa1d5b91be9698c959c787eaff0677bb3c65dd1a578b1a641fa5a1d1715?context=explore
> > >
> > > 
> > >
> > https://hub.docker.com/layers/czcoder/pulsar-grafana/2.9.5/images/sha256-d857951e85b41fe7172769380ffbcac126c75dc476fdcb4755623600b3eb03e0?context=explore
> > >
> > > Please download the source package, and follow the README to build
> > > and run the Pulsar standalone service.
> > >
> > > Thanks
> > > Cong Zhao
> >

Re: [VOTE] Pulsar Release 2.9.5 Candidate 3

2023-04-18 Thread Yunze Xu 
+1 (binding)

* Verified checksum and signatures
* Build from source with JDK 8 on Ubuntu 20.04
* Start the standalone with KoP 2.9.x (build from source) with JDK 8,
test producing and consuming among Pulsar clients (2.11.0) and Kafka
clients (2.8.2)

Thanks,
Yunze

On Wed, Apr 19, 2023 at 11:17 AM guo jiwei  wrote:
>
> +1 (binding)
>
> - Checked the signature
> - Build from the source package
> - Start standalone
> - Validate Functions
> - Validate Connectors
> - Validate Stateful Functions
>
>
> Regards
> Jiwei Guo (Tboy)
>
> On Mon, Apr 17, 2023 at 9:10 PM PengHui Li  wrote:
> >
> > +1 (binding)
> >
> > - Checked the signature
> > - Build from the source package
> > - Start standalone
> > - Checked cassandra connector
> > - Checked state function
> >
> > Regards,
> > Penghui
> >
> > On Sat, Apr 15, 2023 at 6:08 PM  wrote:
> >
> > > +1 (Binding)
> > >
> > >  • Built from the source package (maven 3.8.6 OpenJDK 11.0)
> > >  • Ran binary package standalone with pub/sub
> > >  • Ran docker image(pulsar-all) standalone with pub/sub
> > >  • Ran License check
> > >
> > > Best,
> > > Mattison
> > > On Apr 10, 2023, 15:38 +0800, Cong Zhao , wrote:
> > > > This is the third release candidate for Apache Pulsar, version 2.9.5.
> > > >
> > > > This release contains 105 commits by 32 contributors.
> > > > https://github.com/apache/pulsar/compare/v2.9.4...v2.9.5-candidate-3
> > > >
> > > > *** Please download, test, and vote on this release. This vote will stay
> > > > open
> > > > for at least 72 hours ***
> > > >
> > > > Note that we are voting upon the source (tag), binaries are provided for
> > > > convenience.
> > > >
> > > > Source and binary files:
> > > > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.9.5-candidate-3/
> > > >
> > > > SHA-512 checksums:
> > > >
> > > c5f1b06f2f7249616a07b732daa036546591c1aa9e9f37e88ddce6d3eaf3d78b2c6da548ef42946d20d70f9397cd5b55ea4e01fe5cb4aa96fb4608bd62635f67
> > > > apache-pulsar-2.9.5-bin.tar.gz
> > > >
> > > >
> > > 72dee0fb642a269c5d0aedfa3e56aee503f56af319b67e4628f39da8cdd44f0bbdcd019e121eafca32b7b7665959770a3d91f0618d157837808da9511e9011a40f
> > > > apache-pulsar-2.9.5-src.tar.gz
> > > >
> > > >
> > > 738478bbcf323080487f5645b4c1edb9f45d126a8a2cd7e8c5678c8569ec7b21042ecb6fdd4796b65c6a24bd1e5ad082f5c0d15eb1a64408be4f4c53a06d3ef660
> > > > apache-pulsar-offloaders-2.9.5-bin.tar.gz
> > > >
> > > > Maven staging repo:
> > > > https://repository.apache.org/content/repositories/orgapachepulsar-1223/
> > > >
> > > > The tag to be voted upon:
> > > > v2.9.5-candidate-3 (7337470f33586aa639854266efe95c2fa32f48db)
> > > > https://github.com/apache/pulsar/releases/tag/v2.9.5-candidate-3
> > > >
> > > > Pulsar's KEYS file containing PGP keys you use to sign the release:
> > > > https://downloads.apache.org/pulsar/KEYS
> > > >
> > > > Docker images:
> > > >
> > > > 
> > > >
> > > https://hub.docker.com/layers/czcoder/pulsar/2.9.5/images/sha256-9bcbad78a65a09b3fe5d3acd158574ad0eda4531cc8c49db16760a2ed1364070?context=explore
> > > >
> > > > 
> > > >
> > > https://hub.docker.com/layers/czcoder/pulsar-all/2.9.5/images/sha256-94e50aa1d5b91be9698c959c787eaff0677bb3c65dd1a578b1a641fa5a1d1715?context=explore
> > > >
> > > > 
> > > >
> > > https://hub.docker.com/layers/czcoder/pulsar-grafana/2.9.5/images/sha256-d857951e85b41fe7172769380ffbcac126c75dc476fdcb4755623600b3eb03e0?context=explore
> > > >
> > > > Please download the source package, and follow the README to build
> > > > and run the Pulsar standalone service.
> > > >
> > > > Thanks
> > > > Cong Zhao
> > >

Re: [DISCUSS] Add checklist for PMC binding vote of PIP

2023-04-18 Thread Yunze Xu 
>  you are ok with having a summary template, but have it non-required?

Yes to me.

In addition, I think the root cause of the problems you met is that
some PIPs have low quality. They are not clear and friendly to others.
A good proposal should not require reviewers to have deep knowledge of
a specific domain. I think what PMC members should do to improve it is
to cast the -1 to those ambiguous proposals until they become clear.

Thanks,
Yunze

On Tue, Apr 18, 2023 at 8:14 PM Asaf Mesika  wrote:
>
> The problem I'm trying to solve is: lack of ability to understand PIPs.
> PIPs I had the chance of reading lack:
> * Background information: It should contain all background information
> necessary to understand the problem and the solution
> * Clarity: It should be written in a coherent and easy to understand way.
>
> I thought this could improve using 2 ways:
> 1. Define a clear template for PIPs - this should solve all the missing
> information. This is in progress.
> 2. Provide a checklist to verify the +1 voter check those 3 things:
> background information, clarity, solid technical solution.
>
> Both Enrico and Yunze say, if I understand correctly, that the +1 voter
> checks those 3 things implicitly.
> Yet when I try to learn Pulsar by reading historical PIPs, I find some
> lacking on those things (clarity, background information) making it super
> hard for me to get onboard into Pulsar.
>
> Another aspect worth noting is: community increase. In my own opinion,
> documents with clarity and enough background information produce a feeling
> of quality - high quality. Making Pulsar PIPs clear and have all
> information to understand them will help grow Pulsar adoption.
>
> Maybe incremental improvements are better.. If I understand correctly, both
> Enrico and Yunze - you are ok with having a summary template, but have it
> non-required?
>
> Enrico - Regarding previous suggestions. Root cause - help make Pulsar
> better from my own perspective. Some suggestions may be super bad
> suggestions and hopefully some will be good :)
> This specific one - I validated with the PMC members in the weekly zoom
> meeting roughly 3 weeks ago, and got +1 across the board (we had 5 people).
> I did it since I felt it was a touchy subject.
>
> Thanks,
>
> Asaf
>
>
>
>
>
>
> On Tue, Apr 18, 2023 at 9:15 AM Yunze Xu 
> wrote:
>
> > Basically I think describing how much work the reviewer did to give
> > his +1 is good. Just like the vote for a release, each +1 follows with
> > the verifications he did, e.g. here [1] is a vote for Pulsar 2.11.1
> > candidate 1:
> >
> > > • Built from the source package (maven 3.8.6 OpenJDK 17.0)
> > > • Ran binary package standalone with pub/sub
> > > ...
> >
> > But I don't think forcing the rule is good. The proposal could
> > sometimes be not so complicated. From my personal experience,
> > sometimes I vote my +1 just because I think it's good and there is no
> > serious problem. If you want me to vote again with the checklist, I
> > might still not have an idea of what I should write, unless there is a
> > template and I filled the template. Only if the proposal is somehow
> > complicated will the checklist be meaningful, like the PIP-192, which
> > is a very complicated proposal.
> >
> > > Moreover, this checklist can ensure that all participants have
> > thoroughly reviewed the PIP,
> >
> > Regarding this point from Xiangying, I want to repeat a similar
> > thought [2] for the previous discussion.
> >
> > IF ANYONE WANT, HE CAN STILL COPY A CHECKLIST FROM OTHERS AND JUST
> > PERFORM SOME SLIGHTLY CHANGES.
> >
> > Forcing a checklist won't change anything if there is a PMC that gave
> > his vote without any careful review. It just makes the rule more
> > complicated. If you don't trust a PMC, no rule could restrict him.
> > Rules only make him a better game player.
> >
> > In addition, when a reviewer approves a PR, should he add a checklist
> > as well, instead of a simple LGTM or +1? Huge PRs appear more often
> > than complicated proposals.
> >
> > In conclusion, I am +0 to this suggestion. If this suggestion is
> > passed, I will follow it well. But if I cannot think of a checklist
> > with a proposal, I will try to be a good vote game player.
> >
> > [1] https://lists.apache.org/thread/13xmt4jdwmlo1mo5dhkxlg9pnkfdwjjj
> > [2] https://lists.apache.org/thread/o0vw1dfoo84pscfd46gdm3sm9mvovmr2
> >
> > Thanks,
> > Yunze
> >
> > On Mon, Apr 17, 2023 at 3:48 PM PengHui Li  wrote:
> > >
> > > I don't think it will bring more burden on reviewers.
> > > It will only provide a checklist for reviewers before
> > > you vote +1 or -1. It could be done in 1 minute if you
> > > did a great proposal review. Of course, if you are
> > > missing some aspects that should be reviewed,
> > > This will make the reviewer spend more time reviewing
> > > the missing items, but it is valuable.
> > >
> > > I don't think this proposal is accusing PMCs, but PMCs
> > > might also miss some items. The checklist can

ANNOUNCE] Apache Pulsar 2.10.4 released

2023-04-18 Thread Xiangying Meng 
The Apache Pulsar team is proud to announce Apache Pulsar version 2.10.4.

Pulsar is a highly scalable, low latency messaging platform running on
commodity hardware. It provides simple pub-sub semantics over topics,
guaranteed at-least-once delivery of messages, automatic cursor management
for
subscribers, and cross-datacenter replication.

For Pulsar release details and downloads, visit:

https://pulsar.apache.org/download

Release Notes are at:
https://pulsar.apache.org/release-notes

We would like to thank the contributors that made the release possible.

Regards,

The Pulsar Team

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

2023-04-18 Thread Michael Marshall 
> `delete-all-namespaces` and `delete-all-topics`

I support these terms as well. Regarding the concern about CLI
usability, bash completion could help a lot.

Additional questions:

1. What permission level is required for each action?
2. How will system topics be treated?

Thanks,
Michael


On Mon, Apr 17, 2023 at 2:59 AM Xiangying Meng  wrote:
>
> Hi Penghui,
>
> I appreciate your feedback and completely agree with your concern about the
> learning curve for Pulsar users. Introducing additional keywords could
> potentially increase the complexity for users who need to understand the
> new terms. Therefore, I accept your suggestion to use delete-all-namespaces
> and delete-all-topics for the proposed improvement.
>
> Thank you for sharing your insights, and I look forward to working on this
> enhancement with the community's support.
>
> Best regards,
> Xiangying
>
> On Mon, Apr 17, 2023 at 3:24 PM PengHui Li  wrote:
>
> > The new operation will delete all the data and the metadata under
> > a tenant or namespace. I would like to suggest to use
> >
> > `delete-all-namespaces` and `delete-all-topics`
> >
> > The `delete` actually acts as a fact of deleting metadata and data.
> > And `truncate` is for deleting the data. IMO, we'd better not
> > introduce another new keyword, either `clear` or `wipe`, because
> > it will bring more knowledge to Pulsar users who must understand.
> >
> >
> > Thanks,
> > Penghui
> >
> > On Mon, Apr 17, 2023 at 10:56 AM Xiangying Meng 
> > wrote:
> >
> > > Hi Enrico,
> > >
> > > Thank you for your feedback. While I understand that
> > > "delete-all-namespaces" is more explicit,
> > > I also think it's a bit lengthy for a command-line parameter.
> > > I personally believe the "wipe" option, combined with a safety
> > confirmation
> > > step,
> > >  would be more user-friendly and efficient.
> > >
> > > By adding a safety confirmation step, we can minimize the risk of
> > > accidental mass deletion.
> > > Users would be required to confirm their intention to perform the
> > deletion
> > > by
> > > typing 'YES' or a similar confirmation word before the operation
> > proceeds.
> > >
> > > What do you think about this approach?
> > > If there's a consensus, I can work on implementing this feature with the
> > > "wipe" option and the safety confirmation step.
> > >
> > > Best regards,
> > > Xiangying
> > >
> > > On Sun, Apr 16, 2023 at 11:25 PM Enrico Olivelli 
> > > wrote:
> > >
> > > > Il Dom 16 Apr 2023, 15:45 Asaf Mesika  ha
> > > scritto:
> > > >
> > > > > How about "truncate" instead of "clear"?
> > > > >
> > > >
> > > >
> > > > Truncate is better, or maybe 'wipe' (because truncate means another
> > > > operation for topics currently)
> > > >
> > > > Another alternative, more explicit:
> > > > pulsar-admin tenants delete-all-namespaces TENANT
> > > >
> > > > Enrico
> > > >
> > > > >
> > > > > Just wondering - since it is such a dangerous command, how can we
> > help
> > > > the
> > > > > user not make an accidental mass deletion?
> > > > >
> > > > > On Sat, Apr 15, 2023 at 1:12 PM Girish Sharma <
> > scrapmachi...@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > > However, the current goal is to keep the tenant and namespace
> > > intact
> > > > > > while
> > > > > > > cleaning up their contents.
> > > > > > Ah, I see now. Yes, in that case a clear command is better. Will
> > this
> > > > > > command also take into account the value of the broker config
> > > > > > `forceDeleteNamespaceAllowed` in case someone is clearing the owner
> > > > > tenant?
> > > > > >
> > > > > > Regards
> > > > > >
> > > > > > On Sat, Apr 15, 2023 at 3:39 PM Enrico Olivelli <
> > eolive...@gmail.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > The proposal sounds really useful, especially for automated
> > > testing.
> > > > > > > +1
> > > > > > >
> > > > > > > Enrico
> > > > > > >
> > > > > > > Il giorno sab 15 apr 2023 alle ore 12:07 Xiangying Meng
> > > > > > >  ha scritto:
> > > > > > > >
> > > > > > > > Dear Girish,
> > > > > > > >
> > > > > > > > Thank you for your response and suggestion to extend the use of
> > > the
> > > > > > > > `boolean force` flag for namespaces and tenants.
> > > > > > > > I understand that the `force` flag is already implemented for
> > > > > deleting
> > > > > > > > topics, namespaces, and tenants,
> > > > > > > > and it provides a consistent way to perform these actions.
> > > > > > > >
> > > > > > > > However, the current goal is to keep the tenant and namespace
> > > > intact
> > > > > > > while
> > > > > > > > cleaning up their contents.
> > > > > > > > In other words, I want to have a way to remove all topics
> > within
> > > a
> > > > > > > > namespace or all namespaces and topics
> > > > > > > > within a tenant without actually deleting the namespace or
> > tenant
> > > > > > itself.
> > > > > > > >
> > > > > > > > To achieve this goal, I proposed adding a `clear` command for
> > > > > > > `namespaces`
> > > > > > > > and `tenants`.
> > > > > >