[GitHub] [pulsar-dotpulsar] usaguerrilla closed pull request #93: Feature/microbatching parallel consumer

2021-12-15 Thread GitBox
usaguerrilla closed pull request #93: URL: https://github.com/apache/pulsar-dotpulsar/pull/93 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-

Re: PIP 116: Create Pulsar Writing Style Guide

2021-12-15 Thread Yu Liu
Hi Haiting, There are no automatic tools to check **all** writing style issues, but they can help check some. Steps see https://docs.google.com/document/d/1Qw7LHQdXWBW9t2-r-A7QdFDBwmZh6ytB4guwMoXHqc0/edit Hi Dave, Thanks! I've incorporated your comments. On Thu, Dec 16, 2021 at 2:34 AM Dave Fish

Re: [DISCUSSION] PIP-117: Change Pulsar standalone defaults

2021-12-15 Thread Haiting Jiang
Hi Enrico, This migration tool is a better-to-have and mostly about metadata store part. Some of our user use Standalone in a long running test environment, with this migration tool they do not need to recreate all the topics and subscriptions manually and take these benefits with new default

Re: [DISCUSSION] PIP-118: Do not restart brokers when ZooKeeper session expires

2021-12-15 Thread Matteo Merli
On Wed, Dec 15, 2021 at 5:47 PM Haiting Jiang wrote: > Small question about new parameter "zookeeperSessionExpiredPolicy", now we > have Etcd implementation for metadata store, is it better to use > "metadataStoreExpiredPolicy" ? Good point, we should also use this occasion to deprecate the `zo

Re: [DISCUSSION] PIP-118: Do not restart brokers when ZooKeeper session expires

2021-12-15 Thread Haiting Jiang
+1 Great PIP. Long expected. Small question about new parameter "zookeeperSessionExpiredPolicy", now we have Etcd implementation for metadata store, is it better to use "metadataStoreExpiredPolicy" ? On 2021/12/14 18:03:20 Matteo Merli wrote: > https://github.com/apache/pulsar/issues/13304 >

Re: [VOTE] Pulsar Node.js Client Release 1.5.0 Candidate 1

2021-12-15 Thread Hiroyuki Sakai
+1 (binding) * check the license headers * build the source and run producer/consumer * verify checksum and signatures == Hiroyuki Sakai Yahoo Japan Corp. E-mail: hsa...@yahoo-corp.jp From: Masahiro Sakamoto Sent: Tuesday, December 14, 2021 17:55 To: dev

Re: [DISCUSS] How to handle stale PRs

2021-12-15 Thread Dave Fisher
> On Dec 15, 2021, at 4:06 PM, Matteo Merli wrote: > >> Is #3267 Support set publish time on broker side one of those very valuable >> ideas that was later rejected, likely for performance reasons? > > No, this was one that was superseded by other changes. Then I’ll close it. > >> One pro

Re: [DISCUSS] How to handle stale PRs

2021-12-15 Thread Matteo Merli
> Is #3267 Support set publish time on broker side one of those very valuable > ideas that was later rejected, likely for performance reasons? No, this was one that was superseded by other changes. > One problem with the current state is that PRs and even higher level ideas > have a shelf life.

Re: [DISCUSS] How to handle stale PRs

2021-12-15 Thread Chris Herzog
It isn't even an issue related to OSS - every long lived project suffers from this same issue. Whether it's a long lingering defect report or a fix that never got integrated in a timely manner, time wounds all heels. Careful considered review is perfection which can't be hit; if it could be done,

Re: [DISCUSS] How to handle stale PRs

2021-12-15 Thread Dave Fisher
I’d like to point out that if we label auto-closed PRs properly it is easy enough to find them in the GitHub UI. > On Dec 15, 2021, at 3:05 PM, Jonathan Ellis wrote: > > One problem with the current state is that PRs and even higher level ideas > have a shelf life. Declaring PR bankruptcy does

Re: [DISCUSS] How to handle stale PRs

2021-12-15 Thread Jonathan Ellis
One problem with the current state is that PRs and even higher level ideas have a shelf life. Declaring PR bankruptcy does in fact solve this problem. The other problem is that from a new contributor's perspective it's impossible to tell which issues are relevant and which are clutter that we hav

Re: [DISCUSS] Add definition to our cherry picking process

2021-12-15 Thread Jonathan Ellis
First, it's often not an issue -- new contributors typically start with small patches that the reviewer or committer can easily rebase to the appropriate branch, and by the time they're working on more invasive fixes (we don't have any of those, right? :) they get used to how it works. But when th

Re: [DISCUSS] How to handle stale PRs

2021-12-15 Thread Dave Fisher
> On Dec 15, 2021, at 1:11 PM, Matteo Merli wrote: > > I'm not convinced by having a blanket policy here. > > In several cases, these PRs carried some very valuable ideas that > still needed some work to get merged. By using blanket close, we'd be > losing all that context and we should not d

Re: [DISCUSS] How to handle stale PRs

2021-12-15 Thread Dave Fisher
I think that there should be a label added to any old PR that a contributor wants to keep open. I think “status/hold” would be a good label name. That will keep others who wish to review old PRs and close them from wasting their time. I looked at the labels and I wonder about those that are “tri

Re: [DISCUSS] How to handle stale PRs

2021-12-15 Thread Michael Marshall
> I'm not convinced by having a blanket policy here. I'm fine with stopping short of an automated policy. However, I think it'd be helpful to provide committers, especially new committers, with conditions that make a PR eligible to be closed. Since committers act on behalf of the PMC, documentatio

Re: [DISCUSS] Add definition to our cherry picking process

2021-12-15 Thread Enrico Olivelli
Jonathan, Il Mer 15 Dic 2021, 21:11 Jonathan Ellis ha scritto: > What are the benefits of cherry picking vs committing to the oldest > relevant branch and merging forwards? We (and most of the ASF projects I am involved in) ask contributors to target the master branch for the patches. How can

Re: [DISCUSS] How to handle stale PRs

2021-12-15 Thread Matteo Merli
I'm not convinced by having a blanket policy here. In several cases, these PRs carried some very valuable ideas that still needed some work to get merged. By using blanket close, we'd be losing all that context and we should not do that. What would actually be helpful, is help in reviewing these

Re: [DISCUSS] How to handle stale PRs

2021-12-15 Thread Michael Marshall
I am +1 for closing PRs that are over a year old. Does anyone else in the community have thoughts on these old PRs? Getting consensus and creating a process here could help make our committers more efficient. - Michael On Fri, Dec 3, 2021 at 1:25 PM Jonathan Ellis wrote: > > Agreed. > > I don'

Re: [DISCUSS] Add definition to our cherry picking process

2021-12-15 Thread Jonathan Ellis
What are the benefits of cherry picking vs committing to the oldest relevant branch and merging forwards? Git is designed around merge since that preserves the commit sha which allows it to be tracked across branches. And a merge based workflow avoids the problem here by design. On Wed, Dec 15,

[DISCUSS] Add definition to our cherry picking process

2021-12-15 Thread Michael Marshall
Hello Pulsar Community, As far as I can tell, we do not have a publicly defined process for cherry-picking commits to release branches [0]. As a new committer, I'd like to provide my newly gained perspective and ask that we add some guidance to our wiki. Regarding cherry picking, I see two patter

[DISCUSSION] PIP-123: Introduce Pulsar metadata CLI tool

2021-12-15 Thread Matteo Merli
https://github.com/apache/pulsar/issues/13346 Pasted here for quoting convenience - ## Motivation For a very long time, we have included a CLI command to start the ZooKeeper shell utility: `pulsar zookeeper-shell`, which is essentially a repackaging of the ZooKeeper tool `zkCli.sh`. Thi

Re: PIP 116: Create Pulsar Writing Style Guide

2021-12-15 Thread Dave Fisher
Thanks Yu for this initiative. I made a small correction in the document. Regards, Dave > On Dec 14, 2021, at 7:17 PM, Yu wrote: > > Hi Pulsar enthusiasts, > > > As you may notice, there are more and more documentation contributions > nowadays, which is great! > > > Similar to coding style

Re: [DISCUSS] Converge authz to operate subscription policies of namespace

2021-12-15 Thread Michael Marshall
Hi Ruguo, Thanks for bringing this to the mailing list. I think we should maintain our current authorization granularity, even if we're not currently using it in the default authorization provider. I see you updated your PR [0] to keep the granularity--I'll provide a review soon. Thanks, Michae

Re: [DISCUSSION] PIP-117: Change Pulsar standalone defaults

2021-12-15 Thread Michael Marshall
+1 I think this is a good idea. The tradeoff here is that testing against a standalone "cluster" is even more different than testing against a distributed Pulsar cluster. I think this is fine, but if we accept this PIP, I think we should update our guidance on the Wiki, which currently recommends

Re: Status of Pulsar 2.9.0 and starting 2.9.1

2021-12-15 Thread Enrico Olivelli
Devin, Il giorno mer 15 dic 2021 alle ore 15:32 Devin Bost ha scritto: > If there are known critical bugs in 2.9.0, we should be transparent with > users (and let them know that it's beta, or at least mention what features > are incomplete) if it's going to be announced so trust is not damaged.

Re: Status of Pulsar 2.9.0 and starting 2.9.1

2021-12-15 Thread Devin Bost
If there are known critical bugs in 2.9.0, we should be transparent with users (and let them know that it's beta, or at least mention what features are incomplete) if it's going to be announced so trust is not damaged. Although I typically expect x.x.0 releases to be experimental (based on my ex

Re: Does the #13291 should contains in 2.8.2

2021-12-15 Thread Enrico Olivelli
+1 Enrico Il giorno mer 15 dic 2021 alle ore 14:27 PengHui Li ha scritto: > We should contain #13291 in 2.8.2. > > Penghui > > On Wed, Dec 15, 2021 at 7:19 PM ZhangJian He wrote: > > > Hello, > > @LinLin @lipenghui > > > > I want to discuss if #13291 should contains in release 2.8.2. > > > > It

Re: Does the #13291 should contains in 2.8.2

2021-12-15 Thread PengHui Li
We should contain #13291 in 2.8.2. Penghui On Wed, Dec 15, 2021 at 7:19 PM ZhangJian He wrote: > Hello, > @LinLin @lipenghui > > I want to discuss if #13291 should contains in release 2.8.2. > > It's a compatible problem since 2.8, and we already commited it to master > and branch 2.9 > > > Tha

Re: Status of Pulsar 2.9.0 and starting 2.9.1

2021-12-15 Thread Enrico Olivelli
Hang, sorry, I wanted to say that I am double checking if people forgot to add "cherry-picked" Enrico Il giorno mer 15 dic 2021 alle ore 13:20 Enrico Olivelli < eolive...@gmail.com> ha scritto: > > > Il giorno mer 15 dic 2021 alle ore 12:52 Hang Chen > ha scritto: > >> Hi Enrico, >> Thanks

Re: Status of Pulsar 2.9.0 and starting 2.9.1

2021-12-15 Thread Enrico Olivelli
Il giorno mer 15 dic 2021 alle ore 12:52 Hang Chen ha scritto: > Hi Enrico, > Thanks for your great work! I found 150+ PR labeled as > `release/2.9.1`, but doesn't contain in v2.9.1-candidate-1 tag. Does > those PRs release in next version? > > https://github.com/apache/pulsar/pulls?q=is%3A

Re: Status of Pulsar 2.9.0 and starting 2.9.1

2021-12-15 Thread Hang Chen
Hi Enrico, Thanks for your great work! I found 150+ PR labeled as `release/2.9.1`, but doesn't contain in v2.9.1-candidate-1 tag. Does those PRs release in next version? https://github.com/apache/pulsar/pulls?q=is%3Apr+label%3Arelease%2F2.9.1+-label%3Acherry-picked%2Fbranch-2.9+is%3Aclosed T

Does the #13291 should contains in 2.8.2

2021-12-15 Thread ZhangJian He
Hello, @LinLin @lipenghui I want to discuss if #13291 should contains in release 2.8.2. It's a compatible problem since 2.8, and we already commited it to master and branch 2.9 Thanks ZhangJian He

Re: Status of Pulsar 2.9.0 and starting 2.9.1

2021-12-15 Thread Enrico Olivelli
I had prepared the rc1 for 2.9.1 but today I added https://github.com/apache/pulsar/pull/13291 I will create a new RC and send a VOTE, possibly today The first RC will be rc2, in order to not mess up the git repository and the dist area Enrico Il giorno lun 13 dic 2021 alle ore 19:22 Sijie Guo

Re: [DISCUSSION] PIP-117: Change Pulsar standalone defaults

2021-12-15 Thread Enrico Olivelli
+1 @Haiting Pulsar Standalone is usually used for local development, I am not sure you need a migration tool. You can still keep the configuration to use previously (standalone.conf) Enrico Il giorno mer 15 dic 2021 alle ore 10:56 Haiting Jiang < jianghait...@apache.org> ha scritto: > +1 (non-b

Re: PIP 116: Create Pulsar Writing Style Guide

2021-12-15 Thread Haiting Jiang
+1, Great stuff. I am wondering if there is some tool to check it automatically? like code-style check. Thanks, Haiting On 2021/12/15 03:17:45 Yu wrote: > Hi Pulsar enthusiasts, > > > As you may notice, there are more and more documentation contributions > nowadays, which is great! > > > Si

Re: [DISCUSSION] PIP-120: Enable client memory limit by default

2021-12-15 Thread Haiting Jiang
+1 (non-binding) Haiting On 2021/12/14 19:20:02 Matteo Merli wrote: > https://github.com/apache/pulsar/issues/13306 > > > Pasted below for quoting convenience. > > > > > ## Motivation > > In Pulsar 2.8, we have introduced a setting to control the amount of memory > used by a client

Re: [DISCUSSION] PIP-117: Change Pulsar standalone defaults

2021-12-15 Thread Haiting Jiang
+1 (non-binding) Out of the scope of this PIP, a follow up data migration tool could be added to help migration from old settings without data loss. --- Haiting On 2021/12/15 01:04:21 Hang Chen wrote: > +1 > > Thanks, > Hang > > PengHui Li 于2021年12月15日周三 07:52写道: > > > > +1 > > > > Penghui >

Re: CheckStyle Rule Change

2021-12-15 Thread Ivan Kelly
Ignore this, looked further and it actually does double resolution. sheesh On Wed, Dec 15, 2021 at 10:14 AM Ivan Kelly wrote: > > Maybe this has been discussed elsewhere, but if possible, we should > add a rule that to block any future log4shell type attack area. > In human terms, the rule should

Re: CheckStyle Rule Change

2021-12-15 Thread Ivan Kelly
Maybe this has been discussed elsewhere, but if possible, we should add a rule that to block any future log4shell type attack area. In human terms, the rule should be: Disallow any {log,logger,LOG}.* with a non-const first argument. -Ivan On Wed, Dec 15, 2021 at 7:15 AM Enrico Olivelli wrote: >

Re: [VOTE] Apache Pulsar 2.8.2 candidate 1

2021-12-15 Thread Lin Lin
On 2021/12/14 03:25:45 linlin wrote: > This is the first release candidate for Apache Pulsar, version 2.8.2. > > It fixes the following issues: > https://github.com/apache/pulsar/issues?q=label%3Acherry-picked%2Fbranch-2.8+is%3Aclosed+label%3Arelease%2F2.8.2 > > *** Please download, test and v

Re: [VOTE] Apache Pulsar 2.8.2 candidate 1

2021-12-15 Thread Lari Hotari
-1, I verified the packages, but the signatures are invalid. It looks like the https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.8.2-candidate-1/apache-pulsar-2.8.2-bin.tar.gz.asc file is from December 1st. I checked this in the SVN staging repository history ( https://dist.apache.org/repos

Re: [PIP-78] Reduce redundant producers from partitioned producer

2021-12-15 Thread Yuri Mizushima
Enrico, Thank you for your clarification. > I would add a configuration parameter to allow the client to use this > feature. > So: > - feature enabled -> everything works like in your patch > - feature disabled -> the broker ignores *partial_producer_supported *and > runs as before Okay. That mak

Analysis of impact of most recent Log4j CVE, CVE-2021-45046 to Pulsar

2021-12-15 Thread Lari Hotari
There's a new CVE, CVE-2021-45046 in Log4j < 2.16.0 details: https://blogs.apache.org/foundation/entry/apache-log4j-cves Summary: Pulsar isn't impacted with CVE-2021-44228 when the default log4j configuration is used. However, remember that Pulsar is impacted by the actual Log4Shell CVE and Pulsa