Re: [DISCUSS] Proceeding with PIP-62 plan, before Apache Pulsar 2.10.0 is released

2021-12-21 Thread Enrico Olivelli
Lari, Il giorno mer 22 dic 2021 alle ore 08:31 Lari Hotari ha scritto: > Dear Pulsar community members, > > PIP-62[1], "PIP 62: Move connectors, adapters and Pulsar Presto to separate > repositories" was created in April 2020. The repositories for > pulsar-connectors, pulsar-adapters and pulsar-

Re: WebSite build CI job does work - blocker for announcing release !

2021-12-21 Thread Nicolò Boschi
I think the problem is that it times out (currently there's a job timeout of 2h) I created this PR to increase it to 3h https://github.com/apache/pulsar/pull/13449 Il giorno mer 22 dic 2021 alle ore 08:23 Enrico Olivelli < eolive...@gmail.com> ha scritto: > Hello, > the WebSite build CI job is s

[DISCUSS] Proceeding with PIP-62 plan, before Apache Pulsar 2.10.0 is released

2021-12-21 Thread Lari Hotari
Dear Pulsar community members, PIP-62[1], "PIP 62: Move connectors, adapters and Pulsar Presto to separate repositories" was created in April 2020. The repositories for pulsar-connectors, pulsar-adapters and pulsar-sql were created about a year ago [2]. I'd like to propose that we continue with t

Re: [VOTE] PIP-123: Introduce Pulsar metadata CLI tool

2021-12-21 Thread Enrico Olivelli
very good Enrico Il giorno mer 22 dic 2021 alle ore 03:37 mattison chao < mattisonc...@gmail.com> ha scritto: > +1 > > On Wed, 22 Dec 2021 at 07:59, Matteo Merli wrote: > > > This is the voting thread for PIP-123. It will stay open for at least > 48h. > > > > https://github.com/apache/pulsar/is

Re: [PR] Pulsar non root docker image

2021-12-21 Thread Enrico Olivelli
Michael, I would like to add this item to the list https://github.com/apache/pulsar/pull/10815 Basically if you are running as non root, you cannot add tools on demand, so we need to add basic tools, like netstat/vim/unzip otherwise when you have a problem you are trapped. there are ways to

WebSite build CI job does work - blocker for announcing release !

2021-12-21 Thread Enrico Olivelli
Hello, the WebSite build CI job is stuck https://github.com/apache/pulsar/runs/4601374159?check_suite_focus=true This is probably the problem, it is due to "crowdin" that is timing out File 'master/website/versioned_docs/version-2.6.3/concepts-overview.md'| - SKIPPED File 'master/website/i18n/en.

Re: [VOTE] Apache Pulsar 2.8.2 candidate 2

2021-12-21 Thread Lin Lin
On 2021/12/21 10:48:41 Shivji Kumar Jha wrote: > Hi LinLin, > > Log4j version 2.16.0 has DDoS possibilities in some cases [1] . Can we move > to Log4j 2.17.0 in 2.8.2? > > Apache Log4j2 versions 2.0-alpha1 through 2.16.0 (excluding 2.12.3) did not > > protect from uncontrolled recursion from s

Re: [PR] Pulsar non root docker image

2021-12-21 Thread Haiting Jiang
> 1. Pulsar configuration is read in only from configuration files in > `/pulsar/conf`. A non root user must be able to update these files to > have run with custom configuration. About the configurations, I also see similar require like this lately [0]. IMHO, update any configs without redeploy

Re: [VOTE] PIP-120: Enable client memory limit by default

2021-12-21 Thread mattison chao
+1 On Wed, 22 Dec 2021 at 07:46, Matteo Merli wrote: > This is the voting thread for PIP-120. It will stay open for at least 48h. > > https://github.com/apache/pulsar/issues/13306 > > > > ## Motivation > > In Pulsar 2.8, we have introduced a setting to control the amount of memory > use

Re: [VOTE] PIP-123: Introduce Pulsar metadata CLI tool

2021-12-21 Thread mattison chao
+1 On Wed, 22 Dec 2021 at 07:59, Matteo Merli wrote: > This is the voting thread for PIP-123. It will stay open for at least 48h. > > https://github.com/apache/pulsar/issues/13346 > > > - > ## Motivation > > For a very long time, we have included a CLI command to start the ZooKeeper > sh

[VOTE] PIP-123: Introduce Pulsar metadata CLI tool

2021-12-21 Thread Matteo Merli
This is the voting thread for PIP-123. It will stay open for at least 48h. https://github.com/apache/pulsar/issues/13346 - ## Motivation For a very long time, we have included a CLI command to start the ZooKeeper shell utility: `pulsar zookeeper-shell`, which is essentially a repackagin

[VOTE] PIP-117: Change Pulsar standalone defaults

2021-12-21 Thread Matteo Merli
This is the voting thread for PIP-117. It will stay open for at least 48h. https://github.com/apache/pulsar/issues/13302 ## Motivation Pulsar standalone is the "Pulsar in a box" version of a Pulsar cluster, where all the components are started within the context of a single JVM process. U

Re: [DISCUSSION] PIP-117: Change Pulsar standalone defaults

2021-12-21 Thread Matteo Merli
-- Matteo Merli On Sun, Dec 19, 2021 at 9:42 PM Lan Liang wrote: > > My two cents +1 , In my case, i sometimes need to run pulsar standalone in > some server with limited resources. > The other point i can think of is maybe schema storage also need to change > from using BK to using local fi

[VOTE] PIP-120: Enable client memory limit by default

2021-12-21 Thread Matteo Merli
This is the voting thread for PIP-120. It will stay open for at least 48h. https://github.com/apache/pulsar/issues/13306 ## Motivation In Pulsar 2.8, we have introduced a setting to control the amount of memory used by a client instance. ```java interface ClientBuilder { ClientBui

[VOTE] PIP-119: Enable consistent hashing by default on KeyShared dispatcher

2021-12-21 Thread Matteo Merli
This is the voting thread for PIP-119. It will stay open for at least 48h. https://github.com/apache/pulsar/issues/13305 --- ## Motivation The consistent hashing implementation to uniformly assign keys to consumers in the context of a KeyShared subscription, was introduced in https://gi

[VOTE] PIP-118: Do not restart brokers when ZooKeeper session expires

2021-12-21 Thread Matteo Merli
https://github.com/apache/pulsar/issues/13304 Following the discussion, I have updated the proposal to also include the deprecation and renaming of the config setting name to `metadataSessionExpiredPolicy`. --- ## Motivation After all the work done for PIP-45 that was already included in 2

Re: [DISCUSSION] PIP-120: Enable client memory limit by default

2021-12-21 Thread Matteo Merli
> Did you consider determining the default limit by inspecting the JVM's > runtime? > For example, in the broker, we set `maxMessagePublishBufferSizeInMB` to 1/2 > of direct memory, by default. > > I concede that this config is more complicated than a broker config since > we're talking about a cl

Re: [DISCUSSION] PIP-118: Do not restart brokers when ZooKeeper session expires

2021-12-21 Thread Matteo Merli
On Mon, Dec 20, 2021 at 2:24 PM Michael Marshall wrote: > I haven't been able to follow the PIP 45 work, so I have some high > level questions. Do we have documentation on the implementation of > these re-acquirable resource locks? I'd like to understand how these > work. There is admittedly not

Re: [PR] Pulsar non root docker image

2021-12-21 Thread Michael Marshall
All tests are now passing for this PR [0]. I built the docker image and pushed it to my personal repository to simplify testing [1] for anyone interested in verifying the changes. I would like our docker image to be as close to immutable as possible. As far as I can tell, here are the only reasons

Re: [DISCUSSION] PIP-124: Create init subscription before sending message to DLQ

2021-12-21 Thread Michael Marshall
> IMO, we won't create the init subscription if `allowAutoSubscriptionCreation` > is false. Otherwise, it will confuse the user. Users can explicitly create > that > subscription to handle this case. That is a good point, but I don't think we should attempt to create the subscription implicitly v

Re: [VOTE] Apache Pulsar 2.8.2 candidate 2

2021-12-21 Thread Enrico Olivelli
+1 (binding) - built from sources, JDK8 in MacOS - run pulsar standalone, smoke tests - verified checksum and digital signatures - I took a look to the Maven staging repository (verified that "integration tests jars" are present) Thanks for driving the release Enrico Il giorno mar 21 dic 2021 a

Re: [VOTE] Pulsar Release 2.7.4 Candidate 2

2021-12-21 Thread Enrico Olivelli
+1 (binding) - built from sources, JDK8 in MacOS - run pulsar standalone, smoke tests - verified checksum and digital signatures - I took a look to the Maven staging repository Thanks for driving the release Enrico Il giorno mar 21 dic 2021 alle ore 11:27 Nozomi Kurihara < nkuri...@apache.org>

Re: [DISCUSSION] PIP-124: Create init subscription before sending message to DLQ

2021-12-21 Thread Zike Yang
Thanks for your review Michael. On Tue, Dec 21, 2021 at 5:48 AM Michael Marshall wrote: > > Thanks for creating this PIP Zike Yang. This sounds like an important > improvement. I have a couple thoughts. > > 1. Does the `retryLetterTopic` have the same problem with immediate > message expiration?

Re: [VOTE] Apache Pulsar 2.8.2 candidate 2

2021-12-21 Thread Enrico Olivelli
Il giorno mar 21 dic 2021 alle ore 11:49 Shivji Kumar Jha < shiv4...@gmail.com> ha scritto: > Hi LinLin, > > Log4j version 2.16.0 has DDoS possibilities in some cases [1] . Can we move > to Log4j 2.17.0 in 2.8.2? > is it included https://github.com/apache/pulsar/tree/v2.8.2-candidate-2 Enrico

Re: [VOTE] Apache Pulsar 2.8.2 candidate 2

2021-12-21 Thread Shivji Kumar Jha
Hi LinLin, Log4j version 2.16.0 has DDoS possibilities in some cases [1] . Can we move to Log4j 2.17.0 in 2.8.2? Apache Log4j2 versions 2.0-alpha1 through 2.16.0 (excluding 2.12.3) did not > protect from uncontrolled recursion from self-referential lookups. This > allows an attacker with control

Re: [DISCUSS] Release Pulsar 2.7.4

2021-12-21 Thread Shivji Kumar Jha
Hi Pulsar Team, 2.16.0 is prone to DDoS attacks [1]. Is it possible to move to 2.17.0 in pulsar 2.7.4 ? [1] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-45105 Regards, Shivji Kumar Jha http://www.shivjijha.com/ +91 8884075512 On Tue, 21 Dec 2021 at 02:46, Michael Marshall wrote:

Re: [VOTE] Pulsar Release 2.7.4 Candidate 2

2021-12-21 Thread Nozomi Kurihara
+1 (binding) - Verified checksums and signatures - Verified license headers by apache rat - Built the source - Ran standalone, producer, and consumer - Verified functions, cassandra connectors and stateful functions Thanks for driving the release. Nozomi 2021年12月21日(火) 18:34 guo jiwei : > This

[VOTE] Pulsar Release 2.7.4 Candidate 2

2021-12-21 Thread guo jiwei
This is the second release candidate for Apache Pulsar, version 2.7.4. It fixes the following issues: https://github.com/apache/pulsar/pulls?q=is%3Apr+is%3Aopen+label%3Arelease%2F2.7.4+is%3Apr+ Release note: https://github.com/apache/pulsar/pull/13391 *** Please download, test and vote on this r

RE: [VOTE] Apache Pulsar 2.8.2 candidate 2

2021-12-21 Thread Masahiro Sakamoto
+1 (binding) - Checked checksums and signatures - Checked license headers using Apache Rat - Compiled the source - Ran the standalone server - Confirmed that producer and consumer work properly - Validated functions, connectors, and stateful functions Regards, Masahiro Sakamoto Yahoo Japan Corp.

PR merge

2021-12-21 Thread zhangao
Hi, Please take a look at this two issue, it has been for a lone time. I think it's ready to merge. https://github.com/apache/pulsar/pull/11893 https://github.com/apache/pulsar/pull/12025

OSX packages status....broken ?!?

2021-12-21 Thread Enrico Olivelli
Hello, According to the release procedure I am trying to build the OSX packages but the build fails. This is the issue: https://github.com/apache/pulsar/issues/13429 Apart from fixing the problem itself, I wonder if it would be better to create a CI job that periodically runs such a procedure. T