Re: [VOTE] Pulsar Release 2.10.1 Candidate 1

[Enrico Olivelli]

Hang,

Il giorno lun 27 giu 2022 alle ore 03:55 Hang Chen
 ha scritto:
>
> Hi Penghui,
>  Thanks for your great work!
>
> I found two problems with this candidate, I'm not sure whether it will
> block this release.
>   - There are some CVEs in this candidate. Please refer to:
> https://github.com/apache/pulsar/runs/6854904384?check_suite_focus=true
>   - When I import `pulsar-broker` and `managed-ledger` into my
> project, I found the project build failed with the following
> exceptions.

This is not a regression of 2.10.1, we should not change it.
The solution is to import the Pulsar "BOM" (Bills of Materials) like
we do in Pulsar Adapters
https://github.com/apache/pulsar-adapters/blob/master/pom.xml#L163

Enrico

>
> ```
> [ERROR] Failed to execute goal on project test_pulsar_lib: Could not
> resolve dependencies for project
> org.example:test_pulsar_lib:jar:1.0-SNAPSHOT: Failed to collect
> dependencies for org.example:test_pulsar_lib:jar:1.0-SNAPSHOT: Could
> not resolve version conflict among
> [org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> org.apache.bookkeeper:stream-storage-server:jar:4.14.5 ->
> org.apache.bookkeeper:stream-storage-java-client:jar:4.14.5 ->
> io.grpc:grpc-core:jar:1.42.1,
> org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> org.apache.bookkeeper:stream-storage-server:jar:4.14.5 ->
> org.apache.bookkeeper:stream-storage-java-client:jar:4.14.5 ->
> io.grpc:grpc-testing:jar:1.42.1 ->
> io.grpc:grpc-core:jar:[1.42.1,1.42.1],
> org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-grpclb:jar:1.45.1 ->
> io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-netty:jar:1.45.1 ->
> io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-rls:jar:1.45.1 ->
> io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-services:jar:1.45.1 ->
> io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-xds:jar:1.45.1 ->
> io.grpc:grpc-core:jar:1.45.1,
> org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-xds:jar:1.45.1 ->
> io.grpc:grpc-netty-shaded:jar:[1.45.1,1.45.1] ->
> io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> org.apache.pulsar:managed-ledger:jar:2.10.1 ->
> org.apache.pulsar:pulsar-metadata:jar:2.10.1 ->
> io.etcd:jetcd-core:jar:0.5.11 -> io.etcd:jetcd-common:jar:0.5.11 ->
> io.grpc:grpc-core:jar:1.41.0,
> org.apache.pulsar:managed-ledger:jar:2.10.1 ->
> org.apache.pulsar:pulsar-metadata:jar:2.10.1 ->
> io.etcd:jetcd-core:jar:0.5.11 -> io.grpc:grpc-core:jar:1.41.0] ->
> [Help 1]
>
> ```
>
> Thanks,
> Hang
>
> PengHui Li  于2022年6月27日周一 08:51写道：
> >
> > Hi all,
> >
> > Thanks for all your verification for 2.10.1
> >
> > We currently have 2 (+1) bindings and 5 (+1) non-bindings.
> > We need one more (+1) binding to unblock the 2.10.1 release.
> > Please help validate.
> >
> > Thanks,
> > Penghui
> >
> > On Thu, Jun 16, 2022 at 12:43 PM Shusuke Tsuda 
> > wrote:
> >
> > > +1 (non-binding)
> > >
> > > - verified checksums and signatures
> > > - build from source
> > > - verified pub/sub and java functions
> > > - verified stateful functions
> > >
> > > Thanks,
> > > Shusuke Tsuda
> > >
> > > -Original Message-
> > > From: PengHui Li 
> > > Sent: Monday, June 13, 2022 8:43 PM
> > > To: Dev 
> > > Subject: [VOTE] Pulsar Release 2.10.1 Candidate 1
> > >
> > > This is the first release candidate for Apache Pulsar 2.10.1
> > >
> > > It fixes the following issues:
> > >
> > > https://github.com/apache/pulsar/pulls?q=is%3Amerged+is%3Apr+label%3Arelease%2F2.10.1+
> > >
> > > *** Please download, test and vote on this release. This vote will stay
> > > open for at least 72 hours ***
> > >
> > > Note that we are voting upon the source (tag), binaries are provided for
> > > convenience.
> > >
> > > Source and binary files:
> > >
> > > https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fpulsar%2Fpulsar-2.10.1-candidate-1%2F&data=05%7C01%7Cshustsud%40yahoo-corp.jp%7C5ddcaa29fc224b6f09e608da4d31e6a8%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7

Re: [VOTE] Pulsar Release 2.9.3 Candidate 1

[Enrico Olivelli]

Hang,

Il giorno lun 27 giu 2022 alle ore 04:35 Hang Chen
 ha scritto:
>
> Hi Mattison,
> Thanks for your great work!
>
> I found one problem with this candidate.
> - When I import `pulsar-broker` and `managed-ledger` into my
> project, I found the project build failed with the following
> exceptions.

You can fix this problem by importing the Pulsar BOM like we do in
Pulsar Adapters
https://github.com/apache/pulsar-adapters/blob/master/pom.xml#L163

Enrico

>
> ```
> [ERROR] Failed to execute goal on project test_pulsar_lib: Could not
> resolve dependencies for project
> org.example:test_pulsar_lib:jar:1.0-SNAPSHOT: Failed to collect
> dependencies for org.example:test_pulsar_lib:jar:1.0-SNAPSHOT: Could
> not resolve version conflict among
> [org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
> org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
> org.apache.bookkeeper:stream-storage-server:jar:4.14.5 ->
> org.apache.bookkeeper:stream-storage-java-client:jar:4.14.5 ->
> io.grpc:grpc-core:jar:1.42.1,
> org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
> org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
> org.apache.bookkeeper:stream-storage-server:jar:4.14.5 ->
> org.apache.bookkeeper:stream-storage-java-client:jar:4.14.5 ->
> io.grpc:grpc-testing:jar:1.42.1 ->
> io.grpc:grpc-core:jar:[1.42.1,1.42.1],
> org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
> org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
> org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-grpclb:jar:1.45.1 ->
> io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
> org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-netty:jar:1.45.1 ->
> io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
> org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-rls:jar:1.45.1 ->
> io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
> org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-services:jar:1.45.1 ->
> io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
> org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-xds:jar:1.45.1 ->
> io.grpc:grpc-core:jar:1.45.1,
> org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
> org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-xds:jar:1.45.1 ->
> io.grpc:grpc-netty-shaded:jar:[1.45.1,1.45.1] ->
> io.grpc:grpc-core:jar:[1.45.1,1.45.1]] -> [Help 1]
> ```
>
> There is a PR: https://github.com/apache/pulsar/pull/16109 removed
> pulsar-zookeeper-utils, which fixes this issue. But in branch 2.9, we
> still need this module, we can not cherry-pick this pr into
> branch-2.9. We can exclude the `io.grpc:grpc-core` in one dependency
> to fix this issue.
>
> Thanks,
> Hang
>
> PengHui Li  于2022年6月27日周一 09:51写道：
> >
> > +1 (binding)
> >
> > - verify gpg signatures
> > - run standalone
> > - pub/sub with pulsar-perf
> > - verify connector (Cassandra)
> > - verify stateful function
> > - build image and test internally for the integration tests (all the tests
> > get passed)
> >
> > @mattison It's better to upload the image under your org first so that we
> > can verify the image
> >
> > Thanks,
> > Penghui
> >
> > On Tue, Jun 21, 2022 at 9:24 AM mattison chao 
> > wrote:
> >
> > > This is the first release candidate for Apache Pulsar, version 2.9.3.
> > >
> > > It fixes the following issues:
> > >
> > > https://github.com/apache/pulsar/pulls?q=is%3Amerged+is%3Apr+label%3Arelease%2F2.9.3+
> > >
> > > *** Please download, test and vote on this release. This vote will stay
> > > open
> > > for at least 72 hours ***
> > >
> > > Note that we are voting upon the source (tag), binaries are provided for
> > > convenience.
> > >
> > > Source and binary files:
> > > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.9.3-candidate-1/
> > >
> > > SHA-512 checksums:
> > >
> > >
> > > cbc9efbb6b25704e306442b1f45ab7e309fb7c0476c380997b4541b66e4709baeff7c67107a5c68faefe40a32a559755df54a732b21b5c689612307e688f1bcb
> > >  ./apache-pulsar-2.9.3-bin.tar.gz
> > >
> > >
> > > 35795b824d0775ffcd426ff5b86858148fd55c3d465edc52f23aa938daefcc50dbffa962f26e03e6fa3a77a47c3b5f19f98e1a1fbd5a98f6fe6d800223a8ead8
> > >  ./apache-pulsar-2.9.3-src.tar.gz
> > >
> > > Maven staging repo:
> > > https://repository.apache.org/content/repositories/orgapachepulsar-1158/
> > >
> > > The tag to be voted upon:
> > > v2.9.3-candidate-1 (dd9a5f1f91651b634600f66c53dcc6ad855fb669)
> > > https://github.com/apache/pulsar/releases/tag/v2.9.3-candidate-1
> > >
> > > Pulsar's KEYS file containing PGP keys we use to sign the release:
> > > https://dist.

[DISCUSS] PIP-181: Pulsar Shell

[Nicolò Boschi]

Hi all,

I opened a new PIP about Pulsar CLI tools.
Looking forward to seeing comments and suggestions.

PIP: https://github.com/apache/pulsar/issues/16250

I posted a short video that shows how the new tool will work:
https://user-images.githubusercontent.com/23314389/176125261-35e123a1-1826-4553-b912-28d00914c0e4.mp4




## Motivation

Currently Pulsar comes with a couple of utility scripts with the goal of
managing an existing cluster, test behaviours and verify performances:
these tools are available as SH script inside the `bin` directory.
The `pulsar-admin` is the CLI tool supposed to help users and operators to
configure the system, operate over policies, install functions and much
else.

This proposal basically aims to solve two different problems:

1. `pulsar-admin` is terribly slow. Every time the script is triggered, a
new JVM process is spawned. The JVM process creation is heavy and most of
the time is spent by the JVM initialization process. A very common use case
for cluster operators is to create scripts with several commands with the
goal of initialize the cluster, initialize a specific tenant (namespaces,
topics, policies, functions..); in this case, one JVM is initialized for
each scripts leads to waste of time and resources.

2. User experience. The current design of the Pulsar CLIs can be improved.
There are a couple of aspects that may be annoying for a user and can
discourage a user to use Pulsar.
1. Poking around available commands and options in a CLI tool
(`pulsar-admin` for instance, but it's the same for `pulsar-perf` and
`pulsar-client`) is slow and hard. In order to discover commands and
options you need to use `-h` option and, since the performance issue
pointed at 1., it can be annoying and time-consuming. Autocomplete feature
could be a real game-changer in this context.
2. Different CLI tools. There are a couple of different shell scripts.
They have different goals and it's okay to keep them separated. However,
they raise a barrier for a non Pulsar expert that doesn't have a convenient
entry-point.

## Goal

Address all the issues in the previous section with a single solution.

## API Changes

A new shell script `bin/pulsar-shell` will be introduced. `bin/pulsar
shell` could be a valid alternative but it's not findable for a newbie user
since no direct file exists.

## Implementation

### Concepts
The new script `pulsar-shell` will differ from the existing for the
following reasons:

1. It's a shell. When you start it, it will wait for commands to be
executed. After the command has been executed, despite its result, the
shell session will not be destroyed and it will wait for another command.
2. Unifies all the CLI scripts. In `pulsar-shell` you'll be able to run all
the existing CLI commands. This will be done in a way that when a new
command/option is added, the pulsar-shell will be updated accordingly.
3. It comes with sophisticated autocompletion and command history to highly
improve the UX.
4. Performance. Since JVM is initiated once, it will gain on performance
thanks to the JVM warmup and internal libraries bootstraps.
5. It will accept a file or a list of commands (parameter and stdin) to
start a shell, run the commands and close the shell. We'll call it the
`non-interactive` mode and it will ease the cluster operations automations.

Note that existing tools will not be removed/changed.

### Implementation

The shell implementation will be developed in Java, using a well-known
library called [JLine 3](https://github.com/jline/jline3) for the shell
support.
There will be a new main class that will extends the existing class tools
(e.g. `PulsarAdminTool`)

 Configuration
The configuration file taken by default will be `client.conf`, like current
CLI tools. The env setup will be exactly the same as for `pulsar-admin` and
`pulsar-client`.

 Autocompletion
JLine3 has great support for autocompletion. The shell java class will
translate current `JCommander` tools to the JLine3 completion API. This
will ensure that all APIs will be up-to-date and covered.

 History
JLine3 has built-in support for history, both in-memory and persisted (on
local FS). By default the history will be persisted in the user home
directory. This feature can be turned off for security concerns with a
configuration property.

 List of tools
The proposal is to get the following tools:
- bin/pulsar-admin
- bin/pulsar-client
- bin/pulsar-perf


## Rejected Alternatives

- Create a shell mode for `pulsar-admin`. This won't be a flexible solution
because then we may need the same for other tools. Also it doesn't cover
the CLI unification part.
- Create a brand-new shell mode. Not compatible with current CLI and
another tool to maintain.
- Use another shell library. There are a couple of valid alternatives to
JLine3 but it's wide-spread and the autocompletion API is much more
flexible than others since it's not done with annotations (the shell needs
to translate current JCommander

[GitHub] [pulsar-site] MMirelli opened a new pull request, #126: Update out-of-sync link to get openssl.cnf

[GitBox]

MMirelli opened a new pull request, #126:
URL: https://github.com/apache/pulsar-site/pull/126

   Static files were remove from pulsar repo by 
https://github.com/apache/pulsar/pull/15636.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-helm-chart] abtqian commented on issue #267: helm chart for pulsar version 2.10.0

[GitBox]

abtqian commented on issue #267:
URL: 
https://github.com/apache/pulsar-helm-chart/issues/267#issuecomment-1168457653

   @michaeljmarshall It seems that PR 
https://github.com/apache/pulsar-helm-chart/pull/266 has updated security 
config for zk and bookie. For pulsar broker, does it need the same config? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [VOTE] Pulsar Release 2.10.1 Candidate 1

[Hang Chen]

Thanks for Enrico's help, after importing the following dependency, it
works for me now.
```

org.apache.pulsar
pulsar
${pulsar.version}
import
pom


```

+1 (binding)

- Build from source code, and run the license check
- Import the published jar into my project, and works for me
- Run pulsar-perf produce and consumer on the standalone cluster
- Run pulsar-io-lakehouse connector on the standalone cluster

Thanks,
Hang

Enrico Olivelli  于2022年6月28日周二 15:55写道：
>
> Hang,
>
> Il giorno lun 27 giu 2022 alle ore 03:55 Hang Chen
>  ha scritto:
> >
> > Hi Penghui,
> >  Thanks for your great work!
> >
> > I found two problems with this candidate, I'm not sure whether it will
> > block this release.
> >   - There are some CVEs in this candidate. Please refer to:
> > https://github.com/apache/pulsar/runs/6854904384?check_suite_focus=true
> >   - When I import `pulsar-broker` and `managed-ledger` into my
> > project, I found the project build failed with the following
> > exceptions.
>
> This is not a regression of 2.10.1, we should not change it.
> The solution is to import the Pulsar "BOM" (Bills of Materials) like
> we do in Pulsar Adapters
> https://github.com/apache/pulsar-adapters/blob/master/pom.xml#L163
>
> Enrico
>
> >
> > ```
> > [ERROR] Failed to execute goal on project test_pulsar_lib: Could not
> > resolve dependencies for project
> > org.example:test_pulsar_lib:jar:1.0-SNAPSHOT: Failed to collect
> > dependencies for org.example:test_pulsar_lib:jar:1.0-SNAPSHOT: Could
> > not resolve version conflict among
> > [org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> > org.apache.bookkeeper:stream-storage-server:jar:4.14.5 ->
> > org.apache.bookkeeper:stream-storage-java-client:jar:4.14.5 ->
> > io.grpc:grpc-core:jar:1.42.1,
> > org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> > org.apache.bookkeeper:stream-storage-server:jar:4.14.5 ->
> > org.apache.bookkeeper:stream-storage-java-client:jar:4.14.5 ->
> > io.grpc:grpc-testing:jar:1.42.1 ->
> > io.grpc:grpc-core:jar:[1.42.1,1.42.1],
> > org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> > org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-grpclb:jar:1.45.1 ->
> > io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> > org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-netty:jar:1.45.1 ->
> > io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> > org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-rls:jar:1.45.1 ->
> > io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> > org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-services:jar:1.45.1 ->
> > io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> > org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-xds:jar:1.45.1 ->
> > io.grpc:grpc-core:jar:1.45.1,
> > org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-xds:jar:1.45.1 ->
> > io.grpc:grpc-netty-shaded:jar:[1.45.1,1.45.1] ->
> > io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> > org.apache.pulsar:managed-ledger:jar:2.10.1 ->
> > org.apache.pulsar:pulsar-metadata:jar:2.10.1 ->
> > io.etcd:jetcd-core:jar:0.5.11 -> io.etcd:jetcd-common:jar:0.5.11 ->
> > io.grpc:grpc-core:jar:1.41.0,
> > org.apache.pulsar:managed-ledger:jar:2.10.1 ->
> > org.apache.pulsar:pulsar-metadata:jar:2.10.1 ->
> > io.etcd:jetcd-core:jar:0.5.11 -> io.grpc:grpc-core:jar:1.41.0] ->
> > [Help 1]
> >
> > ```
> >
> > Thanks,
> > Hang
> >
> > PengHui Li  于2022年6月27日周一 08:51写道：
> > >
> > > Hi all,
> > >
> > > Thanks for all your verification for 2.10.1
> > >
> > > We currently have 2 (+1) bindings and 5 (+1) non-bindings.
> > > We need one more (+1) binding to unblock the 2.10.1 release.
> > > Please help validate.
> > >
> > > Thanks,
> > > Penghui
> > >
> > > On Thu, Jun 16, 2022 at 12:43 PM Shusuke Tsuda 
> > > wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > - verified checksums and signatures
> > > > - build from source
> > > > - verified pub/sub and java functions
> > > > - verified stateful functions
> > > >
> > > > Thanks,
> > > > Shusuke Tsuda
> > > >
> > > > -Original Message-
> > > > From: PengHui Li 
> > > > Sent: Monday, June 13, 2022 8:43 PM
> > > > To: Dev 
> > > > Subject: [VOTE] Pulsar Release 2.10.1 Candidate 1
> > > >
> > > > This is the first release

Re: [VOTE] Pulsar Release 2.9.3 Candidate 1

[mattison chao]

>
> @mattison It's better to upload the image under your org first so that we
> can verify the image


Sure, I will do it later.

Best,
Mattison

On Tue, 28 Jun 2022 at 15:57, Enrico Olivelli  wrote:

> Hang,
>
> Il giorno lun 27 giu 2022 alle ore 04:35 Hang Chen
>  ha scritto:
> >
> > Hi Mattison,
> > Thanks for your great work!
> >
> > I found one problem with this candidate.
> > - When I import `pulsar-broker` and `managed-ledger` into my
> > project, I found the project build failed with the following
> > exceptions.
>
> You can fix this problem by importing the Pulsar BOM like we do in
> Pulsar Adapters
> https://github.com/apache/pulsar-adapters/blob/master/pom.xml#L163
>
> Enrico
>
> >
> > ```
> > [ERROR] Failed to execute goal on project test_pulsar_lib: Could not
> > resolve dependencies for project
> > org.example:test_pulsar_lib:jar:1.0-SNAPSHOT: Failed to collect
> > dependencies for org.example:test_pulsar_lib:jar:1.0-SNAPSHOT: Could
> > not resolve version conflict among
> > [org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
> > org.apache.bookkeeper:stream-storage-server:jar:4.14.5 ->
> > org.apache.bookkeeper:stream-storage-java-client:jar:4.14.5 ->
> > io.grpc:grpc-core:jar:1.42.1,
> > org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
> > org.apache.bookkeeper:stream-storage-server:jar:4.14.5 ->
> > org.apache.bookkeeper:stream-storage-java-client:jar:4.14.5 ->
> > io.grpc:grpc-testing:jar:1.42.1 ->
> > io.grpc:grpc-core:jar:[1.42.1,1.42.1],
> > org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> > org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-grpclb:jar:1.45.1 ->
> > io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> > org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-netty:jar:1.45.1 ->
> > io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> > org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-rls:jar:1.45.1 ->
> > io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> > org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-services:jar:1.45.1 ->
> > io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> > org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-xds:jar:1.45.1 ->
> > io.grpc:grpc-core:jar:1.45.1,
> > org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-xds:jar:1.45.1 ->
> > io.grpc:grpc-netty-shaded:jar:[1.45.1,1.45.1] ->
> > io.grpc:grpc-core:jar:[1.45.1,1.45.1]] -> [Help 1]
> > ```
> >
> > There is a PR: https://github.com/apache/pulsar/pull/16109 removed
> > pulsar-zookeeper-utils, which fixes this issue. But in branch 2.9, we
> > still need this module, we can not cherry-pick this pr into
> > branch-2.9. We can exclude the `io.grpc:grpc-core` in one dependency
> > to fix this issue.
> >
> > Thanks,
> > Hang
> >
> > PengHui Li  于2022年6月27日周一 09:51写道：
> > >
> > > +1 (binding)
> > >
> > > - verify gpg signatures
> > > - run standalone
> > > - pub/sub with pulsar-perf
> > > - verify connector (Cassandra)
> > > - verify stateful function
> > > - build image and test internally for the integration tests (all the
> tests
> > > get passed)
> > >
> > > @mattison It's better to upload the image under your org first so that
> we
> > > can verify the image
> > >
> > > Thanks,
> > > Penghui
> > >
> > > On Tue, Jun 21, 2022 at 9:24 AM mattison chao  >
> > > wrote:
> > >
> > > > This is the first release candidate for Apache Pulsar, version 2.9.3.
> > > >
> > > > It fixes the following issues:
> > > >
> > > >
> https://github.com/apache/pulsar/pulls?q=is%3Amerged+is%3Apr+label%3Arelease%2F2.9.3+
> > > >
> > > > *** Please download, test and vote on this release. This vote will
> stay
> > > > open
> > > > for at least 72 hours ***
> > > >
> > > > Note that we are voting upon the source (tag), binaries are provided
> for
> > > > convenience.
> > > >
> > > > Source and binary files:
> > > >
> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.9.3-candidate-1/
> > > >
> > > > SHA-512 checksums:
> > > >
> > > >
> > > >
> cbc9efbb6b25704e306442b1f45ab7e309fb7c0476c380997b4541b66e4709baeff7c67107a5c68faefe40a32a559755df54a732b21b5c689612307e688f1bcb
> > > >  ./apache-pulsar-2.9.3-bin.tar.gz
> > > >
> > > >
> > > >
> 35795b824d0775ffcd426ff5b86858148fd55c3d465edc52f23aa938daefcc50dbffa962f26e03e6fa3a77a47c3b5f19f9

Re: [VOTE] Pulsar Release 2.9.3 Candidate 1

[mattison chao]

>
> I found one problem with this candidate.
> - When I import `pulsar-broker` and `managed-ledger` into my
> project, I found the project build failed with the following
> exceptions.


Hi, Hang.

According to the release 2.10.1 vote email, it looks like this problem gets
solved and won't block the release. Right?

Best,
Mattison

On Tue, 28 Jun 2022 at 20:21, mattison chao  wrote:

> @mattison It's better to upload the image under your org first so that we
>> can verify the image
>
>
> Sure, I will do it later.
>
> Best,
> Mattison
>
> On Tue, 28 Jun 2022 at 15:57, Enrico Olivelli  wrote:
>
>> Hang,
>>
>> Il giorno lun 27 giu 2022 alle ore 04:35 Hang Chen
>>  ha scritto:
>> >
>> > Hi Mattison,
>> > Thanks for your great work!
>> >
>> > I found one problem with this candidate.
>> > - When I import `pulsar-broker` and `managed-ledger` into my
>> > project, I found the project build failed with the following
>> > exceptions.
>>
>> You can fix this problem by importing the Pulsar BOM like we do in
>> Pulsar Adapters
>> https://github.com/apache/pulsar-adapters/blob/master/pom.xml#L163
>>
>> Enrico
>>
>> >
>> > ```
>> > [ERROR] Failed to execute goal on project test_pulsar_lib: Could not
>> > resolve dependencies for project
>> > org.example:test_pulsar_lib:jar:1.0-SNAPSHOT: Failed to collect
>> > dependencies for org.example:test_pulsar_lib:jar:1.0-SNAPSHOT: Could
>> > not resolve version conflict among
>> > [org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
>> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
>> > org.apache.bookkeeper:stream-storage-server:jar:4.14.5 ->
>> > org.apache.bookkeeper:stream-storage-java-client:jar:4.14.5 ->
>> > io.grpc:grpc-core:jar:1.42.1,
>> > org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
>> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
>> > org.apache.bookkeeper:stream-storage-server:jar:4.14.5 ->
>> > org.apache.bookkeeper:stream-storage-java-client:jar:4.14.5 ->
>> > io.grpc:grpc-testing:jar:1.42.1 ->
>> > io.grpc:grpc-core:jar:[1.42.1,1.42.1],
>> > org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
>> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
>> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-core:jar:[1.45.1,1.45.1],
>> > org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
>> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
>> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-grpclb:jar:1.45.1 ->
>> > io.grpc:grpc-core:jar:[1.45.1,1.45.1],
>> > org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
>> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
>> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-netty:jar:1.45.1 ->
>> > io.grpc:grpc-core:jar:[1.45.1,1.45.1],
>> > org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
>> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
>> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-rls:jar:1.45.1 ->
>> > io.grpc:grpc-core:jar:[1.45.1,1.45.1],
>> > org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
>> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
>> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-services:jar:1.45.1 ->
>> > io.grpc:grpc-core:jar:[1.45.1,1.45.1],
>> > org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
>> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
>> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-xds:jar:1.45.1 ->
>> > io.grpc:grpc-core:jar:1.45.1,
>> > org.apache.pulsar:pulsar-broker:jar:2.9.3 ->
>> > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.9.3 ->
>> > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-xds:jar:1.45.1 ->
>> > io.grpc:grpc-netty-shaded:jar:[1.45.1,1.45.1] ->
>> > io.grpc:grpc-core:jar:[1.45.1,1.45.1]] -> [Help 1]
>> > ```
>> >
>> > There is a PR: https://github.com/apache/pulsar/pull/16109 removed
>> > pulsar-zookeeper-utils, which fixes this issue. But in branch 2.9, we
>> > still need this module, we can not cherry-pick this pr into
>> > branch-2.9. We can exclude the `io.grpc:grpc-core` in one dependency
>> > to fix this issue.
>> >
>> > Thanks,
>> > Hang
>> >
>> > PengHui Li  于2022年6月27日周一 09:51写道：
>> > >
>> > > +1 (binding)
>> > >
>> > > - verify gpg signatures
>> > > - run standalone
>> > > - pub/sub with pulsar-perf
>> > > - verify connector (Cassandra)
>> > > - verify stateful function
>> > > - build image and test internally for the integration tests (all the
>> tests
>> > > get passed)
>> > >
>> > > @mattison It's better to upload the image under your org first so
>> that we
>> > > can verify the image
>> > >
>> > > Thanks,
>> > > Penghui
>> > >
>> > > On Tue, Jun 21, 2022 at 9:24 AM mattison chao <
>> mattisonc...@apache.org>
>> > > wrote:
>> > >
>> > > > This is the first release candidate for Apache Pulsar, version
>> 2.9.3.
>> > > >
>> > > > It fixes the following issues:
>> > > >
>> > > >
>> https://github.com/apache/pulsar/pulls?q=is%3Amerged+is%3Apr+label%3Arelease%2F2.9.3+
>> > > >
>> > > > *** Please download, test and vote on this release. This vote will
>> stay
>> > > > open
>> > > > for at least 72 hours ***
>> > > >
>> > > > Note that we are voting upon t

Re: [DISCUSS] Reject partitioned topic creation when the topic name contains the `partition` keyword.

[mattison chao]

I hope everyone can express their views. :)

Best,
Mattison

Re: [VOTE] Pulsar Release 2.10.1 Candidate 1

[PengHui Li]

Thanks, Hang and Enrico.

Close the vote with 3 (+1) bindings, 5 (+1) non-bindings.

I will continue the release process.

Penghui

On Tue, Jun 28, 2022 at 6:15 PM Hang Chen  wrote:

> Thanks for Enrico's help, after importing the following dependency, it
> works for me now.
> ```
> 
> org.apache.pulsar
> pulsar
> ${pulsar.version}
> import
> pom
> 
>
> ```
>
> +1 (binding)
>
> - Build from source code, and run the license check
> - Import the published jar into my project, and works for me
> - Run pulsar-perf produce and consumer on the standalone cluster
> - Run pulsar-io-lakehouse connector on the standalone cluster
>
> Thanks,
> Hang
>
> Enrico Olivelli  于2022年6月28日周二 15:55写道：
> >
> > Hang,
> >
> > Il giorno lun 27 giu 2022 alle ore 03:55 Hang Chen
> >  ha scritto:
> > >
> > > Hi Penghui,
> > >  Thanks for your great work!
> > >
> > > I found two problems with this candidate, I'm not sure whether it will
> > > block this release.
> > >   - There are some CVEs in this candidate. Please refer to:
> > >
> https://github.com/apache/pulsar/runs/6854904384?check_suite_focus=true
> > >   - When I import `pulsar-broker` and `managed-ledger` into my
> > > project, I found the project build failed with the following
> > > exceptions.
> >
> > This is not a regression of 2.10.1, we should not change it.
> > The solution is to import the Pulsar "BOM" (Bills of Materials) like
> > we do in Pulsar Adapters
> > https://github.com/apache/pulsar-adapters/blob/master/pom.xml#L163
> >
> > Enrico
> >
> > >
> > > ```
> > > [ERROR] Failed to execute goal on project test_pulsar_lib: Could not
> > > resolve dependencies for project
> > > org.example:test_pulsar_lib:jar:1.0-SNAPSHOT: Failed to collect
> > > dependencies for org.example:test_pulsar_lib:jar:1.0-SNAPSHOT: Could
> > > not resolve version conflict among
> > > [org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> > > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> > > org.apache.bookkeeper:stream-storage-server:jar:4.14.5 ->
> > > org.apache.bookkeeper:stream-storage-java-client:jar:4.14.5 ->
> > > io.grpc:grpc-core:jar:1.42.1,
> > > org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> > > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> > > org.apache.bookkeeper:stream-storage-server:jar:4.14.5 ->
> > > org.apache.bookkeeper:stream-storage-java-client:jar:4.14.5 ->
> > > io.grpc:grpc-testing:jar:1.42.1 ->
> > > io.grpc:grpc-core:jar:[1.42.1,1.42.1],
> > > org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> > > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> > > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> > > org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> > > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> > > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-grpclb:jar:1.45.1 ->
> > > io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> > > org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> > > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> > > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-netty:jar:1.45.1 ->
> > > io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> > > org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> > > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> > > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-rls:jar:1.45.1 ->
> > > io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> > > org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> > > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> > > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-services:jar:1.45.1 ->
> > > io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> > > org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> > > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> > > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-xds:jar:1.45.1 ->
> > > io.grpc:grpc-core:jar:1.45.1,
> > > org.apache.pulsar:pulsar-broker:jar:2.10.1 ->
> > > org.apache.pulsar:pulsar-zookeeper-utils:jar:2.10.1 ->
> > > io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-xds:jar:1.45.1 ->
> > > io.grpc:grpc-netty-shaded:jar:[1.45.1,1.45.1] ->
> > > io.grpc:grpc-core:jar:[1.45.1,1.45.1],
> > > org.apache.pulsar:managed-ledger:jar:2.10.1 ->
> > > org.apache.pulsar:pulsar-metadata:jar:2.10.1 ->
> > > io.etcd:jetcd-core:jar:0.5.11 -> io.etcd:jetcd-common:jar:0.5.11 ->
> > > io.grpc:grpc-core:jar:1.41.0,
> > > org.apache.pulsar:managed-ledger:jar:2.10.1 ->
> > > org.apache.pulsar:pulsar-metadata:jar:2.10.1 ->
> > > io.etcd:jetcd-core:jar:0.5.11 -> io.grpc:grpc-core:jar:1.41.0] ->
> > > [Help 1]
> > >
> > > ```
> > >
> > > Thanks,
> > > Hang
> > >
> > > PengHui Li  于2022年6月27日周一 08:51写道：
> > > >
> > > > Hi all,
> > > >
> > > > Thanks for all your verification for 2.10.1
> > > >
> > > > We currently have 2 (+1) bindings and 5 (+1) non-bindings.
> > > > We need one more (+1) binding to unblock the 2.10.1 release.
> > > > Please help validate.
> > > >
> > > > Thanks,
> > > > Penghui
> > > >
> > > > On Thu, Jun 16, 2022 at 12:43 PM Shusuke Tsuda <
> shust...@yahoo-corp.jp>
> > > > wrote:
> > > >
> > > > >

Re: [DISCUSS] Enable GitHub Discussions?

[Liu Yu]

Hi Dave Fisher, any thoughts on enabling GitHub Discussions? Thank you.

On 2022/06/27 02:41:12 Yu wrote:
> +1 for enabling the GitHub Discussion as I proposed before.
> 
> At that time, the main concern was not able to sync info between GitHub
> Discussion and Mailing List. Since the concern can be resolved now, we can
> make the most of GitHub Discussion as it's a powerful supplementary to
> community collaboration.
> 

Re: [DISCUSS] Reject partitioned topic creation when the topic name contains the `partition` keyword.

[Baodi Shi]

Hi,

I think the topic name of DLQ is a bit awkward now, should use 
`xxx-sub-name-DLQ-patition-0` instead of `xxx-sub-name-patition-0-DLQ`.


Thanks,
Baodi Shi

> On Jun 28, 2022, at 20:3314, mattison chao  wrote:
> 
> I hope everyone can express their views. :)
> 
> Best,
> Mattison

Re: [DISCUSS] Enable GitHub Discussions?

[Dave Fisher]

Hi -

Now that the ASF has a notifier on GitHub Discussions I have no objection to 
using discussions as long as these are visible on dev@pulsar.apache.org 
 mailing list.

> From the previous thread it seems that the Pulsar community has already
> multiple channels to catch up for contributors, while I only watch at
> mailing list and GitHub issues & PRs

I think that where the notifications are sent is important. Pulsar contributors 
are supposed to be watching dev@pulsar.apache.org 
 and this should be the mailing list where 
discussions are visible to the community. I recommend that discussion 
notifications be sent there.

> Besides, currently dev@ is full of Git notifications while we may sort them
> out to issues@ as in Apache Kvrocks[5].

Regarding other notifications: the project is inconsistent. Also, please note 
that Pulsar does not have an issues@pulsar mailing list.

The main repository is using this default: 
https://gitbox.apache.org/schemes.cgi?pulsar 
 which differs from 
https://gitbox.apache.org/schemes.cgi?pulsar-site 


The overall notification scheme change is a new discussion which must be kept 
separate.

Regards,
Dave


> On Jun 25, 2022, at 12:31 PM, tison  wrote:
> 
> Hi,
> 
> There is a previous discussion at
> https://lists.apache.org/thread/83pst643h9cqcryo3zsjd240jmqzvn73.
> 
> I'd like to bump this thread since INFRA has implemented forward GitHub
> Discussions activities to mailing list[1][2][3].
> 
> It may be a proper time we revisit this topic. My major argument is:
> 
>> GitHub Discussion is friendly for GitHub users and lower the bar to
> participant in discussions. If we can sync all activities to mailing list,
> it should be a nice supplement.
> 
> You may take a look at Apache SkyWalking's practice[4] where it holds an
> easy-to-find entrypoint for participants while keeps all decisions in
> mailing list - VOTE and RESULT threads should happen on mailing list as it
> used to be.
> 
> At least, it can be a supplement of users@ with a Q&A category.
> 
> From the previous thread it seems that the Pulsar community has already
> multiple channels to catch up for contributors, while I only watch at
> mailing list and GitHub issues & PRs. If you find other channel as source
> of truth, I'm glad to learn and see whether we can keep as less source of
> truth as possible - ideally, single source (orthogonal sources) of truth.
> 
> Besides, currently dev@ is full of Git notifications while we may sort them
> out to issues@ as in Apache Kvrocks[5].
> 
> Best,
> tison.
> 
> [1] https://github.com/apache/infrastructure-p6/pull/1120
> [2] https://github.com/apache/infrastructure-github-discussions-notifier/
> [3]
> https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-Notificationsettingsforrepositories
> [4] https://github.com/apache/skywalking/discussions
> [5]
> https://github.com/apache/incubator-kvrocks/blob/dde438f04c577050e299cce981a17018f056c3b3/.asf.yaml#L46-L50

[GitHub] [pulsar-site] urfreespace commented on pull request #126: Update out-of-sync url to get openssl.cnf in TLS transport documentation

[GitBox]

urfreespace commented on PR #126:
URL: https://github.com/apache/pulsar-site/pull/126#issuecomment-1169447282

   hi @MMirelli, thanks for your contribution, but please make pr about docs 
change in repo `pulsar` https://github.com/apache/pulsar/tree/master/site2, 
this repo `pulsar-site` will auto-sync docs change from the repo `pulsar`, We 
keep the docs maintained at the `pulsar` repo side. Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

[Haiting Jiang]

Hi Penghui & Asaf:

Please allow me to provide some more detailes about **metadata** 
synchronization 
between source topic and shadow topic.

1.When shadow topic initializes, it will read from metadata store path 
"/managed-ledgers/{source_topic_ledger_name}", which contains all the managed 
ledger info. We don't 
need to read the  ledger information from source topic broker.

2. When shadow topic received new message from replicator, if the ledger id of 
the message
 is the same as the last ledger, it just updates the LAC. If not, it will 
update ledger list from metadata, 
and then open the new ledger handle and update the LAC.

As for the copy itself and add shadow message id in CommandSend, it mostly 
serves the purpose 
of filling the EntryCache.

Thanks, 
Haiting

On 2022/06/23 02:08:46 PengHui Li wrote:
> > One question comes to mind here: Why not simply read the ledger information
> from original topic, without copy?
> 
> I think this is a good idea.
> 
> Penghui
> On Jun 22, 2022, 23:57 +0800, dev@pulsar.apache.org, wrote:
> >
> > One question comes to mind here: Why not simply read the ledger information
> > from original topic, without copy?
> 

[GitHub] [pulsar-site] horizonzy opened a new pull request, #127: Fix the doc link mismatch problem.

[GitBox]

horizonzy opened a new pull request, #127:
URL: https://github.com/apache/pulsar-site/pull/127

   fixes- https://github.com/apache/pulsar/issues/14928.
   
   Now the rest-api website url is: 
   
https://pulsar.apache.org/admin-rest-api/?version=@pulsar:version_number@&apiversion=v2#operation/healthcheck
   
   But we generate the url is: 
   
https://pulsar.apache.org/admin-rest-api#operation/getActiveBrokers?version=@pulsar:version_number@&apiVersion=v2
   
   We should generate the link to match rest api url.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

[PengHui Li]

Hi Haiting,

Thanks for the explanation. I'm clear for now.

Pulsar functions also can do such things by connecting data from one topic
to another topic.
But the difference is this proposal only copies the data to the cache of
another topic, and the data not
in the cache is also available by reading from ledgers.

And this approach also follows benefits compared with replicating data to
multiple "real" topics.

- reuse the topic metadata
- the same message ID which easy for troubleshooting

Just one question

>>>
```
message CommandSend { // ... // message id for shadow topic optional
   MessageIdData shadow_message_id = 9; }
```

Can we get the message ID from the replicated data to avoid introducing a
new command?
Or use a marker message to avoid broker-to-broker directly protobuf command
interaction.

Thanks,
Penghui

On Wed, Jun 29, 2022 at 10:31 AM Haiting Jiang 
wrote:

> Hi Penghui & Asaf:
>
> Please allow me to provide some more detailes about **metadata**
> synchronization
> between source topic and shadow topic.
>
> 1.When shadow topic initializes, it will read from metadata store path
> "/managed-ledgers/{source_topic_ledger_name}", which contains all the
> managed ledger info. We don't
> need to read the  ledger information from source topic broker.
>
> 2. When shadow topic received new message from replicator, if the ledger
> id of the message
>  is the same as the last ledger, it just updates the LAC. If not, it will
> update ledger list from metadata,
> and then open the new ledger handle and update the LAC.
>
> As for the copy itself and add shadow message id in CommandSend, it mostly
> serves the purpose
> of filling the EntryCache.
>
> Thanks,
> Haiting
>
> On 2022/06/23 02:08:46 PengHui Li wrote:
> > > One question comes to mind here: Why not simply read the ledger
> information
> > from original topic, without copy?
> >
> > I think this is a good idea.
> >
> > Penghui
> > On Jun 22, 2022, 23:57 +0800, dev@pulsar.apache.org, wrote:
> > >
> > > One question comes to mind here: Why not simply read the ledger
> information
> > > from original topic, without copy?
> >
>

[DISCUSS] Releasing Pulsar-client-go 0.9.0

[Rui Fu]

Hello Everyone:

I hope you’ve all been doing well. In the past couple of months, we have fixed 
many bugs and added some new features for the pulsar-client-go. For more 
information, refer to:

https://github.com/apache/pulsar-client-go/milestone/10?closed=1

For that reason, let us start the release of v0.9.0.

Best,

Rui Fu

Re: [DISCUSS] Releasing Pulsar-client-go 0.9.0

[Zixuan Liu]

+1

On 2022/06/29 06:06:01 Rui Fu wrote:
> Hello Everyone:
> 
> I hope you’ve all been doing well. In the past couple of months, we have 
> fixed many bugs and added some new features for the pulsar-client-go. For 
> more information, refer to:
> 
> https://github.com/apache/pulsar-client-go/milestone/10?closed=1
> 
> For that reason, let us start the release of v0.9.0.
> 
> Best,
> 
> Rui Fu
> 

[GitHub] [pulsar-manager] gurleen-gks opened a new issue, #470: Pulsar manager does not support x509 cert and key for tls

[GitBox]

gurleen-gks opened a new issue, #470:
URL: https://github.com/apache/pulsar-manager/issues/470

   Only supports keystore
   
   ```
   tls.enabled=false
   tls.keystore=keystore-file
   tls.keystore.password=keystore-file-password
   tls.hostname.verifier=false
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [DISCUSS] Reject partitioned topic creation when the topic name contains the `partition` keyword.

[Anon Hxy]

Hi Mattison,

I did a quick reading and verify the PR on standalone, and I have a
question:

If the topic name contains `-partition-`, it is expected to reject to
create a partitioned topic. However it will auto-create a non-partition
topic with `-partition-`.
But our `allowAutoTopicCreationType` is `partitioned`, that will confuse
the users I think.

So should we also reject creating non-partition topics with `-partition-`.

Thanks,
Xiaoyu Hou

mattison chao  于2022年6月28日周二 20:33写道：

> I hope everyone can express their views. :)
>
> Best,
> Mattison
>

[GitHub] [pulsar-site] urfreespace merged pull request #127: Fix the doc link mismatch problem.

[GitBox]

urfreespace merged PR #127:
URL: https://github.com/apache/pulsar-site/pull/127


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

[Haiting Jiang]

Hi Dave,

On 2022/06/23 03:59:35 Dave Fisher wrote:
> 
> On Jun 21, 2022, at 1:00 AM, Haiting Jiang  wrote:
> > 
> > Hi Pulsar community:
> > 
> > I open a pip to discuss "Shadow Topic, an alternative way to support 
> > readonly topic ownership."
> > 
> > Proposal Link: https://github.com/apache/pulsar/issues/16153
> > 
> > ---
> > 
> > ## Motivation
> > 
> > The motivation is the same as PIP-63[1], with a new broadcast use case of
> > supporting 100K subscriptions in a single topic.
> > 1. The bandwidth of a broker limits the number of subscriptions for a single
> >   topic.
> > 2. Subscriptions are competing for the network bandwidth on brokers. 
> > Different
> >   subscriptions might have different levels of severity.
> > 3. When synchronizing cross-city message reading, cross-city access needs to
> >   be minimized.
> > 4. [New] Broadcast with 100K subscriptions. There is a limitation of the
> >   subscription number of a single topic. It's tested by Hongjie from NTT Lab
> >   that with 40K subscriptions in a single topic, the client needs about 
> > 20min
> >   to start all client connections, and under 1 msg/s message producer rate,
> >   the average end to end latency is about 2.9s. And for 100K subscriptions,
> >   the time of start connection and E2E latency is beyond consideration.
> 
> Have you tested performance of two topics each with 40k subscriptions at the 
> same time in the same cluster?
> 
> I think that might simulate the notion of shadow topics in action and see if 
> much performance is actually gained by this notion of splitting.

I have not tested it yet. But as long as the bottle neck of this use case is 
not the metadata store, 
from the perspective of current architecture, the number of subscriptions 
pulsar can support can
be scaled horizontally. 

And also, the subscription limitation of one topic can be optimized, like 
Penghui did in github PR #16245, #16243,#16241.

> It seems to me that a better approach would be to have multiple local pulsar 
> clusters and balance the subscriptions between those.

With this approach, we have to replicate data storage. This is not tolerable 
for other use cases (like 1,2,3) when data flow is quite large.
And this is the reason why original PIP-63 dropped it as rejected alternatives, 
see 
https://github.com/apache/pulsar/wiki/PIP-63%3A-Readonly-Topic-Ownership-Support#rejected-alternatives

> I’m concerned that this shadow topic approach is adding new complexity to 
> Pulsar without a clear understanding of all of the impacts.
Yes, this is exactly the reason I prefer this new approach rather than split 
the original PR #11960 just for easier review.
This approach would be much more simpler and less impact on current 
implementation.  It would be appreciated if you can provide 
some more specific impacts.

Thanks,
Haiting

> Thanks,
> Dave
> 
> > 
> > However, it's too complicated to implement with original PIP-63 proposal, 
> > the
> > changed code is already over 3K+ lines, see PR#11960[2], and there are still
> > some problems left,
> > 1. The LAC in readonly topic is updated in a polling pattern, which 
> > increases
> >   the bookie load bookie.
> > 2. The message data of readonly topic won't be cached in broker. Increase 
> > the
> >   network usage between broker and bookie when there are more than one
> >   subscriber is tail-reading.
> > 3. All the subscriptions is managed in original writable-topic, so the 
> > support
> >   max subscription number is not scaleable.
> > 
> > This PIP tries to come up with a simpler solution to support readonly topic
> > ownership and solve the problems the previous PR left. The main idea of this
> > solution is to reuse the feature of geo-replication, but instead of
> > duplicating storage, it shares underlying bookie ledgers between different
> > topics.
> > 
> > ## Goal
> > 
> > The goal is to introduce **Shadow Topic** as a new type of topic to support
> > readonly topic ownership. Just as its name implies, a shadow topic is the
> > shadow of some normal persistent topic (let's call it source topic here). 
> > The
> > source topic and the shadow topic must have the same number of partitions or
> > both non-partitioned. Multiply shadow topics can be created from a source
> > topic.
> > 
> > Shadow topic shares the underlying bookie ledgers from its source topic. 
> > User
> > can't produce any messages to shadow topic directly and shadow topic don't
> > create any new ledger for messages, all messages in shadow topic come from
> > source topic.
> > 
> > Shadow topic have its own subscriptions and don't share with its source 
> > topic.
> > This means the shadow topic have its own cursor ledger to store persistent
> > mark-delete info for each persistent subscriptions.
> > 
> > The message sync procedure of shadow topic is supported by shadow 
> > replication,
> > which is very like geo-replication, with these difference:
> > 1. Geo-replication only works between topic with the same name in di

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

[Haiting Jiang]

Hi Penghui

On 2022/06/29 04:07:35 PengHui Li wrote:
> Hi Haiting,
> 
> Thanks for the explanation. I'm clear for now.
> 
> Pulsar functions also can do such things by connecting data from one topic
> to another topic.
> But the difference is this proposal only copies the data to the cache of
> another topic, and the data not
> in the cache is also available by reading from ledgers.
> 
> And this approach also follows benefits compared with replicating data to
> multiple "real" topics.
> 
> - reuse the topic metadata
> - the same message ID which easy for troubleshooting
> 
> Just one question
> 
> >>>
> ```
> message CommandSend { // ... // message id for shadow topic optional
>MessageIdData shadow_message_id = 9; }
> ```
> 
> Can we get the message ID from the replicated data to avoid introducing a
> new command?
> Or use a marker message to avoid broker-to-broker directly protobuf command
> interaction.
> 
Sorry for not wrote it clearly. CommandSend is not a new command. It's exactly 
the main 
command producer used to send message to broker. The only change is add a new 
field in it. 
The whole command proto would be like this:
```
message CommandSend {
required uint64 producer_id = 1;
required uint64 sequence_id = 2;
optional int32 num_messages = 3 [default = 1];
optional uint64 txnid_least_bits = 4 [default = 0];
optional uint64 txnid_most_bits = 5 [default = 0];

/// Add highest sequence id to support batch message with external sequence 
id
optional uint64 highest_sequence_id = 6 [default = 0];
optional bool is_chunk =7 [default = false];

// Specify if the message being published is a Pulsar marker or not
optional bool marker = 8 [default = false];

// message id for shadow topic
optional MessageIdData shadow_message_id = 9;
}
```
So there won't be any broker-to-broker directly protobuf command interactions.

Thanks,
Haiting

> Thanks,
> Penghui
> 
> On Wed, Jun 29, 2022 at 10:31 AM Haiting Jiang 
> wrote:
> 
> > Hi Penghui & Asaf:
> >
> > Please allow me to provide some more detailes about **metadata**
> > synchronization
> > between source topic and shadow topic.
> >
> > 1.When shadow topic initializes, it will read from metadata store path
> > "/managed-ledgers/{source_topic_ledger_name}", which contains all the
> > managed ledger info. We don't
> > need to read the  ledger information from source topic broker.
> >
> > 2. When shadow topic received new message from replicator, if the ledger
> > id of the message
> >  is the same as the last ledger, it just updates the LAC. If not, it will
> > update ledger list from metadata,
> > and then open the new ledger handle and update the LAC.
> >
> > As for the copy itself and add shadow message id in CommandSend, it mostly
> > serves the purpose
> > of filling the EntryCache.
> >
> > Thanks,
> > Haiting
> >
> > On 2022/06/23 02:08:46 PengHui Li wrote:
> > > > One question comes to mind here: Why not simply read the ledger
> > information
> > > from original topic, without copy?
> > >
> > > I think this is a good idea.
> > >
> > > Penghui
> > > On Jun 22, 2022, 23:57 +0800, dev@pulsar.apache.org, wrote:
> > > >
> > > > One question comes to mind here: Why not simply read the ledger
> > information
> > > > from original topic, without copy?
> > >
> >
>