Call for projects and mentors for OSPP 2022

[PengHui Li]

Hi all,

Open Source Promotion Plan, OSPP[1] is open for project registration,
like Google Summer of Code. This Plan aims to encourage college students
to participate in open-source projects and make contributions.

The project idea is open from 4/21, 2022 to 5/9, 2022(UTC+8).
Apache Pulsar ended up with 6 projects seats. If you have great ideas,
please copy the following template and fill in the fields.
Once you have completed your project ideas, you can reply to this email.
We will help you register for the program. Note that One mentor can
mentor one project at a time and both mentors and students can
receive financial awards for completed projects. Only Pulsar committers
can be the mentors as asked by the OSPP.

The project can include feature catchup, ecological tool construction,
new features of the pulsar and so on.

Welcome to join the Pulsar
slack channel: #ospp-summer for asking any questions about OSPP.

[1] https://summer-ospp.ac.cn

Thanks,
Penghui


Template

Project Name:
Project Description: (at most 1000 words)
Difficulty Level:
- [ ] Basic
- [ ] Advanced
Project Validation Items:
Item 1:__
Item 2:__
Item 3:__
…
Project Mentor:
Your Name:
Your Email:
Your Apache ID:

Re: [VOTE] [PIP-154] Max active transaction limitation for transaction coordinator

[Enrico Olivelli]

+1 (binding)

Enrico

Il Dom 24 Apr 2022, 05:59 Haiting Jiang  ha
scritto:

> +1
>
> Thanks,
> Haiting
>
> On 2022/04/24 02:00:39 丛搏 wrote:
> > Hi Pulsar community,
> >
> > This is the voting thread for PIP-154. It will stay open for at least 48
> hours.
> >
> > The proposal can be found: https://github.com/apache/pulsar/issues/15133
> >
> > Discuss thread:
> https://lists.apache.org/thread/h8gzgs78bm4pzy2rjtgvvmo1jbzttcx8
> >
> > Thanks,
> > Bo
> >
>

[GitHub] [pulsar-helm-chart] yuweisung opened a new pull request, #260: added pdb version detection

[GitBox]

yuweisung opened a new pull request, #260:
URL: https://github.com/apache/pulsar-helm-chart/pull/260

   Fixes pod disruption budget version warning
   
   ### Motivation
   
   PDB policy api version, v1beta1 is deprecated in k8s1.21+ (not available in 
1.25+).
   
   ### Modifications
   
   zookeeper-pdb, proxy-pdb, broker-pdb and bookkeepr-pdb templates are 
modified.  If k8s api-resources container policy/v1, the *-pdb.yaml will 
generate respective apiVersion. 
   
   ### Verifying this change
   
   - [ ] Make sure that the change passes the CI checks.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [VOTE] PIP-156: Build and Run Pulsar Server on Java 17

[PengHui Li]

+1

Penghui

On Thu, Apr 21, 2022 at 1:07 AM Dave Fisher  wrote:

> This is close and much work is done.
>
> +1.
>
> > On Apr 20, 2022, at 9:37 AM, Heesung Sohn 
> > 
> wrote:
> >
> > Dear Pulsar Community,
> >
> > Please review and vote on this PIP.
> >
> > PIP link : https://github.com/apache/pulsar/issues/15207
> >
> > Thank you,
> > --
> >
> > 
> >
> > Heesung Sohn
> >
> > Platform Engineer
> >
> > e: heesung.s...@streamnative.io
> >
> > streamnative.io
> >
> > 
> > 
> > 
>
>

Re: [DISCUSS] PIP-157: Bucketing topic metadata to allow more topics per namespace

[PengHui Li]

+1

Penghui

On Thu, Apr 21, 2022 at 9:17 PM Andras Beni
 wrote:

> Hi everyone,
>
> I've just created a proposal that will help scaling up the number of topics
> per namespace.
> It's available at https://github.com/apache/pulsar/issues/15254 and is
> copied below.
> Let me know what you think.
>
> Thanks,
> Andras
>
> Motivation
>
> Pulsar is able to manage millions of topics but the number of topics within
> a single namespace is limited by metadata storage.
>
> For each topic within a namespace there is a ZooKeeper node. Listing topics
> thus requires listing children of a node, which at around 10K topics hits
> the limits of ZK.
> Goal
>
> This feature will allow a larger number of topics within a namespace by
> inserting an intermediate layer (buckets) before the topic nodes like
> /managed-ledgers/tenant/namespace/domain/bucket/topic.
>
> By default this feature will be switched off and would only be enabled on a
> per namespace basis at the creation of namespaces by setting a policy. This
> eliminates the need for migrating existing installations to this new
> scheme.
>
> Buckets will not have correlation with bundles.
> API Changes
>
> A new policy numberOfTopicBuckets will be added. The default value, 1 means
> no bucketing, the current behaviour will be preserved for the namespace.
> Greater values mean topics will be stored at a path including buckets.
> Users will not be able to change the number of buckets after the namespace
> is created.
> Implementation
>
> The goal is to implement this feature transparently to the user. Clients
> will continue to refer to topics by domain://tenant/namespace/topic but
> pulsar will internally translate to the new persistence naming where
> necessary.
>
> The way metadata stores work will not be affected either.
>
> Assigning topics to buckets will be based on the topic name's hash code's
> absolute value modulo the number of buckets.
>
> The bulk of the changes necessary for this feature is to make namespace
> policies available wherever persistence naming is calculated. Where listing
> of topics within a namespace is necessary, the introduction of the new
> layer will add some overhead in the form of multiple requests to the
> metadata store. These include checking if the limit on topic number per
> namespace has been reached.
> Example
>
> Let's consider the following metadata hierarchy:
>
> managed-ledgers
> \-  tenant
> \-  namespace
> \-  persistent
> +-  nptopic1
> +-  nptopic2
> +-  ptopic-partition-0
> +-  ptopic-partition-1
> +-  ptopic-partition-2
> \-  ptopic-partition-3
>
> In case of 3 buckets the same topic metadata would be laid out the
> following way:
>
> managed-ledgers
> \-  tenant
> \-  namespace
> \-  persistent
> +-  $0
> |   +-  ptopic-partition-0
> |   \-  ptopic-partition-3
> +-  $1
> |   +-  nptopic2
> |   \-  ptopic-partition-1
> \-  $2
> +-  nptopic1
> \-  ptopic-partition-2
>
> Compatibility
>
> Existing namespaces and namespaces created without explicitly activating
> this feature will not be affected.
>
> Namespaces created with this feature activated can be used just as others.
> Rejected alternatives
>
> An alternative approach would be to introduce bucketing globally for all
> namespaces. This would make metadata structure more homogeneous but would
> require complex update logic to atomically move topics from their current
> path to the new place once all brokers are upgraded.
> For similar reasons changing the number of buckets is not a goal of this
> proposal.
>
> Since the proposal intends to solve a problem related to ZK, it could be
> handled within the ZK based metadata store implementation. This would have
> to introduce knowledge of what paths mean thus breaking separation of
> concerns.
>

Re: [VOTE] PIP-155: Drop support for Python2

[PengHui Li]

+1

Penghui

On Thu, Apr 21, 2022 at 7:18 PM r...@apache.org 
wrote:

> +1 (non-binding)
>
> --
> Thanks
> Xiaolong Ran
>
> Jiuming Tao  于2022年4月21日周四 19:00写道：
>
> > +1
> > Thanks,
> > Tao Jiuming
> >
> > > 2022年4月20日 下午8:42，Matteo Merli  写道：
> > >
> > > https://github.com/apache/pulsar/issues/15185
> > >
> > > ## Motivation
> > >
> > > Python 2.x has been deprecated for many years now and it was
> > > officially end-of-lifed 2.5 years ago
> > > (https://www.python.org/doc/sunset-python-2/).
> > >
> > > We have well reached the point by which we need to drop Python 2.7
> > > compatibility for Pulsar client and for Pulsar functions.
> > >
> > > Based on Python end-of-life policies, there are 4 versions of Python
> > > that are currently supported: 3.7, 3.8, 3.9 and 3.10.
> > > https://endoflife.date/python
> > >
> > > That gives a 5 years support schedule from release to EoL.
> > >
> > > ## Goal
> > >
> > > Support only the last 4 Python releases Pulsar client and for Pulsar
> > > functions, currently 3.7, 3.8, 3.9 and 3.10. Once 3.7 reaches
> > > end-of-life, it will be dropped by Pulsar too.
> > >
> > > This policy will be also applied to the Pulsar patch releases which
> > > will stop providing Python wheel files for the deprecated versions.
> > >
> > > ## API Changes
> > >
> > > No changes at this time, though Pulsar Python client library will be
> > > now free to use Python3 specific syntaxes and libraries.
> > >
> > > ## Changes
> > >
> > > 1. Switch the CI build to run Python client lib tests with Python3
> > > 2. Switch integration tests to use Python3
> > > 3. Stop building and distributing wheel files for Python 2.7
> > >
> > > --
> > > Matteo Merli
> > > 
> >
> >
>

[GitHub] [pulsar-site] urfreespace merged pull request #56: ecosystem updates

[GitBox]

urfreespace merged PR #56:
URL: https://github.com/apache/pulsar-site/pull/56


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [DISCUSS] PIP-157: Bucketing topic metadata to allow more topics per namespace

[Hang Chen]

+1

Thanks,
Hang

PengHui Li  于2022年4月25日周一 09:20写道：
>
> +1
>
> Penghui
>
> On Thu, Apr 21, 2022 at 9:17 PM Andras Beni
>  wrote:
>
> > Hi everyone,
> >
> > I've just created a proposal that will help scaling up the number of topics
> > per namespace.
> > It's available at https://github.com/apache/pulsar/issues/15254 and is
> > copied below.
> > Let me know what you think.
> >
> > Thanks,
> > Andras
> >
> > Motivation
> >
> > Pulsar is able to manage millions of topics but the number of topics within
> > a single namespace is limited by metadata storage.
> >
> > For each topic within a namespace there is a ZooKeeper node. Listing topics
> > thus requires listing children of a node, which at around 10K topics hits
> > the limits of ZK.
> > Goal
> >
> > This feature will allow a larger number of topics within a namespace by
> > inserting an intermediate layer (buckets) before the topic nodes like
> > /managed-ledgers/tenant/namespace/domain/bucket/topic.
> >
> > By default this feature will be switched off and would only be enabled on a
> > per namespace basis at the creation of namespaces by setting a policy. This
> > eliminates the need for migrating existing installations to this new
> > scheme.
> >
> > Buckets will not have correlation with bundles.
> > API Changes
> >
> > A new policy numberOfTopicBuckets will be added. The default value, 1 means
> > no bucketing, the current behaviour will be preserved for the namespace.
> > Greater values mean topics will be stored at a path including buckets.
> > Users will not be able to change the number of buckets after the namespace
> > is created.
> > Implementation
> >
> > The goal is to implement this feature transparently to the user. Clients
> > will continue to refer to topics by domain://tenant/namespace/topic but
> > pulsar will internally translate to the new persistence naming where
> > necessary.
> >
> > The way metadata stores work will not be affected either.
> >
> > Assigning topics to buckets will be based on the topic name's hash code's
> > absolute value modulo the number of buckets.
> >
> > The bulk of the changes necessary for this feature is to make namespace
> > policies available wherever persistence naming is calculated. Where listing
> > of topics within a namespace is necessary, the introduction of the new
> > layer will add some overhead in the form of multiple requests to the
> > metadata store. These include checking if the limit on topic number per
> > namespace has been reached.
> > Example
> >
> > Let's consider the following metadata hierarchy:
> >
> > managed-ledgers
> > \-  tenant
> > \-  namespace
> > \-  persistent
> > +-  nptopic1
> > +-  nptopic2
> > +-  ptopic-partition-0
> > +-  ptopic-partition-1
> > +-  ptopic-partition-2
> > \-  ptopic-partition-3
> >
> > In case of 3 buckets the same topic metadata would be laid out the
> > following way:
> >
> > managed-ledgers
> > \-  tenant
> > \-  namespace
> > \-  persistent
> > +-  $0
> > |   +-  ptopic-partition-0
> > |   \-  ptopic-partition-3
> > +-  $1
> > |   +-  nptopic2
> > |   \-  ptopic-partition-1
> > \-  $2
> > +-  nptopic1
> > \-  ptopic-partition-2
> >
> > Compatibility
> >
> > Existing namespaces and namespaces created without explicitly activating
> > this feature will not be affected.
> >
> > Namespaces created with this feature activated can be used just as others.
> > Rejected alternatives
> >
> > An alternative approach would be to introduce bucketing globally for all
> > namespaces. This would make metadata structure more homogeneous but would
> > require complex update logic to atomically move topics from their current
> > path to the new place once all brokers are upgraded.
> > For similar reasons changing the number of buckets is not a goal of this
> > proposal.
> >
> > Since the proposal intends to solve a problem related to ZK, it could be
> > handled within the ZK based metadata store implementation. This would have
> > to introduce knowledge of what paths mean thus breaking separation of
> > concerns.
> >

Re: [VOTE] PIP-155: Drop support for Python2

[Hang Chen]

+1

Thanks,
Hang

PengHui Li  于2022年4月25日周一 09:21写道：
>
> +1
>
> Penghui
>
> On Thu, Apr 21, 2022 at 7:18 PM r...@apache.org 
> wrote:
>
> > +1 (non-binding)
> >
> > --
> > Thanks
> > Xiaolong Ran
> >
> > Jiuming Tao  于2022年4月21日周四 19:00写道：
> >
> > > +1
> > > Thanks,
> > > Tao Jiuming
> > >
> > > > 2022年4月20日 下午8:42，Matteo Merli  写道：
> > > >
> > > > https://github.com/apache/pulsar/issues/15185
> > > >
> > > > ## Motivation
> > > >
> > > > Python 2.x has been deprecated for many years now and it was
> > > > officially end-of-lifed 2.5 years ago
> > > > (https://www.python.org/doc/sunset-python-2/).
> > > >
> > > > We have well reached the point by which we need to drop Python 2.7
> > > > compatibility for Pulsar client and for Pulsar functions.
> > > >
> > > > Based on Python end-of-life policies, there are 4 versions of Python
> > > > that are currently supported: 3.7, 3.8, 3.9 and 3.10.
> > > > https://endoflife.date/python
> > > >
> > > > That gives a 5 years support schedule from release to EoL.
> > > >
> > > > ## Goal
> > > >
> > > > Support only the last 4 Python releases Pulsar client and for Pulsar
> > > > functions, currently 3.7, 3.8, 3.9 and 3.10. Once 3.7 reaches
> > > > end-of-life, it will be dropped by Pulsar too.
> > > >
> > > > This policy will be also applied to the Pulsar patch releases which
> > > > will stop providing Python wheel files for the deprecated versions.
> > > >
> > > > ## API Changes
> > > >
> > > > No changes at this time, though Pulsar Python client library will be
> > > > now free to use Python3 specific syntaxes and libraries.
> > > >
> > > > ## Changes
> > > >
> > > > 1. Switch the CI build to run Python client lib tests with Python3
> > > > 2. Switch integration tests to use Python3
> > > > 3. Stop building and distributing wheel files for Python 2.7
> > > >
> > > > --
> > > > Matteo Merli
> > > > 
> > >
> > >
> >

Re: [VOTE] PIP-156: Build and Run Pulsar Server on Java 17

[Hang Chen]

+1

Thanks,
Hang

PengHui Li  于2022年4月25日周一 09:18写道：
>
> +1
>
> Penghui
>
> On Thu, Apr 21, 2022 at 1:07 AM Dave Fisher  wrote:
>
> > This is close and much work is done.
> >
> > +1.
> >
> > > On Apr 20, 2022, at 9:37 AM, Heesung Sohn 
> > > 
> > wrote:
> > >
> > > Dear Pulsar Community,
> > >
> > > Please review and vote on this PIP.
> > >
> > > PIP link : https://github.com/apache/pulsar/issues/15207
> > >
> > > Thank you,
> > > --
> > >
> > > 
> > >
> > > Heesung Sohn
> > >
> > > Platform Engineer
> > >
> > > e: heesung.s...@streamnative.io
> > >
> > > streamnative.io
> > >
> > > 
> > > 
> > > 
> >
> >

Re: [VOTE] [PIP-154] Max active transaction limitation for transaction coordinator

[Hang Chen]

+1

Thanks,
Hang

Enrico Olivelli  于2022年4月25日周一 00:12写道：
>
> +1 (binding)
>
> Enrico
>
> Il Dom 24 Apr 2022, 05:59 Haiting Jiang  ha
> scritto:
>
> > +1
> >
> > Thanks,
> > Haiting
> >
> > On 2022/04/24 02:00:39 丛搏 wrote:
> > > Hi Pulsar community,
> > >
> > > This is the voting thread for PIP-154. It will stay open for at least 48
> > hours.
> > >
> > > The proposal can be found: https://github.com/apache/pulsar/issues/15133
> > >
> > > Discuss thread:
> > https://lists.apache.org/thread/h8gzgs78bm4pzy2rjtgvvmo1jbzttcx8
> > >
> > > Thanks,
> > > Bo
> > >
> >

[GitHub] [pulsar-site] urfreespace merged pull request #48: fix: migrate algolia to new credencials

[GitBox]

urfreespace merged PR #48:
URL: https://github.com/apache/pulsar-site/pull/48


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[VOTE] [PIP-153] Optimize metadataPositions in MLPendingAckStore

[????????]

Hi Pulsar Community.

This is the voting thread for PIP-153. It will stay open for at least 48 hours.

The proposal can be found:https://github.com/apache/pulsar/issues/15073

Discuss thread:
https://lists.apache.org/thread/svmbp8ybn6l8o0o8dmvsysnb86qj77r3

Thanks,
Xiangying