I normally tend to be a silent observer of the community due to my
limited time schedule.
But I feel the obligation to share my thoughts in this thread so that
people are not misled.
> Most of the PIP monitoring involves adding plugins to existing
> metrics APIs. He has also contributed to the PIP reviews, but his
> contribution is more philosophical rather than technical. Most of his
> comments are comparing Pulsar to other projects, rather than focusing on
> the internal insights that Pulsar brings to the table
Please be aware, per ASF Community Guide[1], "There is nothing at the
Apache Software Foundation that says you must write code in order to be a
committer. Anyone who is supportive of the community and works in any of
the CoPDoC areas is a likely candidate for committership."
And I personally hold this view (which you may disagree): at this stage,
philosophical contributions have a very big impact on the overall health of
Pulsar and community.
And many of Asaf's contributions are guiding us toward the right direction.
> The Pulsar community needs to break away from the monopolies of
> the provider companies, start focusing on stable releases, and let other
> companies make their enhancements to meet their requirements, and
> experienced contributors or Pulsar creators should be active to prevent
> unfairness in the community.
First of all, if there's unfair treatment to any contributor, please
collect the specific proof and file a complaint to the Pulsar PMC and ASF
Board if you want.
I believe they will investigate and decide if that's true or not in an open
and fair way.
Secondly, you can help increase the community diversity by starting working
on Pulsar issues and making more contributions right now.
Thirdly, having a company behind an open source project isn't necessarily a
bad thing. A few examples:
- Apache Spark -- Databricks
- Apache Kafka -- Confluent
- Apache Hadoop -- Cloudera, Hortonworks, MapR
- Apache Cassandra -- DataStax
- Apache Flink -- Ververica
- Apache Beam -- Google
These companies are paying their employees money to work on the open source
project. As long as all the activities follow the community rule, everyone
using these projects are benefiting from these companies' contribution.
> Also, Pulsar may have
> numbers of non-SM PMC’s and committers, but if you look at the numbers
over
> the last 2-3 years, you’ll see that 99% are from SM.
People have replied previously regarding your statement about committers,
so I won't tackle it again.
Instead of just looking at where these new committers are coming from, I
would suggest you think about the following:
1. Does this new committer make meaningful contributions to the
community?
2. Is the standard of becoming a Pulsar committer consistent regardless
of where people are coming from?
3. Did you find anyone who makes great contributions is left behind or
prevented from becoming a committer?
> We also had a very difficult time in finding and running a stable version
> of Pulsar that has no major issues and we are constantly looking for the
> root cause in the main branch or newer versions for possible bug fixes and
> that is very painful.
Apache Pulsar is an evolving open source project, so bugs may be identified
along the way when people are using it in various scenarios.
But the community has invested a lot into its correctness and stability, as
other people mentioned every new release improves significantly.
BTW, if your team is really tired of managing a stable Pulsar, StreamNative
can help you :)
> I'm sure that many users are feeling the same way and
> it's only a question of time until they start to speak up about their
> experiences.
Please focus on sharing your own specific problems and experiences.
I do see many non-SN people speak up in this thread, and they share a
completely different experience than you.
> Many companies have faced similar issues with Confluent. Pulsar was an
> alternative solution, but unfortunately, we have come to the conclusion
> that Pulsar isn’t better with a streamnative provider and reviewers. We
> also found major bugs in each Pulsar release, which forced us to
> continually upgrade Pulsar with little to no stability. With unstable
> releases, a lazy review process, and a single provider-driven system,
> Pulsar will be extremely risky to use for any company and we would rather
> continue with our legacy Kafka clusters.
Pulsar has its own unique advantages over Kafka: multi-tenancy,
geo-replication, messaging queue support, zero-down-time scaling out & in.
We also see many companies(DiDi, Tencent, Paypal, etc) adopting OSS Pulsar.
Some of the problems like bugs in release, responsiveness for PR review can
be improved along the way.
To be fair, DataStax is also a vendor for Pulsar, and many of their folks
are also active in the community.
It's okay to choose other technologies that best fits your own
requirements, but please don't assume everyone is the same as you.
[1] https://community.apache.org/contributors/#anyone-can-become-a-committer
On Wed, Mar 6, 2024 at 11:14 PM Yunze Xu <x...@apache.org> wrote:
> My fault for missing the discussion mails for these PIPs because I
> didn't check the archives of previous months.
>
> Thanks,
> Yunze
>
> On Thu, Mar 7, 2024 at 3:05 PM Lari Hotari <lhot...@apache.org> wrote:
> >
> > On Thu, 7 Mar 2024 at 08:14, Girish Sharma <scrapmachi...@gmail.com>
> wrote:
> > > There is for 310 -
> > > https://lists.apache.org/thread/13ncst2nc311vxok1s75thl2gtnk7w1t
> >
> > Girish, thank you for sharing the link to the PIP-310 thread.
> > We also met in community meetings to discuss PIP-310
> > (https://lists.apache.org/thread/y1sqpyv37fo0k4bm1ox28wggvkb7pbtw).
> >
> > I believe PIP-310 is a successful example of excellent community
> collaboration.
> > With PIP-310, Girish provided insightful feedback and also presented a
> > thorough analysis
> > of the Pulsar Rate Limiters. As a first step, this led to PIP-322,
> > Pulsar Rate Limiting Refactoring
> > (https://github.com/apache/pulsar/blob/master/pip/pip-322.md). PIP-322
> > has been implemented and delivered in Pulsar 3.2.0.
> >
> > There are also plans to continue the improvements with the next set of
> > enhancements
> > following PIP-322 (refer to:
> > https://lists.apache.org/thread/rj14rmzd5z5sy7kfg22kt4l5dzjwk9vn)
> > and to cover the original requirements behind PIP-310.
> >
> > I'm looking forward to continuing this collaboration to deliver
> > further improvements to the
> > existing multi-tenancy features. The areas of improvement are service
> > level management
> > and capacity management of a large Pulsar system. This is also a key
> concern of
> > messaging-as-a-service / streaming-as-a-service platform teams that
> > build upon Apache Pulsar.
> >
> > -Lari
>
