Re: Status of Pulsar 2.9.0 and starting 2.9.1

At this point, if 2.9.0 is non stable, I think we should fast-forward to 2.9.1 which will include security fix. Though, we should start 2.9.1 right now. -- Matteo Merli On Fri, Dec 10, 2021 at 11:23 PM Michael Marshall wrote: > > +1 - thanks Enrico. > > - Michael > > On Sat,

Re: [Vote] PIP 106: Pulsar Broker extensions for operators of enterprise-wide clusters

+1 -- Matteo Merli On Thu, Dec 9, 2021 at 7:36 PM Hang Chen wrote: > > +1 > > Thanks, > Hang > > Li Li 于2021年12月10日周五 09:24写道： > > > > +1 > > > > > On Dec 10, 2021, at 8:45 AM, Sijie Guo wrote: > > > > > > +1 > >

Re: [Vote] PIP 106: Pulsar Broker extensions for operators of enterprise-wide clusters

Closing this vote and changing the status as approved (it's already been >2 days), with * 4 +1s binding - Penghui - Sijie - Matteo - Hang * 1 +1: - Li Li -- Matteo Merli On Mon, Dec 13, 2021 at 5:42 PM Matteo Merli wrote: > > +1 > > > -- > Matteo

Re: [DISCUSS] Release Pulsar 2.7.4

Let's take https://github.com/apache/pulsar/pull/12484 out of the picture since it's failing the tests. -- Matteo Merli On Sun, Dec 12, 2021 at 11:06 PM PengHui Li wrote: > > Yes, > > https://github.com/apache/pulsar/pull/13215 has cherry-picked, so we can > close

Re: [discuss] BacklogQuota param change

This regression was already there in 2.8, and 2.9.x is already delayed by 3 months. We cannot keep waiting indefinitely without shipping the release. -- Matteo Merli On Tue, Dec 14, 2021 at 12:38 AM Enrico Olivelli wrote: > > ZhangJian > > it looks like a regression, > I will w

[DISCUSSION] PIP-117: Change Pulsar standalone defaults

for example `new-2.10-defaults`. If the file is present, or if the data directory is completely missing, we will adopt the new set of default configuration settings. If the file is not there, we will continue to use existing defaults and we will not break the upgrade operation. -- Matteo Merli

[DISCUSSION] PIP-118: Do not restart brokers when ZooKeeper session expires

ing the topics, while attempting to recreate a new session. zookeeperSessionExpiredPolicy=shutdown ``` Change its default value to `reconnect`. -- Matteo Merli

[DISCUSSION] PIP-119: Enable consistent hashing by default on KeyShared dispatcher

removed nor deprecated. Users will still be able to use the old implementation. -- Matteo Merli

[DISCUSSION] PIP-120: Enable client memory limit by default

changes * `maxPendingMessages`: 1000 -> 0 64MB is picked because it's a small enough memory size that will guarantee a very high producer throughput, irrespective of the individual messages size. -- Matteo Merli

[DISCUSSION] PIP-123: Introduce Pulsar metadata CLI tool

; cd /my-path > ls 1 2 3 > delete 1 # Delete keys with relative paths ``` -- Matteo Merli

Re: [DISCUSS] How to handle stale PRs

wing these old PRs to identify what is either already rejected or superseded by other changes and what just needs some help to get completed. Just declaring PR bankrupticity alone won't solve the problem of why more PRs are created than reviewers can review. Matteo -- Matteo Merli On We

Re: [DISCUSS] How to handle stale PRs

legal perspective. IANAL, though *AFAIU*, when a contributor is opening a PR is already assigning the IP to the ASF. A committer will merge that code (after due diligence that it doesn't contain inadmissible code), but the code is already "donated to the ASF" at the moment of the PR.

Re: [DISCUSSION] PIP-118: Do not restart brokers when ZooKeeper session expires

On Wed, Dec 15, 2021 at 5:47 PM Haiting Jiang wrote: > Small question about new parameter "zookeeperSessionExpiredPolicy", now we > have Etcd implementation for metadata store, is it better to use > "metadataStoreExpiredPolicy" ? Good point, we should also use this occasion to deprecate the `zo

Re: [VOTE] Apache Pulsar 2.9.1 candidate 2

* Checked docker images Matteo -- Matteo Merli On Thu, Dec 16, 2021 at 12:53 PM Enrico Olivelli wrote: > > I have pushed the docker images to my personal dockehub account > > eolivelli/pulsar:2.9.1rc2 > eolivelli/pulsar-all:2.9.1rc2 > > Enrico > > Il Gio 16 Di

Re: [DISCUSSION] PIP-118: Do not restart brokers when ZooKeeper session expires

On Mon, Dec 20, 2021 at 2:24 PM Michael Marshall wrote: > I haven't been able to follow the PIP 45 work, so I have some high > level questions. Do we have documentation on the implementation of > these re-acquirable resource locks? I'd like to understand how these > work. There is admittedly not

Re: [DISCUSSION] PIP-120: Enable client memory limit by default

> Did you consider determining the default limit by inspecting the JVM's > runtime? > For example, in the broker, we set `maxMessagePublishBufferSizeInMB` to 1/2 > of direct memory, by default. > > I concede that this config is more complicated than a broker config since > we're talking about a cl

[VOTE] PIP-118: Do not restart brokers when ZooKeeper session expires

https://github.com/apache/pulsar/issues/13304 Following the discussion, I have updated the proposal to also include the deprecation and renaming of the config setting name to `metadataSessionExpiredPolicy`. --- ## Motivation After all the work done for PIP-45 that was already included in 2

[VOTE] PIP-119: Enable consistent hashing by default on KeyShared dispatcher

subscriptionKeySharedUseConsistentHashing=false ``` Change its default value to `true`. The `AUTO_SPLIT` mode will not be removed nor deprecated. Users will still be able to use the old implementation. -- Matteo Merli

[VOTE] PIP-120: Enable client memory limit by default

For the `ProducerBuilder`, changes * `maxPendingMessages`: 1000 -> 0 64MB is picked because it's a small enough memory size that will guarantee a very high producer throughput, irrespective of the individual messages size. -- Matteo Merli

Re: [DISCUSSION] PIP-117: Change Pulsar standalone defaults

-- Matteo Merli On Sun, Dec 19, 2021 at 9:42 PM Lan Liang wrote: > > My two cents +1 , In my case, i sometimes need to run pulsar standalone in > some server with limited resources. > The other point i can think of is maybe schema storage also need to change > from using BK

[VOTE] PIP-117: Change Pulsar standalone defaults

n the `data/standalone` directory, for example `new-2.10-defaults`. If the file is present, or if the data directory is completely missing, we will adopt the new set of default configuration settings. If the file is not there, we will continue to use existing defaults and we will not break the upgrade

[VOTE] PIP-123: Introduce Pulsar metadata CLI tool

te topic my-tenant/my-namespace/my-topic > cd /my-path > ls 1 2 3 > delete 1 # Delete keys with relative paths ``` -- Matteo Merli

Re: [DISCUSSION] PIP-124: Create init subscription before sending message to DLQ

> What if we extended the `CommandProducer` command to add a > `create_subscription` field? Then, any time a topic is auto > created and this field is true, the broker would auto create a > subscription. There are some details to work out, but I think this > feature would fulfill the needs of this

Re: [DISCUSS] Remove Grafana and Prometheus Dockerfiles; Remove DCOS example

ve them. > > > > Let me know what you think. > > > > Thanks, > > Michael > > > > [0] https://github.com/apache/pulsar/pull/13389 > > [1] https://hub.docker.com/u/apachepulsar/ > > [2] https://github.com/apache/pulsar/pull/13387 > > [3] > https://github.com/apache/pulsar/blob/c62ca808fc8c5aed3bb96b99bf6ef0c8ed382e3f/deployment/dcos/PulsarGroups.json#L251 > -- -- Matteo Merli

Re: [DISCUSS] PIP-124: Pulsar Client Shared State API

stem", then to use the BK TableService component. I think this feature should be best viewed as a recipe, as it doesn't depend on or benefits from any internal broker support. If there are enough interest and concrete use cases it can be then included later. -- Matteo Merli On Fri, Dec 2

Re: Voting time period minimum

only the vote of PMC members. I would be required to have a lazy majority of at least 3 binding +1s votes. The vote should stay open for at least 48 hours. -- Matteo Merli

[DISCUSSION] PIP-135: Include MetadataStore backend for Etcd

implementation. Work in progress: https://github.com/apache/pulsar/pull/13225 -- Matteo Merli

Re: [VOTE] PIP-122: Change loadBalancer default loadSheddingStrategy to ThresholdShedder

+1 -- Matteo Merli On Tue, Jan 11, 2022 at 12:05 PM Neng Lu wrote: > > +1 (non-binding) > > On Mon, Jan 10, 2022 at 12:39 AM Enrico Olivelli > wrote: > > > +1 (binding) > > > > Enrico > > > > Il giorno lun 10 gen 2022 alle ore 07:47 Hang Chen >

Re: [VOTE] PIP-121: Pulsar cluster level auto failover on client side

+1 -- Matteo Merli On Tue, Jan 11, 2022 at 12:07 PM Neng Lu wrote: > > +1 (non-binding) > > On Mon, Jan 10, 2022 at 12:40 AM PengHui Li wrote: > > > +1 (binding) > > > > Penghui > > > > On Mon, Jan 10, 2022 at 4:38 PM Enrico Olivelli > &g

Re: [DISCUSSION] PIP-124: Create init subscription before sending message to DLQ

> If we want to hold that the DLQ is not a normal topic, then I can see > why we would have a DLQ specific feature here. I think that, good or bad, the impression that users have that the DLQ is not a "normal" topic comes from 2 factors: 1. The experience with traditional messaging systems JMS an

[VOTE] PIP-135: Include MetadataStore backend for Etcd

implementation. Work in progress: https://github.com/apache/pulsar/pull/13225 -- Matteo Merli

Re: Architecture of function authorization for process mode

what it matters, to restrict the resources that this function has access to (eg: cpu, memory, network, disk..). -- Matteo Merli On Mon, Jan 24, 2022 at 11:54 PM Niclas Hedhman wrote: > > > This sounds quite important, and I would like to hear from the devs if > this is correct or

Re: [DISCUSS] Releasing Pulsar-client-go 0.8.0

+1 -- Matteo Merli On Wed, Feb 9, 2022 at 6:44 PM r...@apache.org wrote: > > Hello Everyone: > > > I hope you’ve all been doing well. In the past two months, we have > > fixed a number of bugs related to connection leaks and added > > some new features. Fo

Re: Architecture of function authorization for process mode

You don't need tenant admin access to create functions, you just need to give "functions" access to a normal user: pulsar-admin namespaces grant-permission --actions produce,consume,functions --role $MY_PRINCIPAL -- Matteo Merli On Fri, Feb 11, 2022 at 5:46 PM Devin Bost wr

Re: [VOTE] Pulsar Client Go Release 0.8.0 Candidate 1

+1 Checked signatures, compiled and run perf tool. -- Matteo Merli On Tue, Feb 15, 2022 at 7:28 PM r...@apache.org wrote: > > Hi everyone, > Please review and vote on the release candidate #1 for the version > 0.8.0, as follows: > [ ] +1, Approve the release > [ ] -1,

Re: [DISCUSS] Apache Pulsar 2.10.0 release

ase by case. The feature freeze will reduce the likelihood of introducing *more* issues by getting it from the master branch, but won't change a comma from issues that were already there. -- Matteo Merli On Wed, Feb 16, 2022 at 10:47 AM Michael Marshall wrote: > > > I will build

Re: [DISCUSS] Apache Pulsar 2.10.0 release

by getting it from the master branch, but > > won't change a comma from issues that were already there. > > I thought you wanted to implement a code/feature freeze to allow for > more release stabilization. Can you clarify what you mean here? > > Thanks, > Michael > >

Re: [DISCUSS] PIP-142 Increase default numHttpServerThreads value to 200 to prevent Admin API unavailability

Hi Lari, thanks for the proposal, though I don't think it would be good to change this default value. Pulsar HTTP API is an administrative service and it is not on the critical data path. If all the threads in the pool are busy, it is good to apply back pressure and slow down the incoming new req

Re: [DISCUSS] Apache Pulsar 2.10.0 release

On Wed, Feb 16, 2022 at 9:55 PM Michael Marshall wrote: > Given that our community members who focus on testing are otherwise > about to prepare for a quick 3 day round of testing, I don't believe > they would object to a last minute change that gives them more time > for testing. In your view, wh

Re: [VOTE] PIP-118: Do not restart brokers when ZooKeeper session expires

I forgot to close this one. Closing now with 5 +1s votes; * Matteo * PengHui * Hang * Enrico * Sijie -- Matteo Merli On Mon, Jan 24, 2022 at 11:47 PM Enrico Olivelli wrote: > > What's the status of this PIP ? > > We have a pending PR, > we can commit it as soon a

Re: [DISCUSS] PIP-142 Increase default numHttpServerThreads value to 200 to prevent Admin API unavailability

On Thu, Feb 17, 2022 at 3:12 AM Yunze Xu wrote: > And we should make it clear whether numHttpServerThread=200 could > solve the existing problem like > https://github.com/apache/pulsar/pull/13666 > Changing the number of threads won't do anything to

Re: [ANNOUNCE] Apache Pulsar Go Client 0.8.0 released

It was released correctly, same as all the prev releases: https://dist.apache.org/repos/dist/release/pulsar/pulsar-client-go-0.8.0/ -- Matteo Merli On Mon, Feb 21, 2022 at 1:03 PM Enrico Olivelli wrote: > > Hi, > Did you store the source code tarball on the dist.apache.org website?

Re: [PIP] #14395 Making SchemaRegistry implementation configurable

hod names. We should double-check these usages to make sure the expected operation is applied. Matteo -- Matteo Merli On Mon, Feb 21, 2022 at 7:53 AM Aparajita Singh wrote: > > Hi, > Please review this proposal: https://github.com/apache/pulsar/issues/14395 > > -- > Thanks, > Aparajita

Re: [DISCUSS] PIP-145: Improve performance of regex subscriptions

On Wed, Mar 2, 2022 at 2:15 PM Michael Marshall wrote: > > A new class, org.apache.pulsar.TopicsListService will keep track > > of watchers and will listen to changes in the metadata. > > I think we should avoid creating a new service to distribute > notifications to consumers. Instead, we should

Re: [DISCUSS] Releasing pulsar-client-go 0.8.1

+1 Thanks Rui, we should eliminate the GPL dependency ASAP. -- Matteo Merli On Thu, Mar 3, 2022 at 2:08 AM Rui Fu wrote: > > Hi everyone, > > I would like to start a discussion here about starting a new release of > pulsar-client-go v0.8.1. Recently we have some of dependenc

Re: [VOTE] Pulsar Client Go Release 0.8.1 Candidate 1

+1 Verified the checksums and tried the Go perf-producer/consumers. Thanks, Matteo -- Matteo Merli On Tue, Mar 8, 2022 at 11:39 PM wrote: > > Hi everyone, > > Please review and vote on the release candidate #1 for the version 0.8.1, as > follows: > [ ] +1, Approve the re

Re: [VOTE] PIP-145: Improve performance of regex subscriptions

+1 -- Matteo Merli On Tue, Mar 8, 2022 at 4:41 PM PengHui Li wrote: > > +1 > > Penghui > > On Tue, Mar 8, 2022 at 6:11 PM Andras Beni > wrote: > > > Hi Pulsar Community, > > > > I would like to start a VOTE on "Improve performance of regex > &g

Re: [DISCUSS] PIP-145: Improve performance of regex subscriptions

-- Matteo Merli On Tue, Mar 8, 2022 at 9:36 PM Michael Marshall wrote: > > My primary concern is that our event driven messaging platform relies > on polling for distribution of system events, like newly created > topics. I would love to find a purely event driven solution. As dis

Re: [DISCUSS] PIP-145: Improve performance of regex subscriptions

On Thu, Mar 10, 2022 at 12:26 PM Michael Marshall wrote: > > I believe we can address the risk by providing a switch to enable/disable > > this feature. > > Would it be worth putting server side filtering behind a flag in the > FeatureFlags protobuf message so that clients can still benefit from

Re: [VOTE] Pulsar Release 2.10.0 Candidate 4

* Checked docker images -- Matteo Merli On Tue, Mar 22, 2022 at 7:17 PM Haiting Jiang wrote: > > +1 > > - Compiled the source > - Checked checksums and signatures > - Validate Pub/Sub and Java Functions > - Validate Connectors > - Validate Stateful Functions > >

Re: [VOTE] Pulsar Release 2.10.0 Candidate 5

repository artifacts * Checked docker images -- Matteo Merli On Fri, Apr 8, 2022 at 6:15 PM Hang Chen wrote: > > +1 (binding) > > - Checked checksum and license > - Build from source code under JDK11 > - Startup standalone and run pulsar-perf to produce and consume > - Startup f

Re: Pulsar Community Meeting Notes 2022/04/14, (8:30 AM PST)

Thanks Michael for sending out the notes. Recording is available here: https://streamnative.zoom.us/rec/share/Eg2E7WfSOfPaHMdSphlrP-fN2NBjh4aT06eVTxv6TbBk4ujTltCcPNvq9kwHqMT4.mBdaRHY5eUXJM5bz Passcode: .H?wa4WM -- Matteo Merli On Thu, Apr 14, 2022 at 10:27 AM Michael Marshall wrote: > &

[DISCUSS] PIP-155: Drop support for Python2

build to run Python client lib tests with Python3 2. Switch integration tests to use Python3 3. Stop building and distributing wheel files for Python 2.7 -- Matteo Merli

Re: [DISCUSS] PIP-155: Drop support for Python2

update this proposal to use 3.7 as the oldest supported Python release and to keep supporting only the last 4 Python releases, following the Python EOL schedule (5 years from release to EOL). So right now: 3.7, 3.8, 3.9 and 3.10. Once 3.11 is out and 3.7 reaches EOL, we drop 3.7. -- Matteo Merli

Re: [DISCUSS] PIP-155: Drop support for Python2

-- Matteo Merli On Tue, Apr 19, 2022 at 8:23 PM Michael Marshall wrote: > Can we cherry pick these changes back to the older release branches > (2.8, 2.9, and 2.10)? I don't see any reason to support python 2.7 for > any future releases. Dropping support primarily saves release m

[VOTE] PIP-155: Drop support for Python2

2.7 -- Matteo Merli

[RESULT] [VOTE] PIP-155: Drop support for Python2

Closing the vote with 4 binding and 5 non-binding +1s Binding +1s: * Matteo * PengHui * Hang * Lin Non-Binding +1s: * Michael * Dezhi * Zike * Jiuming * Xiaolong Thanks, Matteo -- Matteo Merli On Tue, Apr 26, 2022 at 7:42 PM Lin Lin wrote: > > +1 > > Lin Lin

Re: [VOTE] PIP-157: Bucketing topic metadata to allow more topics per namespace

vote at this point. Matteo -- Matteo Merli On Mon, May 2, 2022 at 3:31 AM Lari Hotari wrote: > > -1. It's too early to start a vote. Let's first have discussions. > > -Lari > > ma 2. toukok. 2022 klo 9.50 Andras Beni > kirjoitti: > > > Hi Pulsar Community,

Re: Build Pulsar Server on Java 17- too strict ?

take advantage of in our codebase -- Matteo Merli On Mon, May 9, 2022 at 9:33 AM Neng Lu wrote: > > +1 for requiring JDK11 and prepare for JDK17 > > On 2022/05/09 11:03:27 Enrico Olivelli wrote: > > I am sorry, > > I have missed this thread. > > > > I believe tha

Re: Discuss about creating partitioned topic in V1

disable v1 APIs using the configuration, and actually disabling it by default in next Pulsar versions. Matteo Instead -- Matteo Merli On Mon, May 9, 2022 at 12:06 PM Enrico Olivelli wrote: > > +1 > I agree totally. It is a bug > > Enrico > > Il Lun 9 Mag 2022, 20:59 Michae

Re: Discuss about creating partitioned topic in V1

n it on in the clusters where it's needed, which will be <1% of the currently deployed clusters. -- Matteo Merli On Mon, May 9, 2022 at 9:49 PM Rajan Dhabalia wrote: > > > Having said that, V1 admin API was in effect considered as "soft > deprecated" since Pulsar

Re: Build Pulsar Server on Java 17- too strict ?

On Wed, May 11, 2022 at 1:10 AM Enrico Olivelli wrote: > I am sorry, I missed this discussion. > But until we cut a release we are in time to change our minds, if we > find out that we can do better. Yes, but the precise point of having a PIP process is to have discussions and formalize decisions

Re: [DISCUSS] PIP-157 Exclusive Producer: new mode ExclusiveWithFencing

gt; > a new constant on the Wire Protocol for AccessMode > a new constant in the Java Client API AccessMode#ExclusiveWithFencing > > Implementation > > The new mode will behave mostly like AccessMode#Exclusive but instead > of failing in case of the presence of other Producers

Re: Build Pulsar Server on Java 17- too strict ?

ulsar community to strictly move to jdk17 and > provide jdk-11 support for sometime. > > Thanks, > Rajan > > On Wed, May 11, 2022 at 6:40 AM Matteo Merli > wrote: > > > On Wed, May 11, 2022 at 1:10 AM Enrico Olivelli > > wrote: > > > I am sorry, I misse

Re: Build Pulsar Server on Java 17- too strict ?

-- Matteo Merli On Wed, May 11, 2022 at 11:10 AM Dave Fisher wrote: > > > > > On May 11, 2022, at 6:39 AM, Matteo Merli wrote: > > > > On Wed, May 11, 2022 at 1:10 AM Enrico Olivelli wrote: > >> I am sorry, I missed this discussion. > >> But until

Re: [VOTE] PIP-161 Exclusive Producer: new mode ExclusiveWithFencing

+1 -- Matteo Merli On Mon, May 16, 2022 at 9:09 AM Enrico Olivelli wrote: > > This is the VOTE thread for PIP-161 > > This is the GH issue: https://github.com/apache/pulsar/issues/15528 > This is the PR: https://github.com/apache/pulsar/pull/15488 > > Motivation > >

Re: [VOTE] PIP-157: Bucketing topic metadata to allow more topics per namespace

+1 -- Matteo Merli On Mon, May 2, 2022 at 6:40 PM Matteo Merli wrote: > > Lari & Enrico, the discussion thread was out for 11 days and there > were 2 positive feedbacks. > I don't think this qualifies as "too early for a vote" and it would > have been bett

Re: [VOTE] PIP-157: Bucketing topic metadata to allow more topics per namespace

new elements arise that were not initially considered), or otherwise this PIP process would be just a pointless exercise. Matteo -- Matteo Merli

[DISCUSS] PIP-174: Provide new implementation for broker dispatch cache

https://github.com/apache/pulsar/issues/15954 WIP can be seen at: https://github.com/apache/pulsar/pull/15955 --- ## Motivation The current implementation of the read cache in the Pulsar broker has largely remained unchanged for a long time, except for a few minor tweaks. While the im

Re: [DISCUSS] PIP-174: Provide new implementation for broker dispatch cache

On Tue, Jun 7, 2022 at 6:37 AM Enrico Olivelli wrote: > Great idea. > I wonder which kind of metrics we could have. To see how each > tenant/namespace is using the cache It can be done, at the expense of some CPU cost, for example by adding a tag for the topic and computing the sizes at the time

[DISCUSS] PIP-175: Extend time based release process

updating Pulsar website and the blog post announcing the release, which should (hopefully) happen on the scheduled day. -- Matteo Merli

Re: [DISCUSS] PIP-175: Extend time based release process

> > There is a high cost to maintain a lot of old releases, backport bug > > fixes, and security patches. In general, we actively support the last > > 3 minor releases while continuing to develop the next release. E.g., > > 2.8, 2.9, and 2.10, while 2.11 is under development. > > Is 2.7 EOL? If so

Re: [DISCUSS] PIP-175: Extend time based release process

On Tue, Jun 7, 2022 at 10:14 PM PengHui Li wrote: > > I'm not sure I fully understand the LTS release and feature release. > > > The LTS releases will be identified by being a `.0` version. For example: > > * `3.0` -> LTS > > * `3.1` -> regular release > > * `3.2` -> regular release > > * `4.0` ->

Re: [DISCUSS] PIP-175: Extend time based release process

-- Matteo Merli On Wed, Jun 8, 2022 at 12:33 AM Haiting Jiang wrote: > > > I was actually not thinking of changing the denomination of 2.11. On > > one hand, it could make sense for being the first Java 17 release, but > > on the other, we'd be releasing 2 LTSs supe

Re: [DISCUSS] PIP-175: Extend time based release process

> Schedules always slip. Would you say that if the 3.x feature releases take > too long with these hypothetical dates that 3.5 would be dropped in order to > release 4.0 on schedule? Yes, there needs to be clarity for the users on when releases are to be expected, even more so for LTS releases.

Re: Major problem

Hi Niclas, can you provide the output of `pulsar-admin topics stats $TOPIC` -- Matteo Merli On Wed, Jun 15, 2022 at 9:18 AM Niclas Hedhman wrote: > > Hi, > (perhaps this is a dev@ issue more than a users@ issue) > > I have had a Pulsar system running for ~5 months and have ha

Re: Mark delete rate limiter bug?

There is a background flush process that was added for this precise reason. In broker.conf # How frequently to flush the cursor positions that were accumulated due to rate limiting. (seconds). # Default is 60 seconds managedLedgerCursorPositionFlushSeconds=60 -- Matteo Merli On Wed, Jun

Re: Mark delete rate limiter bug?

limit the forced flush of the cursor. -- Matteo Merli On Thu, Jun 16, 2022 at 2:08 AM Asaf Mesika wrote: > > If I understand correctly: > > this.flushCursorsTask = > scheduledExecutor.scheduleAtFixedRate(catchingAndLoggingThrowables(thi

Re: [DISCUSS] PIP-174: Provide new implementation for broker dispatch cache

> If all topics share the same cache space, and we won't recycle space by topic. > So when some topics are rebalanced to other brokers, the previous broker > keeps its cache > and results in cache space wasting. > The topic shedding would be quite frequent under some cases, I am not sure > how ba

Re: CI - remove cmd line test retries?

+1 The retry that repeats the whole maven job is often hiding the real test failures that were in the 1st failed job. -- Matteo Merli On Mon, Jul 11, 2022 at 9:06 PM Anon Hxy wrote: > > Hi Boschi > > I support this plan and I think we need take some effort to make the Pulsar >

Re: Batch Messages with only 1 message

-- Matteo Merli On Tue, Jul 12, 2022 at 12:54 AM Enrico Olivelli wrote: > > Hello, > I think that we could implement a small but effective enhancement to batching. > > It may happen that even if you enable batching you come to create > entries with 1 only message. > > P

Re: [VOTE] Enable GitHub Discussion for user-facing discussion

+1 -- Matteo Merli On Wed, Jul 13, 2022 at 7:58 AM Dave Fisher wrote: > > +1 - thanks. > > Emails will go to dev@pulsar initially. We should have a separate discussion > about notifications since these are set on a repository by repository basis. > > > On Jul 12, 202

Re: [DISCUSS] PIP-192 New Pulsar Broker Load Balancer

On Wed, Jul 20, 2022 at 1:37 PM Rajan Dhabalia wrote: > Also, here we have to clearly document that it will not impact behavior of > existing load-balancers now or in future. Pulsar is used by many companies > and orgs so, deprecating and not maintaining existing components is not > acceptable in

[VOTE] PIP-174: Provide new implementation for broker dispatch cache

s will be able to configure the old implementation in `broker.conf`. This option will be helpful in case of performance regressions would be seen for some use cases with the new cache implementation. -- Matteo Merli

Re: [VOTE] PIP-174: Provide new implementation for broker dispatch cache

Closing this vote with 3 +1s and no -1s: +1s: * Matteo * PengHui * Dave Thanks, Matteo -- Matteo Merli On Thu, Jul 21, 2022 at 7:58 PM Dave Fisher wrote: > > Sorry I’m late to this discussion. > > I think that the motivation is correct. There is really quite a bit of >

Re: [DISCUSS] PIP-204: Reactive Java client for Apache Pulsar

+1 -- Matteo Merli On Mon, Aug 29, 2022 at 5:56 AM Lari Hotari wrote: > > Hi all, > > I have drafted PIP-204: Reactive Java client for Apache Pulsar. > > PIP link: > https://github.com/apache/pulsar/issues/17335 > > Here's a copy of the contents of

Re: [VOTE] PIP-205: Reactive Java client for Apache Pulsar

+1 -- Matteo Merli On Wed, Aug 31, 2022 at 8:04 AM Nicolò Boschi wrote: > > +1 (non binding) > Nicolò Boschi > > > Il giorno mer 31 ago 2022 alle ore 17:00 Christophe Bornet < > bornet.ch...@gmail.com> ha scritto: > > > +1 (non-binding) > > > > Le

PIP-55: Refresh Authentication Credentials

with a broker is cut. -- Matteo Merli

Re: PIP-55: Refresh Authentication Credentials

abstraction and do not require 1 timer per connection. Matteo -- Matteo Merli On Wed, Jan 15, 2020 at 10:22 AM Rajan Dhabalia wrote: > > Hi, > > We have the same ask and we had created PR to address the issue last year > with the PR: https://github.com/apache/pulsar/pull/3705 >

Re: [DISCUSS] Homebrew libpulsar

In general, Homebrew will only accept updates to the formula once a particular software version is already released. We could certainly include it as a post-step in our release process, to do after the official release announcement is done. -- Matteo Merli On Tue, Jan 21, 2020 at 11:08 AM

[REPORT] Pulsar March 2020

d and 165 closed in the last 3 months - 5218 Stars (up from 4608 in Dec 2019) -- Matteo Merli

Re: [VOTE] Pulsar Client Go Release 0.1.0 Candidate 1

+1 -- Matteo Merli On Thu, Mar 26, 2020 at 4:52 AM Yong Zhang wrote: > > +1 > > Thanks > Yong > > On Thu, 26 Mar 2020 at 19:44, PengHui Li wrote: > > > +1 > > > > Sijie Guo 于2020年3月26日周四 上午8:28写道： > > > > > +1 > > > >

Re: [VOTE] Pulsar Client Go Release 0.1.0 Candidate 1

that, we'd need to specify the Git tag hash, both here in the vote thread as in the release notes. Dave, do you thing that this would be an acceptable way to "release" a blessed tag? -- Matteo Merli On Thu, Mar 26, 2020 at 9:26 AM Dave Fisher wrote: > > -1 - What artif

Re: [VOTE] Pulsar Client Go Release 0.1.0 Candidate 1 again

+1 binding -- Matteo Merli On Wed, Apr 1, 2020 at 9:59 AM ran leon wrote: > > +1 > > Best, > Ran > > PengHui Li 于2020年4月1日周三 下午6:41写道： > > > +1 binding > > > > Guangning E 于2020年3月31日周二 下午6:25写道： > > > > > +1 non-binding > > >

Re: [DISCUSS] PIP-62: Move connectors, adapters, and presto connector to separate repositories

rfect, but it would be easier to setup until we can get a more rigorous daily CI pipeline that cascades the artifacts across builds. In any case, we should have a ~daily build against the "latest" master in order to avoid regressions. -- Matteo Merli On Thu, Apr 9, 2020 at 1:11 PM S

Re: [Discuss] Replace cgo client in Pulsar repo with Native Go Client

> Delete the CGo client code from Pulsar repo I'd rather wait a bit more for that, given that apps are fetching directly from github. Eg: change the Docs but leave the code there for one more release cycle. -- Matteo Merli On Mon, Apr 13, 2020 at 2:37 PM Yuva raj wrote: > >

Re: [DISCUSS] PIP 64: Introduce REST endpoints for producing, consuming and reading messages

that this request must be made to the specific broker holding the consumer instance. <<< That's not necessary to go on the same host. A shared subscription dispatcher will be able to apply the ack even if it's received from a different consumer (and it will clean up the pending

Re: [DISCUSS] PIP 64: Introduce REST endpoints for producing, consuming and reading messages

-- Matteo Merli On Thu, May 14, 2020 at 12:33 AM Sijie Guo wrote: > > On Wed, May 13, 2020 at 10:31 PM Matteo Merli > wrote: > People can use the HOST header for targeting to a specific broker. > > If applications don't use the HOST header, it will fall back to the &g

Re: [VOTE] Pulsar Client Go Release 0.1.1 Candidate 1

+1 (binding) -- Matteo Merli On Thu, Jun 18, 2020 at 6:00 AM lan.liang wrote: > > +1 nb > > > > > > > > > - lan.liang > On 6/18/2020 17:20，ran leon wrote： > +1 (non-binding) > > Thanks > Leon > > On 2020/06/12 09:41:06, xiaolong ran w

[DISCUSSION] PIP 68: Exclusive Producer

ic("my-topic") .readCompacted(true) .create(); while (reader.hasMessageAvailable()) { Message msg = reader.readNext(); myState.put(msg.getKey(), msg.getValue()); } ``` Done. We're now ready to use the producer as a write-ahead-log for the updates on the state ```java void write(String key, T value) { producer.newMessage().key(key).value(value).send(); myState.put(key, value); } ``` -- Matteo Merli

  1   2   3   4   >