Re: [DISSCUSS] Don't retain null-key messages during topic compaction

2023-11-07 Thread Enrico Olivelli
This is a good point. Is it worth to make it configurable per topic and per namespace? Even if this behavior is kind if a side effect or bug, maybe some applications rely on it. Flags that change the behavior per cluster are hard to adopt in nig cluster with many tenants. We should always take m

Re: [DISSCUSS] Don't retain null-key messages during topic compaction

2023-11-07 Thread Cong Zhao
Hello everyone, I open a PIP-318: https://github.com/apache/pulsar/pull/21541 for this discussion. Any feedback and suggestions are welcome. Thanks, Cong Zhao On 2023/11/07 02:55:52 Cong Zhao wrote: > Hi, Pulsar community > > Currently, we retain all null-key messages during topic compaction,

Re: [VOTE] Pulsar Client C++ Release 3.4.0 Candidate 2

2023-11-07 Thread Yubiao Feng
+1 (no-binding) Thanks Yubiao Feng On Tue, Nov 7, 2023 at 3:03 PM Yunze Xu wrote: > This is the second release candidate for Apache Pulsar Client C++, > version 3.4.0. > > It fixes the following issues: > https://github.com/apache/pulsar-client-cpp/milestone/5?closed=1 > > *** Please download,

Re: [DISSCUSS] PIP-317: Add `bookkeeperDeleted` field to show whether a ledger is deleted from the Bookie while using tiered storage

2023-11-07 Thread Dezhi Liu
Hi 刘燊, I think this feature is necessary. On 2023/11/06 11:13:53 刘燊 wrote: > Hi community, > > The motivation behind this PIP is to provide administrators and users with > better insights into the state of the ledgers and the overall storage usage. > By including the `bookkeeperDeleted` field i

Re: [DISCUSS] PIP-310: Support custom publish rate limiters

2023-11-07 Thread Rajan Dhabalia
Hi Lari/Girish, I am sorry for jumping late in the discussion but I would like to acknowledge the requirement of pluggable publish rate-limiter and I had also asked it during implementation of publish rate limiter as well. There are trade-offs between different rate-limiter implementations based o

Re: [DISCUSS] PIP-310: Support custom publish rate limiters

2023-11-07 Thread Lari Hotari
Hi Girish, Replies inline. On Tue, 7 Nov 2023 at 15:26, Girish Sharma wrote: > > Hello Lari, replies inline. > > I will also be going through some textbook rate limiters (the one you shared, > plus others) and propose the one that at least suits our needs in the next > reply. sounds good. I'

Re: [OT] Evaluate Virtual thread [WAS][DISCUSS] Moving to Java 21

2023-11-07 Thread 太上玄元道君
In the past, threads usually blocked on LockUtil.park(…), and AQS is depends on it to park threads. Virtual thread only solved thread blocking on JDK layer by rewriting LockUtik.park(…) In some conditions, the carrier thread will be blocked (I only remember the following points) 1. JNI calling 2.

Re: [DISCUSS] PIP-310: Support custom publish rate limiters

2023-11-07 Thread Asaf Mesika
I just want to add one thing to the mix here. You can see by the amount of plugin interfaces Pulsar has, somebody "left the door open" for too long. You can agree with me that the number of those interfaces is not normal for any open source software. I know HBase for example, or Kafka - never seen

Re: [DISCUSS] Replace stale bot with ping-pong workflow

2023-11-07 Thread Asaf Mesika
Tison let's start as you suggested by disabling it On Tue, May 16, 2023 at 5:13 AM Yunze Xu wrote: > +1 to me > > Thanks, > Yunze > > On Sun, May 14, 2023 at 9:28 PM Dave Fisher wrote: > > > > Hi - > > > > I have not looked at all your links but I think this is a great idea. > This will help e

Re: [DISCUSS] PIP-310: Support custom publish rate limiters

2023-11-07 Thread Girish Sharma
Hello Lari, replies inline. I will also be going through some textbook rate limiters (the one you shared, plus others) and propose the one that at least suits our needs in the next reply. On Tue, Nov 7, 2023 at 2:49 PM Lari Hotari wrote: > > It is bi-weekly on Thursdays. The meeting calendar, z

Pulsar Flaky test report 2023-10-27 to 2023-11-06 for PR builds in CI

2023-11-07 Thread Lari Hotari
Dear Pulsar community, Here's a report of the flaky tests in Pulsar CI during the observation period of 2023-10-27 to 2023-11-06. The Pulsar CI is in fairly good shape at the moment. We have been able to reduce flakiness, and it is not currently slowing down PR processing significantly. The flak

[OT] Evaluate Virtual thread [WAS][DISCUSS] Moving to Java 21

2023-11-07 Thread tison
Hi, I check the docs for Virtual Threads[1][2][3]. It comes up to me with two major concerns about its real-world improvement for Pulsar's scenario: 1. All of the virtual threads share the same schedule pool, which means that all tasks run on virtual threads competing with each other. It can be b

[ANNOUNCE] Apache Pulsar Go Client 0.11.1 released

2023-11-07 Thread Zike Yang
The Apache Pulsar team is proud to announce Apache Pulsar Go Client version 0.11.1. Pulsar is a highly scalable, low latency messaging platform running on commodity hardware. It provides simple pub-sub semantics over topics, guaranteed at-least-once delivery of messages, automatic cursor managemen

Re: [DISCUSS] PIP-310: Support custom publish rate limiters

2023-11-07 Thread Lari Hotari
Hi Girish, I think we are starting to get into the concrete details of rate limiters and how we could start improving the existing feature. It is very helpful that you are sharing your insight and experience of operating Pulsar at scale. Replies inline. On Mon, 6 Nov 2023 at 15:37, Girish Sharma

Re: [VOTE] Pulsar Client Go Release 0.11.1 Candidate 1

2023-11-07 Thread Zike Yang
Close this vote by 3 binding +1: - Yunze - Penghui - Mattison BR, Zike Yang On Tue, Nov 7, 2023 at 4:43 PM Zike Yang wrote: > > > The KEYS file link is dead, I think it should be > https://downloads.apache.org/pulsar/KEYS > > Thanks for your reminder. Yes, that should be the correct link. I hav

Re: [VOTE] Pulsar Client Go Release 0.11.1 Candidate 1

2023-11-07 Thread Zike Yang
> The KEYS file link is dead, I think it should be https://downloads.apache.org/pulsar/KEYS Thanks for your reminder. Yes, that should be the correct link. I have also pushed a PR to fix the release process: https://github.com/apache/pulsar-client-go/pull/1127 PTAL. Thanks! BR, Zike Yang On Tue,

Re: [VOTE] Pulsar Client Go Release 0.11.1 Candidate 1

2023-11-07 Thread mattison chao
+1 (binding) - Built from the source - Ran the test on pulsar 3.0.1 Best, Mattison > On Sep 11, 2023, at 18:07, Zike Yang wrote: > > Hi everyone, > Please review and vote on the release candidate #1 for the version > 0.11.1, as follows: > [ ] +1, Approve the release > [ ] -1, Do not approve th