[Discussion] Breaking change of Pulsar standalone's start

2022-12-13 Thread Yunze Xu
Hi all, I found a breaking change introduced from #15186 [1] that changed the behavior of Pulsar standalone. Before #15186, it used Pulsar's built-in admin client to create tenant (public) and namespace (public/default). After #15186, it created the metadata nodes directly to the metadata store.

Re: [Discussion] Breaking change of Pulsar standalone's start

2022-12-13 Thread Yunze Xu
pts/pulsar-test-service-start.sh#L55 Thanks, Yunze Xu On Tue, Dec 13, 2022 at 7:40 PM Yunze Xu wrote: > > Hi all, > > I found a breaking change introduced from #15186 [1] that changed the > behavior of Pulsar standalone. > > Before #15186, it used Pulsar's built-in adm

Re: [VOTE] Pulsar Release 2.11.0 Candidate-2

2022-12-13 Thread Yunze Xu
ssue. > > > > Thanks, > > Zixuan > > > > Zixuan Liu 于2022年12月9日周五 17:47写道: > > > > > Ok, let me make a new PR to fix this. > > > > > > Thanks, > > > Zixuan > > > > > > Yunze Xu 于2022年12月9日周五 17:41写道: >

Re: [Discussion] Breaking change of Pulsar standalone's start

2022-12-13 Thread Yunze Xu
FYI, I pushed a PR: https://github.com/apache/pulsar/pull/18909 Thanks, Yunze On Tue, Dec 13, 2022 at 7:55 PM Yunze Xu wrote: > > In addition to the setup process, Go client assumes the cluster was > not created [1] while C++ and Python clients don't care if it's > create

[DISCUSSION] Any idea about simplify the configuration file?

2022-12-13 Thread Yunze Xu
Hi all, As more people joined the development of Pulsar and more PIPs are opened, I found the configurations became very large. At the moment for commit 9917aac, there are 426 configuration items in broker.conf, which is too many. ```bash $ grep "^[^#]" conf/broker.conf | wc -l 426 ``` For begin

Re: [DISCUSSION] Any idea about simplify the configuration file?

2022-12-13 Thread Yunze Xu
nks, Yunze Xu On Tue, Dec 13, 2022 at 8:53 PM Yunze Xu wrote: > > Hi all, > > As more people joined the development of Pulsar and more PIPs are > opened, I found the configurations became very large. At the moment > for commit 9917aac, there are 426 configuration items in brok

Re: [DISCUSS] Release Nodejs Client 1.8.0

2022-12-13 Thread Yunze Xu
+1 Thanks, Yunze On Tue, Dec 13, 2022 at 9:05 PM Matteo Merli wrote: > > +1 > > On Tue, Dec 13, 2022 at 2:03 AM Zike Yang wrote: > > > Hi everyone, > > > > I would like to propose releasing the Pulsar Client Nodejs 1.8.0 > > > > There are several new features and bug fixes in the master branch[

Re: [DISCUSS] Release Pulsar Python Client 3.0.0

2022-12-13 Thread Yunze Xu
; > Zike Yang > > > > On Wed, Nov 23, 2022 at 10:40 PM PengHui Li wrote: > > > > > > +1 > > > > > > Penghui > > > > > > > On Nov 23, 2022, at 21:38, Enrico Olivelli wrote: > > > > > > > > +1 > >

Re: [DISCUSSION] Any idea about simplify the configuration file?

2022-12-13 Thread Yunze Xu
be a big pain. > Maybe we have to change the script that applies the ENV to the .conf file > > > Enrico > > > > > Thanks, > > Kai > > On Dec 13, 2022 at 9:03 PM +0800, Yunze Xu , > > wrote: > > > For example, when running a standalone (without TLS enabl

[DISCUSSIONS] Should we use AUTO_PRODUCE schema?

2022-12-13 Thread Yunze Xu
Hi all, Pulsar supports AUTO_PRODUCE schema, but this feature was introduced at an early time [1] when there was no PIP. I have read the documents [2] and found the example scenario. > Suppose that: > - You have a producer processing messages from a Kafka topic K. > - You have a Pulsar topic P, a

[VOTE] Pulsar Client Python Release 3.0.0 Candidate 1

2022-12-13 Thread Yunze Xu
This is the first release candidate for Apache Pulsar Client Python, version 3.0.0. It fixes the following issues: https://github.com/apache/pulsar-client-python/milestone/1?closed=1 *** Please download, test and vote on this release. This vote will stay open for at least 72 hours *** Python whe

Re: [DISCUSSIONS] Should we use AUTO_PRODUCE schema?

2022-12-13 Thread Yunze Xu
PM 丛搏 wrote: > > Hi, Yunze: > > Yunze Xu 于2022年12月14日周三 02:26写道: > > > First, how do you guarantee the schema can be used to encode the raw > > bytes whose format is unknown? > I think this is what the user needs to ensure that the user knows all > the schema from the Ka

Re: [VOTE] Pulsar Client Python Release 3.0.0 Candidate 1

2022-12-13 Thread Yunze Xu
github.com/apache/pulsar-client-python/blob/main/version.txt > > > -- > Matteo Merli > > > On Tue, Dec 13, 2022 at 8:10 PM Yunze Xu wrote: > > > > This is the first release candidate for Apache Pulsar Client Python, > > version 3.0.0. > > > > It f

Re: [DISCUSSIONS] Should we use AUTO_PRODUCE schema?

2022-12-14 Thread Yunze Xu
bytes are > generated by a specific schema. I think this is something that users > should consider rather than something that pulsar should guarantee > because you can't control the data sent by users that is generated by > this schema only for client verification. so, we don

Re: [DISCUSSIONS] Should we use AUTO_PRODUCE schema?

2022-12-14 Thread Yunze Xu
> > > > The performance problem of the AUTO_PRODUCE schema is that the > > > validation happens twice and it cannot be controlled. > > > > Our data verification is the behavior of the client, not the behavior > > of the broker. Therefore, we cannot effective

Re: [DISCUSSIONS] Should we use AUTO_PRODUCE schema?

2022-12-14 Thread Yunze Xu
4, 2022 at 10:37 PM 丛搏 wrote: > > Yunze Xu 于2022年12月14日周三 20:37写道: > > > > > how do you can create two Student.class in one java process? and use > > the same namespace? > > > > Could you give an example to show how `AUTO_PRODUCE` schema makes a > >

[VOTE] Pulsar Client Python Release 3.0.0 Candidate 2

2022-12-14 Thread Yunze Xu
This is the second release candidate for Apache Pulsar Client Python, version 3.0.0. It fixes the following issues: https://github.com/apache/pulsar-client-python/milestone/1?closed=1 *** Please download, test and vote on this release. This vote will stay open for at least 72 hours *** Python wh

Re: [VOTE] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-15 Thread Yunze Xu
> > Best, > > > > > > Mattison > > > > > > On Dec 5, 2022, 15:09 +0800, Zike Yang , wrote: > > > > > > > +1(non-binding) > > > > > > > > > > > > > > Best, > > > > > > >

Re: [VOTE] Pulsar Client Python Release 3.0.0 Candidate 2

2022-12-15 Thread Yunze Xu
eel file on Mac ARM, publishing and consuming messages > > * Wheel file on Alpine Linux, publishing and consuming messages > > > > > > -- > > Matteo Merli > > > > > >> On Wed, Dec 14, 2022 at 7:52 AM Yunze Xu > >> wrote: > >> > >&

Re: [DISCUSS] Modify MessageIdImpl and BatchMessageIdImpl compareTo(MessageId o) method

2022-12-15 Thread Yunze Xu
roduced by the user using the current compareTo() method manually. I > > > don’t think this is a breaking change. On the contrary, it is a bug > > > that needs to be fixed. Because we cannot guarantee that everyone can > > > find the problem of compareTo()

PIP-229: Add a common interface to get fields of MessageIdData

2022-12-15 Thread Yunze Xu
Hi all, I've opened a PIP to discuss: https://github.com/apache/pulsar/issues/18950 Currently the `MessageId` interface is not friendly to developers of Pulsar core and ecosystems. There is no abstraction of the `MessageIdData` defined in `PulsarApi.proto`. This proposal aims at solving this pro

Re: [VOTE] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-15 Thread Yunze Xu
e/pulsar/pull/18890/files [2] https://lists.apache.org/thread/x52zpwlo8pxzp81oxllh5vw82kyrzgpk On Fri, Dec 16, 2022 at 11:34 AM 丛搏 wrote: > > Although unrelated, it adds a lot of confusing code. > > Thanks, > Bo > > Yunze Xu 于2022年12月16日周五 08:05写道: > > >

Re: [DISCUSS] Modify MessageIdImpl and BatchMessageIdImpl compareTo(MessageId o) method

2022-12-15 Thread Yunze Xu
it clearly in > https://lists.apache.org/thread/mbrpjsgrgwrlkdpvkk738jxnlk7rf4qk and > modify it. But either newly opened PIP or in > https://lists.apache.org/thread/mbrpjsgrgwrlkdpvkk738jxnlk7rf4q, my > point is that we need to fix it, not compatible with it. > > Thanks, > Bo > > Yunze Xu 于2022年1

Re: [DISCUSS] Modify MessageIdImpl and BatchMessageIdImpl compareTo(MessageId o) method

2022-12-15 Thread Yunze Xu
In short, this change is okay for me. But since it's a breaking change, the Pulsar PMCs should be responsible to make the idea pass. Thanks, Yunze On Fri, Dec 16, 2022 at 11:53 AM Yunze Xu wrote: > > PIP is required when you want to make a breaking change. [1] What you > think

Re: [VOTE] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-15 Thread Yunze Xu
If this breaking change can pass the PMC votes, I will keep the new semantics in PIP-229. Otherwise, it would not make sense to adopt the new semantics in PIP-229. Thanks, Yunze On Fri, Dec 16, 2022 at 11:46 AM Yunze Xu wrote: > > I cannot find any confusing code from the proposal itself.

Re: [DISCUSSIONS] Should we use AUTO_PRODUCE schema?

2022-12-15 Thread Yunze Xu
distinguish it from BYTES rather than delete or deprecate it. > > Thanks, > Bo > > > Yunze Xu 于2022年12月14日周三 23:36写道: > > > > > Why not use the following code with a BYTES producer in your case? > > > > ```java > > var schema0 = Schema.AVRO(SchemaDefi

[VOTE] Pulsar Client Python Release 3.0.0 Candidate 3

2022-12-15 Thread Yunze Xu
This is the third release candidate for Apache Pulsar Client Python, version 3.0.0. It fixes the following issues: https://github.com/apache/pulsar-client-python/milestone/1?closed=1 *** Please download, test and vote on this release. This vote will stay open for at least 72 hours *** Python whe

Re: [VOTE] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-15 Thread Yunze Xu
hMessageIdImpl` and `MessageIdImpl` > `compareTo()` method, now I have no problem with this PIP > +1 (non-binding) > Sorry to bother this PIP vote. > > Thanks, > Bo > > Yunze Xu 于2022年12月16日周五 11:58写道: > > > > If this breaking change can pass the PMC votes, I will keep th

[DISCUSS] PIP-229: Add a common interface to get fields of MessageIdData

2022-12-15 Thread Yunze Xu
Hi all, I've opened a PIP to discuss: https://github.com/apache/pulsar/issues/18950 Currently the `MessageId` interface is not friendly to developers of Pulsar core and ecosystems. There is no abstraction of the `MessageIdData` defined in `PulsarApi.proto`. This proposal aims at solving this pro

Re: [DISCUSS] PIP-230: Modify MessageIdImpl and BatchMessageIdImpl compareTo(MessageId o) method

2022-12-16 Thread Yunze Xu
+1, LGTM but it should only be applied in 2.12.0 or later. Thanks, Yunze On Fri, Dec 16, 2022 at 6:26 PM Enrico Olivelli wrote: > > I support this solution. > > Unfortunately it is better to not port this to the previous versions > because it is a behaviour change. > > Thanks for driving this ef

Re: [DISCUSS] PIP-229: Add a common interface to get fields of MessageIdData

2022-12-17 Thread Yunze Xu
it? > > All I can think of is PulsarApiMessageId extend > TopicMessageId(PIP-224[1]) right? > > Thanks, > Bo > [1] https://github.com/apache/pulsar/issues/18616 > > Yunze Xu 于2022年12月16日周五 15:59写道: > > > > Hi all, > > > > I've opened a PIP to d

Re: [DISCUSS] PIP-229: Add a common interface to get fields of MessageIdData

2022-12-18 Thread Yunze Xu
. LedgerId and EntryId > are in PulsarApiMessageId, why shouldn't `topicName` be added in? > > Thanks, > Bo > > Yunze Xu 于2022年12月18日周日 14:23写道: > > > > Hi Bo, > > > > Because the topic name is not a part of MessageIdData. It's only used

Re: [DISCUSS] PIP-229: Add a common interface to get fields of MessageIdData

2022-12-18 Thread Yunze Xu
of useless fields at > > the network layer and not added to MessageIdData. LedgerId and EntryId > > are in PulsarApiMessageId, why shouldn't `topicName` be added in? > > > > Thanks, > > Bo > > > > Yunze Xu 于2022年12月18日周日 14:23写道: > > > > >

Re: [PROPOSAL] Website precommit and move the source of docs to the site repo

2022-12-18 Thread Yunze Xu
+1. The most significant point to me is that we can preview all the content of the website without synchronizing contents from the apache/pulsar repo. Thanks, Yunze On Mon, Dec 19, 2022 at 9:53 AM Li Li wrote: > > +1, That’s a good idea. > > > On Dec 16, 2022, at 07:07, tison wrote: > > > > Hi,

Re: [DISCUSS] PIP-229: Add a common interface to get fields of MessageIdData

2022-12-18 Thread Yunze Xu
needed. > > Thanks, > Penghui > > On Mon, Dec 19, 2022 at 11:45 AM Yunze Xu > wrote: > > > Hi Penghui, > > > > > It looks like LedgerHandleAdv to LedgerHandle in the bookkeeper. > > > > It's different. IMO, LedgerHandleAdv to LedgerHandle, or

Re: [VOTE] PIP-228: Refactor the information architecture of Pulsar client docs

2022-12-19 Thread Yunze Xu
+1 (non-binding) Thanks, Yunze On Tue, Dec 20, 2022 at 3:06 PM Zike Yang wrote: > > +1 (non-binding) > > Thanks, > Zike Yang > > On Tue, Dec 13, 2022 at 4:38 PM Jun Ma wrote: > > > > Hi all, > > > > I'm going to start the vote for PIP-228 [Refactor the information > > architecture of Pulsar cl

Re: [DISCUSS] PIP-229: Add a common interface to get fields of MessageIdData

2022-12-19 Thread Yunze Xu
sarApiMessageId = (PulsarApiMessageId) messageId` > > These are just some of my views, and it will not block this PIP. > > Thanks, > Bo > > Yunze Xu 于2022年12月19日周一 10:41写道: > > > > Hi Bo, > > > > For a single-topic consumer, the MessageId implement

Re: [DISCUSS] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-21 Thread Yunze Xu
If the interface is designed in this way, it may be simpler, easier to > understand, and more intuitive for users, and MessageID will not be > coupled with TopicName. > > because this PIP has already initiated a VOTE, so I will sync this > reply to PIP-224-VOTE[0] > > Thanks, &

Re: [DISCUSS] Change the default IO threads and listener threads of Java Client

2022-12-21 Thread Yunze Xu
I have a concern about the message ordering. If we have more than 1 listener thread by default, could messages from the same topic be passed to different listener threads? Thanks, Yunze On Wed, Dec 21, 2022 at 11:12 AM 丛搏 wrote: > > +1 > Our default configuration is best for most users. Multiple

Re: [DISCUSS] [GO-Client] The behavior of Seek is different with Java Client

2022-12-21 Thread Yunze Xu
+1 to me. It's a breaking change, but I think it's more like a wrong implementation in Go client. Thanks, Yunze On Wed, Dec 21, 2022 at 5:00 PM Jiaqi Shen wrote: > > Hi all. > > I noticed that the behavior of go client Seek is different with Java > client, which may confuse the users. > > More d

Re: [DISCUSS] Introduce oshi library to sensory OS resources

2022-12-21 Thread Yunze Xu
+1 Thanks, Yunze On Wed, Dec 21, 2022 at 5:02 PM houxiaoyu wrote: > > +1 > > Yubiao Feng 于2022年12月21日周三 11:59写道: > > > +1 > > > > On Mon, Dec 19, 2022 at 6:19 PM wrote: > > > > > > > > Hi, All > > > > > > I would like to introduce a new library oshi[1] to help Apache Pulsar > > > sensory OS re

Re: [VOTE] Pulsar Release 2.9.4 Candidate 3

2022-12-21 Thread Yunze Xu
+1 (non-binding) - Checked the signature - Build from source - Start standalone with KoP 2.9.3.20 - Verified Pulsar client 2.9.4, master (05e6f5e3), and Kafka clients 3.3.1 (through KoP) Thanks, Yunze On Mon, Dec 19, 2022 at 11:13 AM Xiangying Meng wrote: > > +1 (non-binding) > > - Start standa

Re: [DISCUSS] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-21 Thread Yunze Xu
e > > topic name for a partition. It could be a very confusing thing from my > > experience when I explained the "partition" concept in community. > > if using `TopicMessageId` also has the same problem, why we need to > use `TopicMessageId` not `MessageId` &

Re: [VOTE] Pulsar Release 2.11.0 Candidate-3

2022-12-21 Thread Yunze Xu
+1 (non-binding) - Checked the signature - Build from source - Start standalone with KoP master (1e5433d) - Verify Pulsar client 2.11.0, Kafka client 3.3.1 (through KoP) - Verify standalone with ZooKeeper as the metadata store - Verify Docker images Thanks, Yunze On Mon, Dec 19, 2022 at 12:47 P

Re: [DISCUSS] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-21 Thread Yunze Xu
topicMessageId)` for a > single topic consumer is also strange. My point is either TopicName > belongs to MessageId or separate the two. it's not a good interface > implementation to couple them together. Very unclear. > > Thanks, > Bo > > Yunze Xu 于2022年12月21日周三 22

Re: [DISCUSS] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-21 Thread Yunze Xu
> CompletableFuture seekAsync(TopicMessageId topicMessageId); > > > ``` > > > > > > Users can use > > > ``` > > > MessageId.fromByteArrayWithTopic() > > > ``` > > > or > > > ``` > > > TopicMessageId.fromByteArray(); &g

[VOTE] PIP-229: Add a common interface to get fields of MessageIdData

2022-12-22 Thread Yunze Xu
Hi all, I'm starting the VOTE for PIP-229: Add a common interface to get fields of MessageIdData: https://github.com/apache/pulsar/issues/18950 Discussion thread: https://lists.apache.org/thread/25rzflmkfmvxhf3my0ombnbpv7bvgy32 The vote will be open for at least 3 days. Thanks, Yunze

Re: [DISCUSS] Move GitHub discussion notifications to commits@

2022-12-22 Thread Yunze Xu
+1 Thanks, Yunze On Fri, Dec 23, 2022 at 6:04 AM Michael Marshall wrote: > > Hello Pulsar Community, > > We've had GitHub discussions enabled for a while now. I think they're > a helpful way for users to ask questions. I also think they are > creating too many notifications to the dev@ list, whi

Re: [VOTE] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-22 Thread Yunze Xu
ageID will not be > coupled with TopicName. > > Thanks, > Bo > > Yunze Xu 于2022年12月16日周五 15:31写道: > > > > Yeah, it's an implementation detail and I will keep the same semantics > > with the latest master when I push my PR. > > > > Thanks, > > Yunze

Re: [VOTE] Pulsar Client Python Release 3.0.0 Candidate 3

2022-12-26 Thread Yunze Xu
hon .whl file (pip3 install > >>> pulsar_client-3.0.0-cp38-cp38-macosx_10_15_universal2.whl) on macOS > >> 13.0.1 > >>> (22A400) > >>> - Start the standalone (build from the master branch) > >>> - Start consumer (python3 ./examples/

Re: [VOTE] Pulsar Client Python Release 3.0.0 Candidate 3

2022-12-26 Thread Yunze Xu
t; files and release files. > > BR, > Zike Yang > > On Mon, Dec 26, 2022 at 5:13 PM Yunze Xu wrote: > > > > I'm closing this vote. > > > > 3 binding +1s: > > * Matteo > > * Penghui > > * Jiwei > > > > 1 non-binding +1:

[ANNOUNCE] Apache Pulsar Client Python 3.0.0 released

2022-12-27 Thread Yunze Xu
The Apache Pulsar team is proud to announce Apache Pulsar Client Python version 3.0.0. Pulsar is a highly scalable, low latency messaging platform running on commodity hardware. It provides simple pub-sub semantics over topics, guaranteed at-least-once delivery of messages, automatic cursor manage

Re: [VOTE] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-27 Thread Yunze Xu
at 1:22 AM Enrico Olivelli wrote: > > +1 (binding) > > Enrico > > Il Ven 23 Dic 2022, 05:46 Yunze Xu ha > scritto: > > > It needs the 3rd binding +1 yet. Could anyone else take a look? > > > > Thanks, > > Yunze > > > > On

Re: [VOTE] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-28 Thread Yunze Xu
ha scritto: > > > > +1 (binding) > > > > Thanks, > > Hang > > > > Yunze Xu 于2022年12月28日周三 11:53写道: > > > > > > After discussing with @hangc0276 offline, I decided to add a > > > `Schema` implementation in this proposal to serialize >

Re: [DISCUSS] PIP-234: Support using shared thread pool across multiple Pulsar client instance

2022-12-28 Thread Yunze Xu
> Maybe we can add the new API to the pulsar-client-original module? like we > currently do. Adding stable APIs to the pulsar-client-original module that represents the internal implementations is not a good way IMO. I'm wondering why we need to configure a Netty EventLoop? Is there any alternati

Re: [DISCUSS] Reject create non-existent persistent partitions.

2022-12-29 Thread Yunze Xu
+1 Thanks, Yunze On Thu, Dec 29, 2022 at 5:31 PM Zike Yang wrote: > > Hi, Mattison, > > > Should we allow the user to create the non-partitioned topic name like > > `persistent://tenant/namespace/localname-partition-0`? > > +1 for rejecting this operation. Otherwise, the same issue will also >

Re: [VOTE] Pulsar Node.js Client Release 1.8.0 Candidate 2

2022-12-29 Thread Yunze Xu
+1 (binding) - Check signatures - Install from source - Install the NPM package via `npm install pulsar-client@1.8.0-rc.2` - Verify the producer, consumer, custom_logger examples with both two installation methods bove However, I'm still wondering whether we still need to upload the `napi-xx

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2022-12-31 Thread Yunze Xu
Defining `AutoConsume` as -3 is somehow strange. Could you clarify if backward compatibility is guaranteed? i.e. if the new Pulsar client uploaded the AUTO_CONSUME schema to the broker, can the old Pulsar clients produce or consume the same topic anymore? Thanks, Yunze On Fri, Dec 30, 2022 at 11:

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-02 Thread Yunze Xu
the error message as IncompatibleSchemaException (" > Topic does not have a schema to check "). The bug at issue17354 is not > fixed in this case. > All the other cases will be normal. > > Yunze Xu 于2022年12月31日周六 20:23写道: > > > Defining `AutoConsume` as -3 is

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-03 Thread Yunze Xu
rule on a topic with AUTO_CONSUME > > schema? > > it's only the consumer schema compatibility check, not on topic. if a > consume with AUTO_CONSUME schema will do any compatibility check > > Thanks, > Bo > > Yunze Xu 于2023年1月3日周二 10:16写道: > > > > W

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-03 Thread Yunze Xu
. But I'm still wondering if we can avoid the API changes. I will look deeper into this issue. Thanks, Yunze On Wed, Jan 4, 2023 at 12:12 AM Enrico Olivelli wrote: > > Il Mar 3 Gen 2023, 14:37 Yunze Xu ha scritto: > > > Hi Bo, > > > > I got it now. The PIP title

Re: [VOTE] Pulsar Release 2.11.0 Candidate-5

2023-01-03 Thread Yunze Xu
I found there are both apache-pulsar-shell-2.11.0-bin.tar.gz and apache-pulsar-shell-2.11.0-bin.zip, are they duplicates? Thanks, Yunze On Mon, Jan 2, 2023 at 8:01 PM guo jiwei wrote: > > This is the fifth release candidate for Apache Pulsar, version 2.11.0. > > This release contains 1617 commit

Re: [VOTE] Pulsar Release 2.11.0 Candidate-5

2023-01-03 Thread Yunze Xu
I noticed here is the discussion about this part in Slack > https://apache-pulsar.slack.com/archives/C5ZSVEN4E/p1672411025574369 > > Penghui > > On Wed, Jan 4, 2023 at 2:35 PM Yunze Xu > wrote: > > > I found there are both apache-pulsar-shell-2.11.0-bin.tar.gz and

Re: [VOTE] Pulsar Release 2.11.0 Candidate-5

2023-01-04 Thread Yunze Xu
license issues which will block users move > to the new version. > > Thanks, > Penghui > > > > On Wed, Jan 4, 2023 at 3:36 PM Yunze Xu > wrote: > > > +0 (binding) > > > > - Checked the signature > > - Build from source (Java 17, Ubuntu 20.04 W

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-04 Thread Yunze Xu
On Wed, Jan 4, 2023 at 11:07 AM Yunze Xu wrote: > > Modifying the subscribe request is better than exposing AUTO_CONSUME > schema type IMO. The negative value of a schema type, like BYTES, > AUTO_PRODUCE, means this schema type should only be used internally. > Adding the negativ

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-04 Thread Yunze Xu
I > upload negative schema types only changes the proto, if using `schema > type = NONE` and `schema data = "AUTO_CONSUME" `, every different > language client will code the special logic. This special logic can > easily be ignored. > > Thanks, > Bo > > Yunze Xu 于2

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-04 Thread Yunze Xu
ribe request and > compatibility seems to be fine. You guys can have a look at my PR ( > https://github.com/apache/pulsar/pull/17449). > > Yunze Xu 于2023年1月4日周三 21:31写道: > > > > Why can't we upload negative schema types? > > > > I want to avoid the changes t

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-04 Thread Yunze Xu
Thanks, Yunze On Wed, Jan 4, 2023 at 11:49 PM Yunze Xu wrote: > > Could you also update the PIP issue? This solution is totally > different from your original proposal. Since it still introduces > changes to `PulsarApi.proto`, it also requires a PIP (we can reuse > this one). >

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-04 Thread Yunze Xu
ight. Yunze. You mean that in > PulsarApi.proto, take `optional bool is_auto_consume_schema = 6 [default = > false]; ` in CommandSubscribe instead of Schema? But shouldn't > schema-related stuff be in Schema? > > Thanks, > Sinan > > Yunze Xu 于 2023年1月5日周四 上午12:31写道: >

Re: [VOTE] Pulsar Node.js Client Release 1.8.0 Candidate 2

2023-01-05 Thread Yunze Xu
Hi Nozomi, AFAIK, the napi-xxx.tar.gz will be uploaded to SVN so that `npm install` can download these pre-built binaries directly. I didn't look deeper into the `npm install` process, it might also download the pre-built binaries from the temporary download links from the candidate directory. Th

Re: [Vote] PIP-231: Add metric for topic load failed

2023-01-05 Thread Yunze Xu
+1 (binding) Thanks, Yunze On Thu, Jan 5, 2023 at 5:45 PM Jiuming Tao wrote: > > bump > > Jiuming Tao 于2022年12月27日周二 16:27写道: > > > Hello pulsar community, > > > > I'm starting the VOTE for PIP-231: Add metric for topic load failed. > > > > Motivation: > > Currently, we have a `topic_load_times

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-05 Thread Yunze Xu
e PIP issue and title, you guys can have a look. issue19113 > <https://github.com/apache/pulsar/issues/19113> > I added `check_schema_compatibility` in CommandSubscribe, and I also made > many other changes. > > Yunze Xu 于2023年1月5日周四 14:33写道: > > > It's not rel

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-05 Thread Yunze Xu
pdated the PIP issue and title, you guys can have a look. issue19113 > > <https://github.com/apache/pulsar/issues/19113> > > I added `check_schema_compatibility` in CommandSubscribe, and I also made > > many other changes. > > > > Yunze Xu 于2023年1月5日周四 14:33写道

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-06 Thread Yunze Xu
, don't create sub command with `AUTO_CONSUME` and bring this > problem. We have written a lot of complicated code to solve this > historical problem, If you add new fields it will be more unacceptable > than negative schema type. We should choose a simpler and more direct >

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-06 Thread Yunze Xu
6, 2023 at 6:23 PM Yunze Xu wrote: > > Hi Bo, > > I don't think it's a bad implementation. Instead, it's a natural > thought that if a schema info won't be persisted into the schema > registry, there is no need to carry the schema info in > Comman

Re: [VOTE] PIP-239: Retry Mechanism per Message title

2023-01-08 Thread Yunze Xu
+1 (binding) Thanks, Yunze On Sun, Jan 8, 2023 at 4:46 PM Zike Yang wrote: > > +1 (non-binding) > > Thanks, > Zike Yang > > On Sun, Jan 8, 2023 at 3:22 PM Haiting Jiang wrote: > > > > +1 binding > > > > Although I think we should use more unique keys for system properties in > > pulsar, > > e.

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-08 Thread Yunze Xu
ibing. So > > we should set a "*default*" schemaInfo(schemaType = AUTO_CONSUME) for > > AutoConsumeSchema, > > this is because schemaInfo is also null when `si.getType` is > > SchemaType.BYTES or SchemaType.NONE. > > And checkSchemaCompatibility can be set in

Re: [DISCUSS] Should we bump up Node.js Client for C++ Client's upgrade

2023-01-09 Thread Yunze Xu
My answer is yes. Some bug fixes might be included in a newer C++ client. But I think we don't need to start a release in a hurry once a newer C++ client is released. Thanks, Yunze On Tue, Jan 10, 2023 at 3:03 PM Zike Yang wrote: > > Hi, Yuto Furuta, > > Currently, the latest Nodejs client 1.8.0

Re: [DISCUSS] PIP-241: TopicEventListener / topic events for the BrokerService

2023-01-13 Thread Yunze Xu
I have some questions about the EventProcessingResult: 1. What's the difference between FAILURE and NOT_ALLOWED? 2. If we need to return the `message`, which indicates the error IIUC, would it be better to replace the returned value with a checked exception? Thanks, Yunze On Fri, Jan 13, 2023 at

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-15 Thread Yunze Xu
> } > > > > return null; > > > > > > > > } > > > > > > > > ``` > > > > And checkSchemaCompatibility must be set in > > > > `org.apache.pulsar.common.protocol.Commands#newSubscribe`, > > > > and we

Re: [DISCUSS] PIP-241: TopicEventListener / topic events for the BrokerService

2023-01-16 Thread Yunze Xu
in the PR. > > On Fri, Jan 13, 2023 at 4:08 AM Yunze Xu > wrote: > > > I have some questions about the EventProcessingResult: > > 1. What's the difference between FAILURE and NOT_ALLOWED? > > 2. If we need to return the `message`, which indicates the error IIUC,

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-16 Thread Yunze Xu
ema types like BYTES, AUTO and AUTO_PUBLISH. > > That sounds good to me to use a positive number for `AUTO_CONSUME` > in the protocol. Maybe 100 or something. > > Thanks, > Penghui > > On Mon, Jan 16, 2023 at 10:38 AM Yunze Xu > wrote: > > > Is there any probl

[ANNOUNCE] New Committer: Baodi Shi

2023-01-18 Thread Yunze Xu
The Project Management Committee (PMC) for Apache Pulsar has invited Baodi Shi (https://github.com/shibd) to become a committer and we are pleased to announce that he has accepted. Being a committer enables easier contribution to the project since there is no need to go via the patch submission pr

[DISCUSS] Redundant ServiceUrlProvider design and improper use of PIP-121

2023-01-18 Thread Yunze Xu
Hi all, Currently we have a `ServiceUrlProvider` interface to configure when constructing a PulsarClient in `ClientBuilder#serviceUrlProvider`. >From the beginning, I thought the `getServiceUrl` method is called each time the service URL is used, e.g. topic metadata lookup. However, the `getServic

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-18 Thread Yunze Xu
> > On Tue, Jan 17, 2023 at 9:11 PM SiNan Liu wrote: > > > The value of enum has been updated in the PIP issue > > <https://github.com/apache/pulsar/issues/19113>. > > > > Thanks, > > Sinan > > > > Yunze Xu 于2023年1月17日周二 12:11写道: > > &

Re: [DISCUSS] Code freeze for Pulsar 2.12

2023-01-19 Thread Yunze Xu
I would like to include PIP-224 (even and PIP-229) in the next major releases. These two PIPs have some impacts on the API and could bring many benefits to ecosystem developers. But unfortunately the first PR of PIP-224 [1] is still not reviewed by anyone. The code has already been added locally an

Re: [VOTE] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2023-01-19 Thread Yunze Xu
Hi guys, Please help review https://github.com/apache/pulsar/pull/19158, which has been pending for a while. I see 2.12.0 (or 3.0.0?) is coming, I hope we can include this PIP in the next major release. Thanks, Yunze On Wed, Dec 28, 2022 at 10:40 PM Yunze Xu wrote: > > Hi Enrico, &g

Re: [DISCUSS] Code freeze for Pulsar 2.12

2023-01-19 Thread Yunze Xu
In addition, next week is the Chinese New Year [1] in China and there is a long holiday (a week) for Chinese developers. I hope we can delay this release for a while. [1] https://en.wikipedia.org/wiki/Chinese_New_Year Thanks, Yunze On Fri, Jan 20, 2023 at 10:23 AM Yunze Xu wrote: > > I

Re: [ANNOUNCE] Bo Cong as new PMC member in Apache Pulsar

2023-01-19 Thread Yunze Xu
Congratulations! Thanks, Yunze On Fri, Jan 20, 2023 at 10:42 AM Dave Fisher wrote: > > Bo Cong! Congratulations! > > Best Regards, > Dave > > Sent from my iPhone > > > On Jan 18, 2023, at 5:50 AM, PengHui Li wrote: > > > > Hi all, > > > > The Apache Pulsar Project Management Committee (PMC) ha

Re: [VOTE] Pulsar Client C++ Release 3.1.1 Candidate 1

2023-01-20 Thread Yunze Xu
+1 (binding) * verify checksum and signatures * build from source (Ubuntu 20.04 WSL) * verify artifacts on Windows (`pulsar.dll` and `pulsarWithDeps.a` for both x86 and x64 architectures) * verify rpm, deb, apk packages in x86_64 architecture (`libpulsar.so` and `libpulsarwithdeps.a`) All the ver

Re: [DISCUSS] Redundant ServiceUrlProvider design and improper use of PIP-121

2023-01-20 Thread Yunze Xu
t; On Thu, Jan 19, 2023 at 3:43 PM Baodi Shi wrote: > > > Hi, Yunze: > > > > Obviously, the `ServiceUrlProvider` config is redundant. > > > > > > > Agree. In fact, The client already provides the updateServiceUrl method, > > which the user can use to implement

Re: [DISCUSS] Redundant ServiceUrlProvider design and improper use of PIP-121

2023-01-22 Thread Yunze Xu
Yes. The poll happens in the client's internal thread. Thanks, Yunze On Sun, Jan 22, 2023 at 6:56 PM Asaf Mesika wrote: > > and the client will poll the ConnectInfoProvider and check if something was > changed? > > On Fri, Jan 20, 2023 at 11:19 AM Yunze Xu > wro

Re: [VOTE] PIP-236: Record schema in the request and carry to the broker when subscribing with AUTO_CONSUME schema.

2023-01-23 Thread Yunze Xu
+1 (binding) Thanks, Yunze On Fri, Jan 20, 2023 at 7:06 PM SiNan Liu wrote: > > Hello community, > > I'm starting the VOTE for PIP-236: Record schema in the request and carry > to the broker when subscribing with AUTO_CONSUME schema. > > PIP link: > https://github.com/apache/pulsar/issues/19113

Re: [VOTE] Pulsar Client C++ Release 3.1.1 Candidate 1

2023-01-23 Thread Yunze Xu
Japan Corp. > E-mail: massa...@yahoo-corp.jp > > -Original Message- > From: Yunze Xu > Sent: Friday, January 20, 2023 5:53 PM > To: dev@pulsar.apache.org > Subject: Re: [VOTE] Pulsar Client C++ Release 3.1.1 Candidate 1 > > +1 (binding) > > * verify

Re: [VOTE] PIP-241: TopicEventListener / topic events for the BrokerService.

2023-01-25 Thread Yunze Xu
+1 (binding) Thanks, Yunze On Wed, Jan 25, 2023 at 4:22 PM Enrico Olivelli wrote: > > +1 (binding) > > Enrico > > Il giorno mer 25 gen 2023 alle ore 09:14 Nicolò Boschi > ha scritto: > > > > +1 (binding) > > > > Thanks, > > Nicolò Boschi > > > > > > Il giorno mar 24 gen 2023 alle ore 20:11 Mich

Re: [VOTE] Pulsar Client C++ Release 3.1.1 Candidate 2

2023-01-25 Thread Yunze Xu
+1 (binding) * verify checksum and signatures * build from source (Ubuntu 20.04 WSL) * verify artifacts on Windows (`pulsar.dll` and `pulsarWithDeps.a` for both x86 and x64 architectures) * verify rpm, deb, apk packages in x86_64 architecture (`libpulsar.so` and `libpulsarwithdeps.a`) All the ver

Re: [DISCUSS] Redundant ServiceUrlProvider design and improper use of PIP-121

2023-01-25 Thread Yunze Xu
it make sense that it will trigger the change > to the client instead of the client polling it and check it self if > something has changed? > > > On Mon, Jan 23, 2023 at 5:00 AM Yunze Xu > wrote: > > > Yes. The poll happens in the client's internal thread. > > &

Re: [DISCUSS] Redundant ServiceUrlProvider design and improper use of PIP-121

2023-01-29 Thread Yunze Xu
ed to `PulsarClient`, should also be exposed. Thanks, Yunze On Sun, Jan 29, 2023 at 11:57 PM Asaf Mesika wrote: > > Yunze - in summary - your proposal is to get rid of the ServiceUrlProvider > right? You just want to have setServiceUrl on the client instead? > > > On Wed, Jan 25, 2

Re: [DISCUSS] PIP-244: Refactor ByteBuf release method

2023-01-29 Thread Yunze Xu
I disagree with using `ReferenceCountUtil.safeRelease()` everywhere. The implementation is throwing any Throwable: ```java public static void safeRelease(Object msg) { try { release(msg); } catch (Throwable var2) { logger.warn("Failed to release a message: {}", msg, var2);

Re: [DISCUSS] PIP-242: Introduce enableStrictTopicName to reject creating topic with -partition- keyword.

2023-01-29 Thread Yunze Xu
> Otherwise, we end up having so many "feature flags" turned off, it's hard to > navigate and improve Pulsar. +1 to me. We already have too many feature flags. Thanks, Yunze On Mon, Jan 30, 2023 at 12:17 AM Asaf Mesika wrote: > > > > > Users can only create a topic with it if the topic is par

<    1   2   3   4   5   6   7   >