Re: [DISCUSS] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-21 Thread Yunze Xu
Hi Penghui and Bo, I realized this problem as well. No new APIs should be added. I added the new API mainly to express that the semantics of seek will be modified while the acknowledge semantics keeps not changed. Now I removed the new APIs and updated this proposal. Thanks, Yunze On Thu, Dec 2

[GitHub] [pulsar] momo-jun added a comment to the discussion: GCP Tiered Storage org.jclouds.rest.ResourceNotFoundException - Pulsar 2.6.1

2022-12-21 Thread GitBox
GitHub user momo-jun added a comment to the discussion: GCP Tiered Storage org.jclouds.rest.ResourceNotFoundException - Pulsar 2.6.1 Sorry for my late reply. As discussed with @RobertIndie, we both think the docs are okay by clarifying the attribute (bucket name) and providing an example (pul

[VOTE][PIP-226] Add JWKS support for AuthenticationProviderToken

2022-12-21 Thread Zixuan Liu
Hi all, Voting for https://github.com/apache/pulsar/issues/18798. Thanks, Zixuan

Re: [DISCUSS] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-21 Thread 丛搏
> Users can use > ``` > MessageId.fromByteArrayWithTopic() > ``` > or > ``` > TopicMessageId.fromByteArray(); > TopicMessageId.create(); > ``` I think this is a good point. `TopicName` don't in the `MessageId` means that multiConsumer must know that this topic is a partitioned topic and needs to a

Re: [VOTE] PIP-228: Refactor the information architecture of Pulsar client docs

2022-12-21 Thread PengHui Li
+1 (binding) Thanks, Penghui On Wed, Dec 21, 2022 at 10:18 AM 丛搏 wrote: > +1 (non-binding) > > Thanks, > Bo > > Yu 于2022年12月21日周三 09:51写道: > > > > +1 > > > > On Tue, Dec 20, 2022 at 3:12 PM Yunze Xu > > wrote: > > > > > +1 (non-binding) > > > > > > Thanks, > > > Yunze > > > > > > On Tue, Dec

Re: [DISCUSS] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-21 Thread PengHui Li
> Because `TopicMessageId` is constructed by the Pulsar Client library itself, which can guarantee `getOwnerTopic()` returns the correct topic name. The benefit of passing a `TopicMessageId` rather than the combination of a topic name and a `MessageId` is, users won't need to care about how to get

Re: [DISCUSS] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-21 Thread PengHui Li
> In short, `seek(msgId)` will call `seek(TopicMessageId)` if `msgId` is a `TopicMessageId`. Does it look like we don't need to add the following new APIs? ``` void seek(TopicMessageId topicMessageId) throws PulsarClientException; CompletableFuture seekAsync(TopicMessageId topicMessageId); ``` U

Re: [DISCUSS] Change the default IO threads and listener threads of Java Client

2022-12-21 Thread PengHui Li
> I notice that if changing the default value of conf `ioThreads` of pulsar > client, it does not just affect the num of io threads, and it also affects > these thread pools: > - internal executor of the client > - scheduled executor of the client > - HTTP client io thread pool of pulsar admin clie

RE: Re: [DISCUSS] forbid user to upload `BYTES` schema

2022-12-21 Thread netease
Hi Xiangying Meng, For the Pulsar client, even if users set BYTES schema, it won’t store anything in schema registry and we don’t need any special compatibility check. The key point here is, users can use admin api to upload a BYTES schema and it store a NONE type of schema in schema registry,

Re: [DISCUSS] Change the default IO threads and listener threads of Java Client

2022-12-21 Thread Yubiao Feng
Hi PengHui I notice that if changing the default value of conf `ioThreads` of pulsar client, it does not just affect the num of io threads, and it also affects these thread pools: - internal executor of the client - scheduled executor of the client - HTTP client io thread pool of pulsar admin clie

Re: [DISCUSS] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-21 Thread Yunze Xu
> but it needs to cast the `TopicMessageId` from `MessageId`, which is very > user-unfriendly. Sorry I think my proposal doesn't express it well. In my original thought, no cast is needed, please see the update in https://github.com/apache/pulsar/issues/18616. In short, `seek(msgId)` will call `

Re: [VOTE] Pulsar Release 2.11.0 Candidate-3

2022-12-21 Thread Yunze Xu
+1 (non-binding) - Checked the signature - Build from source - Start standalone with KoP master (1e5433d) - Verify Pulsar client 2.11.0, Kafka client 3.3.1 (through KoP) - Verify standalone with ZooKeeper as the metadata store - Verify Docker images Thanks, Yunze On Mon, Dec 19, 2022 at 12:47 P

Re: [DISCUSS] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-21 Thread 丛搏
> If you mean `msg.getTopicName()`, how can you declare it's better than > `msgId.getOwnerTopic()`? > It can. Because the `TopicMessageIdImpl` already contains the correct > topic. That's the point. ``` var msgId = (TopicMessageId) multiTopicsConsumer.getMessageId(); ``` if `msgId.getOwnerTopic(

Re: [DISCUSS] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-21 Thread Yunze Xu
> If messageID does not contain `TopicName`, the `TopicName` is best get from > msg. If you mean `msg.getTopicName()`, how can you declare it's better than `msgId.getOwnerTopic()`? > but it still cannot avoid `TopicName` for marking this `MessageID` belongs to > this topic. It can. Because the

Re: [VOTE] Pulsar Client Python Release 3.0.0 Candidate 3

2022-12-21 Thread 丛搏
+1 (non-binding) python version: 3.7.6 - Checked the signature - Install the python .whl file (pip3 install pulsar_client-3.0.0-cp37-cp37m-macosx_10_15_universal2.whl) on macOS 12.3.1 - Start the standalone (2.10.2) - Start consumer (python3 ./examples/consumer.py) - Start producer (python3 ./exam

Re: [DISCUSS] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-21 Thread 丛搏
Hi, Yunze: < ```java < var msg = multiTopicsConsumer.receive(); < var msgId = (TopicMessageId) multiTopicsConsumer.getMessageId(); < consumer.seek(msgId.getOwnerTopic(), msgId); < ``` the code can be like this: ```java var msg = anyConsumer.receive(); var msgId = anyConsume.getMessageId(); consu

Re: [VOTE] Pulsar Release 2.9.4 Candidate 3

2022-12-21 Thread Yunze Xu
+1 (non-binding) - Checked the signature - Build from source - Start standalone with KoP 2.9.3.20 - Verified Pulsar client 2.9.4, master (05e6f5e3), and Kafka clients 3.3.1 (through KoP) Thanks, Yunze On Mon, Dec 19, 2022 at 11:13 AM Xiangying Meng wrote: > > +1 (non-binding) > > - Start standa

Re: [VOTE] Pulsar Client Python Release 3.0.0 Candidate 3

2022-12-21 Thread PengHui Li
+1 - Checked the signature - Install the python .whl file (pip3 install pulsar_client-3.0.0-cp38-cp38-macosx_10_15_universal2.whl) on macOS 13.0.1 (22A400) - Start the standalone (build from the master branch) - Start consumer (python3 ./examples/consumer.py) - Start producer (python3 ./examples/p

Re: [DISCUSS] Introduce oshi library to sensory OS resources

2022-12-21 Thread Zike Yang
+1 Thanks, Zike Yang On Wed, Dec 21, 2022 at 5:06 PM Yunze Xu wrote: > > +1 > > Thanks, > Yunze > > On Wed, Dec 21, 2022 at 5:02 PM houxiaoyu wrote: > > > > +1 > > > > Yubiao Feng 于2022年12月21日周三 11:59写道: > > > > > +1 > > > > > > On Mon, Dec 19, 2022 at 6:19 PM wrote: > > > > > > > > > > > Hi,

Re: [DISCUSS] [GO-Client] The behavior of Seek is different with Java Client

2022-12-21 Thread Zike Yang
Hi Jiaqi, Thanks for raising up this discussion. This is the wrong behavior for the go client, and it will make it confusing for the go client user. +1 for fixing this wrong implementation even though it will introduce breaking changes. Thanks, Zike Yang On Wed, Dec 21, 2022 at 5:05 PM Yunze Xu

[VOTE] Pulsar Node.js Client Release 1.8.0 Candidate 1

2022-12-21 Thread Zike Yang
Hi everyone, This is the first release candidate for the Apache Pulsar Node.js client, version 1.8.0. It fixes the following issues: https://github.com/apache/pulsar-client-node/milestone/13?closed=1 Please download the source files and review this release candidate: - Download the source packag

Re: [DISCUSS] Introduce oshi library to sensory OS resources

2022-12-21 Thread Yunze Xu
+1 Thanks, Yunze On Wed, Dec 21, 2022 at 5:02 PM houxiaoyu wrote: > > +1 > > Yubiao Feng 于2022年12月21日周三 11:59写道: > > > +1 > > > > On Mon, Dec 19, 2022 at 6:19 PM wrote: > > > > > > > > Hi, All > > > > > > I would like to introduce a new library oshi[1] to help Apache Pulsar > > > sensory OS re

Re: [DISCUSS] [GO-Client] The behavior of Seek is different with Java Client

2022-12-21 Thread Yunze Xu
+1 to me. It's a breaking change, but I think it's more like a wrong implementation in Go client. Thanks, Yunze On Wed, Dec 21, 2022 at 5:00 PM Jiaqi Shen wrote: > > Hi all. > > I noticed that the behavior of go client Seek is different with Java > client, which may confuse the users. > > More d

Re: [DISCUSS] Introduce oshi library to sensory OS resources

2022-12-21 Thread houxiaoyu
+1 Yubiao Feng 于2022年12月21日周三 11:59写道: > +1 > > On Mon, Dec 19, 2022 at 6:19 PM wrote: > > > > > Hi, All > > > > I would like to introduce a new library oshi[1] to help Apache Pulsar > > sensory OS resources. It can help us to get away from the complex file > > manipulation and cross-platform c

Re: [DISCUSS] Change the default IO threads and listener threads of Java Client

2022-12-21 Thread Yunze Xu
I have a concern about the message ordering. If we have more than 1 listener thread by default, could messages from the same topic be passed to different listener threads? Thanks, Yunze On Wed, Dec 21, 2022 at 11:12 AM 丛搏 wrote: > > +1 > Our default configuration is best for most users. Multiple

[DISCUSS] [GO-Client] The behavior of Seek is different with Java Client

2022-12-21 Thread Jiaqi Shen
Hi all. I noticed that the behavior of go client Seek is different with Java client, which may confuse the users. More details here: https://github.com/apache/pulsar-client-go/issues/919. Overall, if go client executes `Seek(id)`, it will reset its position in `id` instead of `id+1` by default.

Re: [DISCUSS] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs

2022-12-21 Thread Yunze Xu
Hi Bo, If we have the `seek` API that accepts a topic name, how to use seek for a single topic consumer and multi-topics consumer will be different. ```java var msg = singleTopicConsumer.receive(); var msgId = singleTopicConsumer.getMessageId(); consumer.seek(msgId); ``` ```java var msg = multiT