Re: [DISCUSS] Change the default IO threads and listener threads of Java Client
+1 This change might bring thread number increment in case users create many clients, but too many pulsar clients run in one machine is not a good use case I think, so this change looks good to me. Thanks, Xiaoyu Hou 于2022年12月20日周二 12:25写道： > +1 > > My concern is whether this change will affect some users who are creating > many clients. I think we can wait for other users to confirm it. (If this > will be affected, maybe we can give it a max_io_thread_num and then expand > the size from 1 to max_io_thread_num when adding a new consumer or producer) > > > Best, > Mattison > On Dec 20, 2022, 11:17 +0800, PengHui Li , wrote: > > Hi all, > > > > I noticed the Java Client (I haven't checked other clients) uses 1 IO > > thread and 1 listener > > thread by default. It will require users to update the thread > configuration > > if they have > > multiple cores and desired high throughput. > > > > Here is the example that we change to 16 IO threads in > > openmessaging benchmark > > > https://github.com/openmessaging/benchmark/blob/master/driver-pulsar/pulsar.yaml#L22 > > > > We can apply the configuration of the threads based on the CPU cores. So > > that for the > > most common cases, users don't need to touch the thread configuration. > > > > ``` > > private int numIoThreads = Runtime.getRuntime().availableProcessors(); > > private int numListenerThreads = > Runtime.getRuntime().availableProcessors(); > > ``` > > > > WDYT? > > > > Thanks, > > Penghui >
Re: [PROPOSAL] Website precommit and move the source of docs to the site repo
+1, it makes sense to me. Thanks, Jiaqi Shen Yu 于2022年12月19日周一 20:57写道： > Hi tison, > > Thanks for raising this up! > > Our community had a similar discussion previously and chose to "keep" the > doc repo stay in the Pulsar main repo at that time. > > [1] lists the pros and cons of "keep" and "not keep" solutions. > > I'm +0 on this proposal because I think the total scores of these two > solutions are almost equal after weighing the pros and cons. > > > > [1] https://lists.apache.org/thread/mf2xwntfgn84dq78ksqv22jk3drq6xb3 > > > On Mon, Dec 19, 2022 at 5:40 PM tison wrote: > > > Thanks for your feedback! > > > > @Asaf > > > > > pre-commit > > > > I mean CI checks before merging a patch. Currently, we don't run checks > for > > the content before merging them. This causes a series of syntax errors > and > > broken links issues. If we hold docs under site2 folder in the main repo > > and then copied to the site repo, we have two places to build such CI > > checks. What's worse, the checks for the main repo will be quite > > cumbersome (that you do some if-else logic in the whole Pulsar CI > > workflows, and do the sync sequentially in that workflow). > > > > If we hold the source of docs only in the site repo, we can extend the > > "precommit" workflow[1] I added recently to check for syntax errors and > > broken links also. > > > > > What does the apache/pulsar-site repo contain today? > > > > It should be covered by the documentation guide page[2]. It holds the > > source of the official website and the user docs are synced from the main > > repo. > > > > > What content do we have today in the pulsar repo related to the site? > > > > After issue-18014[3] is done, we host only user docs and some JSON > metadata > > in the main repo, which is synced by site_syncer.py[4]. > > > > > Can you explain that better? Are you saying pulsar source JARs contain > > the documentation? > > > > No. Source JARs contain only the Java files and necessary copyrights > info. > > The source release is, for example, > > > > > https://archive.apache.org/dist/pulsar/pulsar-2.10.2/apache-pulsar-2.10.2-src.tar.gz > > , > > which is extracted to 173M where 129M is occupied by the site2 folder. > > > > This also affects when developers do git clone to clone the repo. > > > > > I mean, if you wish to document a bug fix in 2.9.x, for example, would > > you do it in the 2.9.x branch under site2/docs or > > site2/website/versioned_docs/2.9.5? > > > > This is another question. Ideally, we should have hosted versioned docs > > associated with the specific version to that branch, like Apache Flink > does > > as I mentioned[5]. But we do not, and actually the situation is we update > > the versioned docs under the master branch and thus, the docs can be > synced > > properly. > > > > See also the "Alternatives" section in the original email. > > > > @All > > > > Since we don't have objections to the possible cons listed above or any > new > > ones, I'm going to create a tracking issue later this week and show what > > will be changed in PRs for further review. > > > > Best, > > tison. > > > > [1] > > > > > https://github.com/apache/pulsar-site/blob/f7abc615d57d9846ed093922d24bff952dc0e838/.github/workflows/ci-precommit.yml > > [2] > > > > > https://pulsar.apache.org/contribute/document-contribution/#source-repositories > > [3] https://github.com/apache/pulsar/issues/18014 > > [4] > > > > > https://github.com/apache/pulsar-site/blob/f7abc615d57d9846ed093922d24bff952dc0e838/tools/pytools/lib/execute/site_syncer.py > > [5] https://github.com/apache/flink/tree/master/docs > > > > > > PengHui Li 于2022年12月19日周一 16:26写道： > > > > > +1 > > > > > > I support moving them to the website repo. > > > > > > Thanks, > > > Penghui > > > > > > On Mon, Dec 19, 2022 at 12:04 PM Yunze Xu > > > > wrote: > > > > > > > +1. The most significant point to me is that we can preview all the > > > > content of the website without synchronizing contents from the > > > > apache/pulsar repo. > > > > > > > > Thanks, > > > > Yunze > > > > > > > > On Mon, Dec 19, 2022 at 9:53 AM Li Li wrote: > > > > > > > > > > +1, That’s a good idea. > > > > > > > > > > > On Dec 16, 2022, at 07:07, tison wrote: > > > > > > > > > > > > Hi, > > > > > > > > > > > > After several works around the build flow of our official > > > > website[1][2][3], > > > > > > the content sync and site build flow is debuggable and > reproducible > > > > now. > > > > > > > > > > > > However, compared to other Apache projects' websites' project > > layouts > > > > and > > > > > > workflow, we still meet two challenges on the Pulsar site: > > > > > > > > > > > > 1. We don't have a pre-commit workflow for any website-related > > > changes. > > > > > > Thus, we don't detect broken links or syntax errors when > reviewing > > > new > > > > > > patches[4][5][6]. > > > > > > 2. The website's content is two-level down in > `site2/website-next` > > > for > > > > > > historical reasons, which i
[ANNOUNCE] Released Spring for Apache Pulsar, version 0.1.0
The Spring for Apache Pulsar team is proud to announce the initial GA minor release of the framework. The project aims to provide Spring-friendly APIs, building blocks, and programming models for writing Java applications that interact with Apache Pulsar. More details can be found here: https://spring.io/blog/2022/12/15/spring-for-apache-pulsar-0-1-0-available-now We would like to thank the contributors that made the release possible. Regards, The Spring for Apache Pulsar Team
Re: [ANNOUNCE] Released Spring for Apache Pulsar, version 0.1.0
That's awesome Chris, Many thanks to VMWare and all the contributors. This is a really great addition to the Apache Pulsar eco-system ! Christophe Le mar. 20 déc. 2022 à 18:09, Chris Bo a écrit : > The Spring for Apache Pulsar team is proud to announce the initial GA minor > release of the framework. > > The project aims to provide Spring-friendly APIs, building blocks, and > programming models for writing Java applications that interact with Apache > Pulsar. > > More details can be found here: > > https://spring.io/blog/2022/12/15/spring-for-apache-pulsar-0-1-0-available-now > > We would like to thank the contributors that made the release possible. > > Regards, > > The Spring for Apache Pulsar Team >
Re: [VOTE] PIP-228: Refactor the information architecture of Pulsar client docs
+1 On Tue, Dec 20, 2022 at 3:12 PM Yunze Xu wrote: > +1 (non-binding) > > Thanks, > Yunze > > On Tue, Dec 20, 2022 at 3:06 PM Zike Yang wrote: > > > > +1 (non-binding) > > > > Thanks, > > Zike Yang > > > > On Tue, Dec 13, 2022 at 4:38 PM Jun Ma wrote: > > > > > > Hi all, > > > > > > I'm going to start the vote for PIP-228 [Refactor the information > architecture of Pulsar client docs]( > https://github.com/apache/pulsar/issues/18822). > > > > > > And this is the original thread for discussion: > https://lists.apache.org/thread/bv6lwnt708dxst173knyzv2bfy4d1ox4. > > > > > > The vote will be open for at least three days. > > > > > > > > > Thank you. > > > Jun >
Re: [VOTE] PIP-228: Refactor the information architecture of Pulsar client docs
+1 (non-binding) Thanks, Bo Yu 于2022年12月21日周三 09:51写道： > > +1 > > On Tue, Dec 20, 2022 at 3:12 PM Yunze Xu > wrote: > > > +1 (non-binding) > > > > Thanks, > > Yunze > > > > On Tue, Dec 20, 2022 at 3:06 PM Zike Yang wrote: > > > > > > +1 (non-binding) > > > > > > Thanks, > > > Zike Yang > > > > > > On Tue, Dec 13, 2022 at 4:38 PM Jun Ma wrote: > > > > > > > > Hi all, > > > > > > > > I'm going to start the vote for PIP-228 [Refactor the information > > architecture of Pulsar client docs]( > > https://github.com/apache/pulsar/issues/18822). > > > > > > > > And this is the original thread for discussion: > > https://lists.apache.org/thread/bv6lwnt708dxst173knyzv2bfy4d1ox4. > > > > > > > > The vote will be open for at least three days. > > > > > > > > > > > > Thank you. > > > > Jun > >
Re: [DISCUSS] Change the default IO threads and listener threads of Java Client
+1 Our default configuration is best for most users. Multiple clients are a few cases. Thanks, Bo houxiaoyu 于2022年12月20日周二 16:02写道： > > +1 > > This change might bring thread number increment in case users create many > clients, but too many pulsar clients run in one machine is not a good use > case I think, so this change looks good to me. > > Thanks, > Xiaoyu Hou > > 于2022年12月20日周二 12:25写道： > > > +1 > > > > My concern is whether this change will affect some users who are creating > > many clients. I think we can wait for other users to confirm it. (If this > > will be affected, maybe we can give it a max_io_thread_num and then expand > > the size from 1 to max_io_thread_num when adding a new consumer or producer) > > > > > > Best, > > Mattison > > On Dec 20, 2022, 11:17 +0800, PengHui Li , wrote: > > > Hi all, > > > > > > I noticed the Java Client (I haven't checked other clients) uses 1 IO > > > thread and 1 listener > > > thread by default. It will require users to update the thread > > configuration > > > if they have > > > multiple cores and desired high throughput. > > > > > > Here is the example that we change to 16 IO threads in > > > openmessaging benchmark > > > > > https://github.com/openmessaging/benchmark/blob/master/driver-pulsar/pulsar.yaml#L22 > > > > > > We can apply the configuration of the threads based on the CPU cores. So > > > that for the > > > most common cases, users don't need to touch the thread configuration. > > > > > > ``` > > > private int numIoThreads = Runtime.getRuntime().availableProcessors(); > > > private int numListenerThreads = > > Runtime.getRuntime().availableProcessors(); > > > ``` > > > > > > WDYT? > > > > > > Thanks, > > > Penghui > >
Re: [PROPOSAL] Website precommit and move the source of docs to the site repo
Is it possible to come up with a compromised solution that has the pros of both sides but minimizes the side effect? I'm thinking maybe it's not necessary to sacrifice the current contribution process, as long as it can greatly reduce the load of back-end actions and source size. For example, if we only move out the versioned docs to the site repo but keep the source of the NEXT docs in the pulsar repo, does this help to win a large proportion of those pros when people can still contribute as usual? From: Jiaqi Shen Sent: Tuesday, December 20, 2022 17:15 To: dev@pulsar.apache.org Subject: Re: [PROPOSAL] Website precommit and move the source of docs to the site repo +1, it makes sense to me. Thanks, Jiaqi Shen Yu 于2022年12月19日周一 20:57写道： > Hi tison, > > Thanks for raising this up! > > Our community had a similar discussion previously and chose to "keep" the > doc repo stay in the Pulsar main repo at that time. > > [1] lists the pros and cons of "keep" and "not keep" solutions. > > I'm +0 on this proposal because I think the total scores of these two > solutions are almost equal after weighing the pros and cons. > > > > [1] https://lists.apache.org/thread/mf2xwntfgn84dq78ksqv22jk3drq6xb3 > > > On Mon, Dec 19, 2022 at 5:40 PM tison wrote: > > > Thanks for your feedback! > > > > @Asaf > > > > > pre-commit > > > > I mean CI checks before merging a patch. Currently, we don't run checks > for > > the content before merging them. This causes a series of syntax errors > and > > broken links issues. If we hold docs under site2 folder in the main repo > > and then copied to the site repo, we have two places to build such CI > > checks. What's worse, the checks for the main repo will be quite > > cumbersome (that you do some if-else logic in the whole Pulsar CI > > workflows, and do the sync sequentially in that workflow). > > > > If we hold the source of docs only in the site repo, we can extend the > > "precommit" workflow[1] I added recently to check for syntax errors and > > broken links also. > > > > > What does the apache/pulsar-site repo contain today? > > > > It should be covered by the documentation guide page[2]. It holds the > > source of the official website and the user docs are synced from the main > > repo. > > > > > What content do we have today in the pulsar repo related to the site? > > > > After issue-18014[3] is done, we host only user docs and some JSON > metadata > > in the main repo, which is synced by site_syncer.py[4]. > > > > > Can you explain that better? Are you saying pulsar source JARs contain > > the documentation? > > > > No. Source JARs contain only the Java files and necessary copyrights > info. > > The source release is, for example, > > > > > https://archive.apache.org/dist/pulsar/pulsar-2.10.2/apache-pulsar-2.10.2-src.tar.gz > > , > > which is extracted to 173M where 129M is occupied by the site2 folder. > > > > This also affects when developers do git clone to clone the repo. > > > > > I mean, if you wish to document a bug fix in 2.9.x, for example, would > > you do it in the 2.9.x branch under site2/docs or > > site2/website/versioned_docs/2.9.5? > > > > This is another question. Ideally, we should have hosted versioned docs > > associated with the specific version to that branch, like Apache Flink > does > > as I mentioned[5]. But we do not, and actually the situation is we update > > the versioned docs under the master branch and thus, the docs can be > synced > > properly. > > > > See also the "Alternatives" section in the original email. > > > > @All > > > > Since we don't have objections to the possible cons listed above or any > new > > ones, I'm going to create a tracking issue later this week and show what > > will be changed in PRs for further review. > > > > Best, > > tison. > > > > [1] > > > > > https://github.com/apache/pulsar-site/blob/f7abc615d57d9846ed093922d24bff952dc0e838/.github/workflows/ci-precommit.yml > > [2] > > > > > https://pulsar.apache.org/contribute/document-contribution/#source-repositories > > [3] https://github.com/apache/pulsar/issues/18014 > > [4] > > > > > https://github.com/apache/pulsar-site/blob/f7abc615d57d9846ed093922d24bff952dc0e838/tools/pytools/lib/execute/site_syncer.py > > [5] https://github.com/apache/flink/tree/master/docs > > > > > > PengHui Li 于2022年12月19日周一 16:26写道： > > > > > +1 > > > > > > I support moving them to the website repo. > > > > > > Thanks, > > > Penghui > > > > > > On Mon, Dec 19, 2022 at 12:04 PM Yunze Xu > > > > wrote: > > > > > > > +1. The most significant point to me is that we can preview all the > > > > content of the website without synchronizing contents from the > > > > apache/pulsar repo. > > > > > > > > Thanks, > > > > Yunze > > > > > > > > On Mon, Dec 19, 2022 at 9:53 AM Li Li wrote: > > > > > > > > > > +1, That’s a good idea. > > > > > > > > > > > On Dec 16, 2022, at 07:07, tison wrote: > > > > > > > > > > > > Hi, > > > > > > > > > >
Re: [PROPOSAL] Website precommit and move the source of docs to the site repo
Thanks for your feedback! @Yu Thanks for sharing the previous thread. I looped in @michaeljmarshall here. @Jun It's possible but causes a new shortcoming: Now you should tell the contributor that the versioned docs are different from the NEXT version docs, lol. If our developers don't complain about these separated sources. Like @Asaf comment: > We can take, let's say, five features and see if they were actually done in > the same PR or separate PR. I guess that most documentation is actually > updated separately. Thus, from that perspective, maybe it’s not a con. Then we can do this refactor thoroughgoing. Also, if we keep, somehow several sources in the main repo. We still have shortcomings: 1. Duplicated CI workflows. 2. Cumbersome preview scaffolding in the main repo. ... which is the original purpose I'd like to overcome. Best, tison. Jun Ma 于2022年12月21日周三 11:19写道： > Is it possible to come up with a compromised solution that has the pros of > both sides but minimizes the side effect? I'm thinking maybe it's not > necessary to sacrifice the current contribution process, as long as it can > greatly reduce the load of back-end actions and source size. For example, > if we only move out the versioned docs to the site repo but keep the source > of the NEXT docs in the pulsar repo, does this help to win a large > proportion of those pros when people can still contribute as usual? > > > From: Jiaqi Shen > Sent: Tuesday, December 20, 2022 17:15 > To: dev@pulsar.apache.org > Subject: Re: [PROPOSAL] Website precommit and move the source of docs to > the site repo > > +1, it makes sense to me. > > Thanks, > Jiaqi Shen > > > Yu 于2022年12月19日周一 20:57写道： > > > Hi tison, > > > > Thanks for raising this up! > > > > Our community had a similar discussion previously and chose to "keep" the > > doc repo stay in the Pulsar main repo at that time. > > > > [1] lists the pros and cons of "keep" and "not keep" solutions. > > > > I'm +0 on this proposal because I think the total scores of these two > > solutions are almost equal after weighing the pros and cons. > > > > > > > > [1] https://lists.apache.org/thread/mf2xwntfgn84dq78ksqv22jk3drq6xb3 > > > > > > On Mon, Dec 19, 2022 at 5:40 PM tison wrote: > > > > > Thanks for your feedback! > > > > > > @Asaf > > > > > > > pre-commit > > > > > > I mean CI checks before merging a patch. Currently, we don't run checks > > for > > > the content before merging them. This causes a series of syntax errors > > and > > > broken links issues. If we hold docs under site2 folder in the main > repo > > > and then copied to the site repo, we have two places to build such CI > > > checks. What's worse, the checks for the main repo will be quite > > > cumbersome (that you do some if-else logic in the whole Pulsar CI > > > workflows, and do the sync sequentially in that workflow). > > > > > > If we hold the source of docs only in the site repo, we can extend the > > > "precommit" workflow[1] I added recently to check for syntax errors and > > > broken links also. > > > > > > > What does the apache/pulsar-site repo contain today? > > > > > > It should be covered by the documentation guide page[2]. It holds the > > > source of the official website and the user docs are synced from the > main > > > repo. > > > > > > > What content do we have today in the pulsar repo related to the site? > > > > > > After issue-18014[3] is done, we host only user docs and some JSON > > metadata > > > in the main repo, which is synced by site_syncer.py[4]. > > > > > > > Can you explain that better? Are you saying pulsar source JARs > contain > > > the documentation? > > > > > > No. Source JARs contain only the Java files and necessary copyrights > > info. > > > The source release is, for example, > > > > > > > > > https://archive.apache.org/dist/pulsar/pulsar-2.10.2/apache-pulsar-2.10.2-src.tar.gz > > > , > > > which is extracted to 173M where 129M is occupied by the site2 folder. > > > > > > This also affects when developers do git clone to clone the repo. > > > > > > > I mean, if you wish to document a bug fix in 2.9.x, for example, > would > > > you do it in the 2.9.x branch under site2/docs or > > > site2/website/versioned_docs/2.9.5? > > > > > > This is another question. Ideally, we should have hosted versioned docs > > > associated with the specific version to that branch, like Apache Flink > > does > > > as I mentioned[5]. But we do not, and actually the situation is we > update > > > the versioned docs under the master branch and thus, the docs can be > > synced > > > properly. > > > > > > See also the "Alternatives" section in the original email. > > > > > > @All > > > > > > Since we don't have objections to the possible cons listed above or any > > new > > > ones, I'm going to create a tracking issue later this week and show > what > > > will be changed in PRs for further review. > > > > > > Best, > > > tison. > > > > > > [1] > > > > > >
Re: [DISCUSS] Introduce oshi library to sensory OS resources
+1 Thanks, Penghui On Mon, Dec 19, 2022 at 6:19 PM wrote: > > Hi, All > > I would like to introduce a new library oshi[1] to help Apache Pulsar > sensory OS resources. It can help us to get away from the complex file > manipulation and cross-platform compatibility issues in some operating > systems. > > code example: https://github.com/apache/pulsar/pull/18984 > > Please feel free to left comments. > > > Best, > Mattison > > > [1] https://github.com/oshi/oshi >
Re: [DISCUSS] Introduce oshi library to sensory OS resources
+1 (non-binding) Thanks, Bo PengHui Li 于2022年12月21日周三 11:48写道： > > +1 > > Thanks, > Penghui > > On Mon, Dec 19, 2022 at 6:19 PM wrote: > > > > > Hi, All > > > > I would like to introduce a new library oshi[1] to help Apache Pulsar > > sensory OS resources. It can help us to get away from the complex file > > manipulation and cross-platform compatibility issues in some operating > > systems. > > > > code example: https://github.com/apache/pulsar/pull/18984 > > > > Please feel free to left comments. > > > > > > Best, > > Mattison > > > > > > [1] https://github.com/oshi/oshi > >
Re: [DISCUSS] Introduce oshi library to sensory OS resources
+1 On Mon, Dec 19, 2022 at 6:19 PM wrote: > > Hi, All > > I would like to introduce a new library oshi[1] to help Apache Pulsar > sensory OS resources. It can help us to get away from the complex file > manipulation and cross-platform compatibility issues in some operating > systems. > > code example: https://github.com/apache/pulsar/pull/18984 > > Please feel free to left comments. > > > Best, > Mattison > > > [1] https://github.com/oshi/oshi >
Re: [DISCUSS] PIP-229: Add a common interface to get fields of MessageIdData
Hi, Yunze
> This interface is introduced mainly because we
> don't have a String parameter in seek and acknowledge before. It's a
> tradeoff between compatibility and complexity.
I think `MessageIdAdv` and adding `topic name` is more confusing for
non-partition-topic and redundant. If we don't couple TopicName and
MessageID together, It is better to change our interface design into
two parameters (topic name, MessageId). This is clearer and more
intuitive, and it is easier for users to understand MessageId.
MessageId also becomes simpler. I will sync this point to
PIP-224-DISCUSS [0].
I don't have any problem with this pip now, thanks for your reply.
+1 (non-binding)
Thanks,
Bo
[0] https://lists.apache.org/thread/jhqy65cdyxzmmxnfsjm8rv9pbk76noxy
Yunze Xu 于2022年12月20日周二 15:27写道：
>
> Hi Bo,
>
> > But when `topicName` is not in the `PulsarApiMessageId`, they should use
> > `messageId instanceof TopicMessageId` to get the `topicName`
>
> The scenario only happens for `acknowledge` APIs because I don't want
> to add more overloads in PIP-224. The new `seek` overloads in PIP-224
> just accept the `TopicMessageId` parameter.
>
> Combining the PIP-224 and PIP-229, the code will look like:
>
> ```java
> // MultiTopicsConsumerImpl
> void acknowledge(MessageId msgId) {
> if (!(msgId instanceof TopicMessageId)) {
> throw new PulsarClientException.NotAllowedException("");
> }
> consumers.get(((TopicMessageId)
> msgId).getOwnerTopic()).acknowledge(msgId);
> }
> void seek(TopicMessageId topicMessageId) {
> consumers.get(topicMessageId.getOwnerTopic()).seek(topicMessageId);
> }
> ```
>
> ```java
> // ConsumerImpl
> void acknowledge(MessageId msgId) {
> internalAcknowledge((MessageIdAdv) msgId); // we never need the
> owner topic now
> }
> void seek(TopicMessageId topicMessageId) {
> internalSeek((MessageIdAdv) topicMessageId);
> }
> ```
>
> That's why we need two interfaces. The use cases of TopicMessageId are
> very rare and limited. This interface is introduced mainly because we
> don't have a String parameter in seek and acknowledge before. It's a
> tradeoff between compatibility and complexity. But the use cases of
> MessageIdAdv are very common.
>
> Thanks,
> Yunze
>
> On Mon, Dec 19, 2022 at 9:00 PM 丛搏 wrote:
> >
> > Hi Yunze :
> >
> > I have no reason to must add the `getTopicName()` into the
> > `PulsarApiMessageId` (or named `MessageIdAdv `).
> > I just thought of a scenario that users would use. For the consumer
> > side, they don't know whether the topic is a partitioned topic or a
> > non-partitioned topic. Maybe they don't care about it. But when
> > `topicName` is not in the `PulsarApiMessageId`, they should use
> > `messageId instanceof TopicMessageId` to get the `topicName`, if this
> > `topicName` is in the `PulsarApiMessageId`, they only use the unified
> > logic to get the field from the messageId `PulsarApiMessageId
> > pulsarApiMessageId = (PulsarApiMessageId) messageId`
> >
> > These are just some of my views, and it will not block this PIP.
> >
> > Thanks,
> > Bo
> >
> > Yunze Xu 于2022年12月19日周一 10:41写道：
> > >
> > > Hi Bo,
> > >
> > > For a single-topic consumer, the MessageId implementations returned by
> > > receive() should be a MessageIdImpl or BatchedMessageIdImpl. Is there
> > > any reason to add a `getTopicName()` method for them (via extending
> > > TopicMessageId)? If yes, we have to use the default implementation of
> > > `getTopicName()` that returns null without modifying any existing
> > > code. If you don't want to return a null value, you have to add new
> > > logic that sets the topic explicitly when creating the received
> > > message.
> > >
> > > Thanks,
> > > Yunze
> > >
> > > On Sun, Dec 18, 2022 at 3:44 PM 丛搏 wrote:
> > > >
> > > > < For a single-topic consumer, wrapping the topic name is
> > > > < redundant and might break the existing behavior. In this case, if
> > > > < `PulsarApiMessageId` extends `TopicMessageId`, the `getTopicName()`
> > > > < method should return null, which is not a good design [1][2].
> > > >
> > > > For `TopicMessageIdImpl`, it is an original method. for
> > > > `PulsarApiMessageId` if extend `TopicMessageId` it is a new method for
> > > > any `MessageId` extend `PulsarApiMessageId`, why do we have to return
> > > > null? I think it just reduces the transmission of useless fields at
> > > > the network layer and not added to MessageIdData. LedgerId and EntryId
> > > > are in PulsarApiMessageId, why shouldn't `topicName` be added in?
> > > >
> > > > Thanks,
> > > > Bo
> > > >
> > > > Yunze Xu 于2022年12月18日周日 14:23写道：
> > > > >
> > > > > Hi Bo,
> > > > >
> > > > > Because the topic name is not a part of MessageIdData. It's only used
> > > > > to find the correct internal consumer of a multi-topics consumer.
> > > > >
> > > > > > All I can think of is PulsarApiMessageId extend
> > > > > > TopicMessageId(PIP-224[1]) right?
> > > > >
> > > > > No. The `TopicMessageId` could only be used in a multi-topics
> > > >
Re: [DISCUSS] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs
Hi Yunze, add `TopicMessageId ` will couple messageID and `topic name` together, which is very unclear for non-partition-topic. ``` void seek(String topicName, MessageId messageId) throws PulsarClientException; List> getLastTopicMessageId() throws PulsarClientException; ``` If the interface is designed in this way, it may be simpler, easier to understand, and more intuitive for users, and MessageID will not be coupled with TopicName. because this PIP has already initiated a VOTE, so I will sync this reply to PIP-224-VOTE[0] Thanks, Bo [0] https://lists.apache.org/thread/mbrpjsgrgwrlkdpvkk738jxnlk7rf4qk Yunze Xu 于2022年12月9日周五 14:33写道： > > Hi Jiaqi, > > Let's move to https://lists.apache.org/thread/mbrpjsgrgwrlkdpvkk738jxnlk7rf4qk > for the vote. > > Thanks, > Yunze > > On Fri, Dec 9, 2022 at 1:54 PM Jiaqi Shen wrote: > > > > This is make sense to me, +1 > > > > Thanks, > > Jiaqi Shen > > > > > > Yunze Xu 于2022年12月7日周三 13:51写道： > > > > > Hi Baodi, > > > > > > I decided not to change the behavior of the `negativeAcknowledge` > > > method. I just checked again that there is no exception signature for > > > this method and there is no asynchronous version like > > > `negativeAcknowledgeAsync`. To keep the API compatible, we should not > > > add an exception signature, which would be required if a > > > `PulsarClientException` was thrown. > > > > > > Thanks, > > > Yunze > > > > > > On Tue, Nov 29, 2022 at 10:12 PM Baodi Shi > > > wrote: > > > > > > > > Hi, Yunze: > > > > > > > > Thanks for your proposal. That Looks good to me. > > > > > > > > `negativeAcknowledge` also needs to add the same checks as the new > > > acknowledge interface. > > > > > > > > > This interface doesn't add any acknowledge overload because the > > > overloads are already too many. But it will make the behavior clear. > > > > I think since we exposed the TopicMessageId, it would be better to add > > > overloaded interfaces (even if the overloads are a lot). This can users to > > > clearly associate the use cases of MultiTopicConsumer and TopicMessageId. > > > > > > > > Also, while it's okay to use TopicMessageId param on a single consumer, > > > I guess we shouldn't allow users to use it. > > > > > > > > In this way, users are clearly aware that TopicMessageId is used when > > > using MultiTopicConsumer and MessageId is used when using > > > SingleTopicConsumer.(Maybe it's not a good idea) > > > > > > > > > > > > Thanks, > > > > Baodi Shi > > > > > > > > > 2022年11月29日 15:57，Yunze Xu 写道： > > > > > > > > > >> Is there a case where the user uses the messageId returned by the > > > > > producer to seek in the consumer? Is this a good behavior? > > > > > > > > > > Yes. I think it should be acceptable. To correct my previous point, > > > > > now I think the MessageId returned by send should also be able to be > > > > > applied for seek or acknowledge. > > > > > > > > > >> even with the > > > > > current proposal, it may return null when getting the topic from > > > > > TopicMessageId for backward compatibility. > > > > > > > > > > No. It may return null just because Java doesn't allow a non-null > > > > > returned value. The internal implementations of > > > > > TopicMessageId#getOwerTopic should return a non-null topic name to > > > > > avoid null check. > > > > > > > > > > When I mentioned **the implementation of getTopicName() must return > > > > > null**, the assumption is that MessageId#toByteArray serializes the > > > > > topic name if adding the `getTopicName()` method. However, in this > > > > > proposal, `TopicMessageId#toByteArray` won't. See the implementation > > > > > of `TopicMessageId#create`. It's only a wrapper for an arbitrary > > > > > MessageId implementation. > > > > > > > > > > Thanks, > > > > > Yunze > > > > > > > > > > On Tue, Nov 29, 2022 at 2:47 PM Zike Yang wrote: > > > > >> > > > > >> Hi Yunze, > > > > >> > > > > >> Thanks for your proposal. Quoted from your GitHub comments[0]: > > > > >> > > > > >>> There is also a case when MessageId is returned from Producer#send. > > > In this case, the returned MessageId should only used for serialization > > > > >> > > > > >> Is there a case where the user uses the messageId returned by the > > > > >> producer to seek in the consumer? Is this a good behavior? > > > > >> > > > > >>> If we added the method directly to MessageId, to keep the backward > > > compatibility, the implementation of getTopicName() must return null, > > > which > > > is not a good design. > > > > >> > > > > >> I think it's a trade-off. If I understand correctly, even with the > > > > >> current proposal, it may return null when getting the topic from > > > > >> TopicMessageId for backward compatibility. The current > > > > >> TopicMessageIdImpl doesn't serialize the topic information. > > > > >> > > > > >> > > > > >> [0] > > > https://github.com/apache/pulsar/issues/18616#issuecomment-1328609346 > > > > >> > > > > >> Thanks, > > > > >> Zike Yang > > > > >> > > > > >> On Mon, Nov 28, 2022 at 12:
Re: [VOTE] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs
Hi Yunze, add `TopicMessageId ` will couple messageID and `topic name` together, which is very unclear for non-partition-topic. ``` void seek(String topicName, MessageId messageId) throws PulsarClientException; List> getLastTopicMessageId() throws PulsarClientException; ``` If the interface is designed in this way, it may be simpler, easier to understand, and more intuitive for users, and MessageID will not be coupled with TopicName. Thanks, Bo Yunze Xu 于2022年12月16日周五 15:31写道： > > Yeah, it's an implementation detail and I will keep the same semantics > with the latest master when I push my PR. > > Thanks, > Yunze > > On Fri, Dec 16, 2022 at 3:03 PM 丛搏 wrote: > > > > if you don't change this in PIP-229 or PIP-224, I will create a new > > PIP to handle the `BatchMessageIdImpl` and `MessageIdImpl` > > `compareTo()` method, now I have no problem with this PIP > > +1 (non-binding) > > Sorry to bother this PIP vote. > > > > Thanks, > > Bo > > > > Yunze Xu 于2022年12月16日周五 11:58写道： > > > > > > If this breaking change can pass the PMC votes, I will keep the new > > > semantics in PIP-229. Otherwise, it would not make sense to adopt the > > > new semantics in PIP-229. > > > > > > Thanks, > > > Yunze > > > > > > On Fri, Dec 16, 2022 at 11:46 AM Yunze Xu wrote: > > > > > > > > I cannot find any confusing code from the proposal itself. Could you > > > > point it out? If you are mentioning the `legacyCompare` and `compare` > > > > methods in #18890 [1], it's not related to this proposal. And I have > > > > opened PIP-229 [2] for discussion. > > > > > > > > BTW, the PIP-229 itself doesn't mention the compare logic. But I'm not > > > > going to adopt the new semantics because it's actually a breaking > > > > change, just as I've replied. You might think it's a bug, but it's a > > > > public API. Any change of the semantics in the public API is a > > > > breaking change. > > > > > > > > [1] https://github.com/apache/pulsar/pull/18890/files > > > > [2] https://lists.apache.org/thread/x52zpwlo8pxzp81oxllh5vw82kyrzgpk > > > > > > > > On Fri, Dec 16, 2022 at 11:34 AM 丛搏 wrote: > > > > > > > > > > Although unrelated, it adds a lot of confusing code. > > > > > > > > > > Thanks, > > > > > Bo > > > > > > > > > > Yunze Xu 于2022年12月16日周五 08:05写道： > > > > > > > > > > > > > > > > > This proposal is not related to the comparison logic between > > > > > > BatchMessageIdImpl and MessageIdImpl. > > > > > > > > > > > > Thanks, > > > > > > Yunze > > > > > > > > > > > > On Thu, Dec 15, 2022 at 12:58 PM 丛搏 wrote: > > > > > > > > > > > > > > -1 (non-binding) > > > > > > > sorry, I have one question about the BatchMessageId compareTo() > > > > > > > method. the discussion mail : > > > > > > > https://lists.apache.org/thread/8n3oyk2hdsskkotnj4lnlvfnndctpqbg. > > > > > > > I hope it can be this issue can be discussed clearly. > > > > > > > > > > > > > > I hope it can be this issue can be discussed clearly. I will > > > > > > > retry to > > > > > > > vote until this issue clearly : > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > Bo > > > > > > > > > > > > > > 丛搏 于2022年12月14日周三 22:56写道： > > > > > > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Bo > > > > > > > > > > > > > > > > PengHui Li 于2022年12月14日周三 19:12写道： > > > > > > > > > > > > > > > > > > +1 (binding) > > > > > > > > > > > > > > > > > > - Penghui > > > > > > > > > > > > > > > > > > On Sun, Dec 11, 2022 at 6:36 AM Enrico Olivelli > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > +1 (binding) > > > > > > > > > > > > > > > > > > > > Enrico > > > > > > > > > > > > > > > > > > > > Il Ven 9 Dic 2022, 10:41 Jiaqi Shen > > > > > > > > > > ha scritto: > > > > > > > > > > > > > > > > > > > > > +1(non-binding) > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > Jiaqi Shen > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 于2022年12月5日周一 15:23写道： > > > > > > > > > > > > > > > > > > > > > > > +1(non-binding) > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > Mattison > > > > > > > > > > > > On Dec 5, 2022, 15:09 +0800, Zike Yang > > > > > > > > > > > > , wrote: > > > > > > > > > > > > > +1(non-binding) > > > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > Zike Yang > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Dec 5, 2022 at 2:41 PM Baodi Shi > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > +1(non-binding) > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Baodi Shi > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2022年12月5日 12:51，Yunze Xu > > > > > > > > > > > > > > > > 写道： > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'm starting the vote for PIP-224: Introduce > > > > > > > > >
