Re: [VOTE] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs
+1 (binding) Thanks, Hang Yunze Xu 于2022年12月28日周三 11:53写道： > > After discussing with @hangc0276 offline, I decided to add a > `Schema` implementation in this proposal to serialize > both the owner topic and the base `MessageId`. You can see the latest > update on GitHub. If any of you have any concern, feel free to let me > know. > > Thanks, > Yunze > > On Mon, Dec 26, 2022 at 1:22 AM Enrico Olivelli wrote: > > > > +1 (binding) > > > > Enrico > > > > Il Ven 23 Dic 2022, 05:46 Yunze Xu ha > > scritto: > > > > > It needs the 3rd binding +1 yet. Could anyone else take a look? > > > > > > Thanks, > > > Yunze > > > > > > On Wed, Dec 21, 2022 at 3:21 PM 丛搏 wrote: > > > > > > > > Hi Yunze, > > > > > > > > add `TopicMessageId ` will couple messageID and `topic name` together, > > > > which is very unclear for non-partition-topic. > > > > > > > > ``` > > > > void seek(String topicName, MessageId messageId) throws > > > PulsarClientException; > > > > List> getLastTopicMessageId() throws > > > > PulsarClientException; > > > > ``` > > > > If the interface is designed in this way, it may be simpler, easier to > > > > understand, and more intuitive for users, and MessageID will not be > > > > coupled with TopicName. > > > > > > > > Thanks, > > > > Bo > > > > > > > > Yunze Xu 于2022年12月16日周五 15:31写道： > > > > > > > > > > Yeah, it's an implementation detail and I will keep the same semantics > > > > > with the latest master when I push my PR. > > > > > > > > > > Thanks, > > > > > Yunze > > > > > > > > > > On Fri, Dec 16, 2022 at 3:03 PM 丛搏 wrote: > > > > > > > > > > > > if you don't change this in PIP-229 or PIP-224, I will create a new > > > > > > PIP to handle the `BatchMessageIdImpl` and `MessageIdImpl` > > > > > > `compareTo()` method, now I have no problem with this PIP > > > > > > +1 (non-binding) > > > > > > Sorry to bother this PIP vote. > > > > > > > > > > > > Thanks, > > > > > > Bo > > > > > > > > > > > > Yunze Xu 于2022年12月16日周五 11:58写道： > > > > > > > > > > > > > > If this breaking change can pass the PMC votes, I will keep the > > > > > > > new > > > > > > > semantics in PIP-229. Otherwise, it would not make sense to adopt > > > the > > > > > > > new semantics in PIP-229. > > > > > > > > > > > > > > Thanks, > > > > > > > Yunze > > > > > > > > > > > > > > On Fri, Dec 16, 2022 at 11:46 AM Yunze Xu > > > wrote: > > > > > > > > > > > > > > > > I cannot find any confusing code from the proposal itself. Could > > > you > > > > > > > > point it out? If you are mentioning the `legacyCompare` and > > > `compare` > > > > > > > > methods in #18890 [1], it's not related to this proposal. And I > > > have > > > > > > > > opened PIP-229 [2] for discussion. > > > > > > > > > > > > > > > > BTW, the PIP-229 itself doesn't mention the compare logic. But > > > I'm not > > > > > > > > going to adopt the new semantics because it's actually a > > > > > > > > breaking > > > > > > > > change, just as I've replied. You might think it's a bug, but > > > it's a > > > > > > > > public API. Any change of the semantics in the public API is a > > > > > > > > breaking change. > > > > > > > > > > > > > > > > [1] https://github.com/apache/pulsar/pull/18890/files > > > > > > > > [2] > > > https://lists.apache.org/thread/x52zpwlo8pxzp81oxllh5vw82kyrzgpk > > > > > > > > > > > > > > > > On Fri, Dec 16, 2022 at 11:34 AM 丛搏 > > > wrote: > > > > > > > > > > > > > > > > > > Although unrelated, it adds a lot of confusing code. > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Bo > > > > > > > > > > > > > > > > > > Yunze Xu 于2022年12月16日周五 > > > 08:05写道： > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This proposal is not related to the comparison logic between > > > > > > > > > > BatchMessageIdImpl and MessageIdImpl. > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Yunze > > > > > > > > > > > > > > > > > > > > On Thu, Dec 15, 2022 at 12:58 PM 丛搏 > > > wrote: > > > > > > > > > > > > > > > > > > > > > > -1 (non-binding) > > > > > > > > > > > sorry, I have one question about the BatchMessageId > > > compareTo() > > > > > > > > > > > method. the discussion mail : > > > > > > > > > > > > > > https://lists.apache.org/thread/8n3oyk2hdsskkotnj4lnlvfnndctpqbg. > > > > > > > > > > > I hope it can be this issue can be discussed clearly. > > > > > > > > > > > > > > > > > > > > > > I hope it can be this issue can be discussed clearly. I > > > will retry to > > > > > > > > > > > vote until this issue clearly : > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > Bo > > > > > > > > > > > > > > > > > > > > > > 丛搏 于2022年12月14日周三 22:56写道： > > > > > > > > > > > > > > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Bo > > > > > > > > > > > > > > > > > > > > > > > > PengHui Li 于2022年12月14日周三 19:12写道： > > > > > > > > > > > > > > > > > > > > > > > > > > +1 (binding) > > > > > > > > > > > > > > > > > > > > > > >
Re: [VOTE] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs
Schema doesn't make much sense to me. Schema is a Pulsar Client API to deal with Message serialisation, it is not a general purpose serde framework. I still approve the PIP overall but I don't think we should expose such things to the user facing APIs Il giorno mer 28 dic 2022 alle ore 09:15 Hang Chen ha scritto: > > +1 (binding) > > Thanks, > Hang > > Yunze Xu 于2022年12月28日周三 11:53写道： > > > > After discussing with @hangc0276 offline, I decided to add a > > `Schema` implementation in this proposal to serialize > > both the owner topic and the base `MessageId`. You can see the latest > > update on GitHub. If any of you have any concern, feel free to let me > > know. > > > > Thanks, > > Yunze > > > > On Mon, Dec 26, 2022 at 1:22 AM Enrico Olivelli wrote: > > > > > > +1 (binding) > > > > > > Enrico > > > > > > Il Ven 23 Dic 2022, 05:46 Yunze Xu ha > > > scritto: > > > > > > > It needs the 3rd binding +1 yet. Could anyone else take a look? > > > > > > > > Thanks, > > > > Yunze > > > > > > > > On Wed, Dec 21, 2022 at 3:21 PM 丛搏 wrote: > > > > > > > > > > Hi Yunze, > > > > > > > > > > add `TopicMessageId ` will couple messageID and `topic name` together, > > > > > which is very unclear for non-partition-topic. > > > > > > > > > > ``` > > > > > void seek(String topicName, MessageId messageId) throws > > > > PulsarClientException; > > > > > List> getLastTopicMessageId() throws > > > > > PulsarClientException; > > > > > ``` > > > > > If the interface is designed in this way, it may be simpler, easier to > > > > > understand, and more intuitive for users, and MessageID will not be > > > > > coupled with TopicName. > > > > > > > > > > Thanks, > > > > > Bo > > > > > > > > > > Yunze Xu 于2022年12月16日周五 15:31写道： > > > > > > > > > > > > Yeah, it's an implementation detail and I will keep the same > > > > > > semantics > > > > > > with the latest master when I push my PR. > > > > > > > > > > > > Thanks, > > > > > > Yunze > > > > > > > > > > > > On Fri, Dec 16, 2022 at 3:03 PM 丛搏 wrote: > > > > > > > > > > > > > > if you don't change this in PIP-229 or PIP-224, I will create a > > > > > > > new > > > > > > > PIP to handle the `BatchMessageIdImpl` and `MessageIdImpl` > > > > > > > `compareTo()` method, now I have no problem with this PIP > > > > > > > +1 (non-binding) > > > > > > > Sorry to bother this PIP vote. > > > > > > > > > > > > > > Thanks, > > > > > > > Bo > > > > > > > > > > > > > > Yunze Xu 于2022年12月16日周五 11:58写道： > > > > > > > > > > > > > > > > If this breaking change can pass the PMC votes, I will keep the > > > > > > > > new > > > > > > > > semantics in PIP-229. Otherwise, it would not make sense to > > > > > > > > adopt > > > > the > > > > > > > > new semantics in PIP-229. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Yunze > > > > > > > > > > > > > > > > On Fri, Dec 16, 2022 at 11:46 AM Yunze Xu > > > > wrote: > > > > > > > > > > > > > > > > > > I cannot find any confusing code from the proposal itself. > > > > > > > > > Could > > > > you > > > > > > > > > point it out? If you are mentioning the `legacyCompare` and > > > > `compare` > > > > > > > > > methods in #18890 [1], it's not related to this proposal. And > > > > > > > > > I > > > > have > > > > > > > > > opened PIP-229 [2] for discussion. > > > > > > > > > > > > > > > > > > BTW, the PIP-229 itself doesn't mention the compare logic. But > > > > I'm not > > > > > > > > > going to adopt the new semantics because it's actually a > > > > > > > > > breaking > > > > > > > > > change, just as I've replied. You might think it's a bug, but > > > > it's a > > > > > > > > > public API. Any change of the semantics in the public API is a > > > > > > > > > breaking change. > > > > > > > > > > > > > > > > > > [1] https://github.com/apache/pulsar/pull/18890/files > > > > > > > > > [2] > > > > https://lists.apache.org/thread/x52zpwlo8pxzp81oxllh5vw82kyrzgpk > > > > > > > > > > > > > > > > > > On Fri, Dec 16, 2022 at 11:34 AM 丛搏 > > > > wrote: > > > > > > > > > > > > > > > > > > > > Although unrelated, it adds a lot of confusing code. > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Bo > > > > > > > > > > > > > > > > > > > > Yunze Xu 于2022年12月16日周五 > > > > 08:05写道： > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This proposal is not related to the comparison logic > > > > > > > > > > > between > > > > > > > > > > > BatchMessageIdImpl and MessageIdImpl. > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > Yunze > > > > > > > > > > > > > > > > > > > > > > On Thu, Dec 15, 2022 at 12:58 PM 丛搏 > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > -1 (non-binding) > > > > > > > > > > > > sorry, I have one question about the BatchMessageId > > > > compareTo() > > > > > > > > > > > > method. the discussion mail : > > > > > > > > > > > > > > > > https://lists.apache.org/thread/8n3oyk2hdsskkotnj4lnlvfnndctpqbg. > > > > > > > > > > > > I hope it can be this issue can be discussed clearly.
Re: [DISCUSS] Reject create non-existent persistent partitions.
I agree with you. Please note that the new test case is about non-persistent topics is it expected ? Enrico Il giorno mer 28 dic 2022 alle ore 07:58 Yubiao Feng ha scritto: > > Hi qiang > > I think this is a necessary fix, and it would be nice if more explicit > errors were given to the client. > > Thanks > Yubiao > > On Wed, Dec 28, 2022 at 12:43 PM wrote: > > > Hi, All > > > > I'd like to start a discussion of this behaviour change as follow. > > > > The issue is described here: > > https://github.com/apache/pulsar/issues/19085 > > And the fix PR here: https://github.com/apache/pulsar/pull/19086 > > > > --- > > > > Behaviour change: > > > > Before: we can create non-existent persistent partitions. > > > > After: we will get `PulsarClientException.TopicDoesNotExistException` when > > we create non-existent persistent partitions. > > > > Please feel free to leave comments if you have any concerns. > > > > Best, > > Mattison > >
Re: [DISCUSS] Reject create non-existent persistent partitions.
Hi Mattison, I'm not sure if this is the current behavior, I left a comment in the PR Thanks, Bo Enrico Olivelli 于2022年12月28日周三 16:27写道： > > I agree with you. > > Please note that the new test case is about non-persistent topics > > is it expected ? > > Enrico > > Il giorno mer 28 dic 2022 alle ore 07:58 Yubiao Feng > ha scritto: > > > > Hi qiang > > > > I think this is a necessary fix, and it would be nice if more explicit > > errors were given to the client. > > > > Thanks > > Yubiao > > > > On Wed, Dec 28, 2022 at 12:43 PM wrote: > > > > > Hi, All > > > > > > I'd like to start a discussion of this behaviour change as follow. > > > > > > The issue is described here: > > > https://github.com/apache/pulsar/issues/19085 > > > And the fix PR here: https://github.com/apache/pulsar/pull/19086 > > > > > > --- > > > > > > Behaviour change: > > > > > > Before: we can create non-existent persistent partitions. > > > > > > After: we will get `PulsarClientException.TopicDoesNotExistException` when > > > we create non-existent persistent partitions. > > > > > > Please feel free to leave comments if you have any concerns. > > > > > > Best, > > > Mattison > > >
Re: [VOTE] Pulsar Release 2.9.4 Candidate 3
+1 (binding) Verified - Checksum and signatures - Build from source with JDK8 and maven 3.8.6 - Checked BookKeeper so lib - Start standalone cluster and run pulsar perf produce and consume - Run pulsar-lakehouse-connectors based on this release Thanks, Hang Haiting Jiang 于2022年12月25日周日 16:43写道： > > +1 binding > > - Checksum and signatures > - Built from sources with JDK11 > - Run Pulsar standalone > - Validate Pub/Sub and Java Functions > - Validate Connectors > - Validate Stateful Functions > - Run a simple performance check > > Thanks, > Haiting > > On Sun, Dec 25, 2022 at 3:25 PM Haiting Jiang wrote: > > > > Hi Tison and Enrico, > > > > Thanks for your information. > > > > Hi Bo, > > > > > I look at some discussions, https://github.com/apache/pulsar/issues/12166. > > > Maybe we need to upgrade the version of the bookkeeper, I am not sure > > > whether we should upgrade the bookkeeper in branch-2.9. > > > > From the discussion in the issue, I think we should not add support > > for Apple M1 in old branches. > > We don't do major version upgrades of BK in pulsar minor versions. > > > > And I will continue to verify this version on the Intel chips based laptops. > > > > Thanks, > > Haiting > > > > On Sat, Dec 24, 2022 at 8:35 PM Enrico Olivelli wrote: > > > > > > Il Sab 24 Dic 2022, 11:44 丛搏 ha scritto: > > > > > > > thanks for the information > > > > > > > > Thanks, > > > > Bo > > > > > > > > tison 于2022年12月24日周六 18:25写道： > > > > > > > > > > Well it's easy to find: > > > > > https://github.com/apache/pulsar/issues/12166#issuecomment-1237601981 > > > > > > > > > > tison 于2022年12月24日 周六18:22写道： > > > > > > > > > > > Hi Haiting, > > > > > > > > > > > > I think it's 2.11. You can search the issue on this error message. I > > > > > > remember I refer to such one previously. > > > > > > > > > > > > Sorry I'm outing so cannot do the search for you. > > > > > > > > > > > > Haiting Jiang 于2022年12月24日 周六17:20写道： > > > > > > > > > > > >> Hi Bo, > > > > > >> > > > > > >> I started standalone failed with the following errors on M1 mac, > > > > > >> ``` > > > > > >> 2022-12-24T17:08:58,944+0800 [main] ERROR > > > > > >> org.apache.pulsar.PulsarStandaloneStarter - Failed to start pulsar > > > > > >> service. > > > > > >> java.io.IOException: Failed to load RocksDB JNI library > > > > > >> at > > > > > >> > > > > org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.(KeyValueStorageRocksDB.java:98) > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > >> at > > > > > >> > > > > org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.(KeyValueStorageRocksDB.java:89) > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > >> at > > > > > >> > > > > org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.lambda$static$0(KeyValueStorageRocksDB.java:63) > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > >> at > > > > > >> > > > > org.apache.bookkeeper.bookie.storage.ldb.LedgerMetadataIndex.(LedgerMetadataIndex.java:68) > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > >> at > > > > > >> > > > > org.apache.bookkeeper.bookie.storage.ldb.SingleDirectoryDbLedgerStorage.(SingleDirectoryDbLedgerStorage.java:170) > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > >> at > > > > > >> > > > > org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage.newSingleDirectoryDbLedgerStorage(DbLedgerStorage.java:150) > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > >> at > > > > > >> > > > > org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage.initialize(DbLedgerStorage.java:129) > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > >> at org.apache.bookkeeper.bookie.Bookie.(Bookie.java:818) > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > >> at > > > > > >> > > > > org.apache.bookkeeper.proto.BookieServer.newBookie(BookieServer.java:152) > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > >> at > > > > org.apache.bookkeeper.proto.BookieServer.(BookieServer.java:120) > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > >> at > > > > > >> > > > > org.apache.pulsar.zookeeper.LocalBookkeeperEnsemble.runBookies(LocalBookkeeperEnsemble.java:317) > > > > > >> ~[org.apache.pulsar-pulsar-zookeeper-utils-2.9.4.jar:2.9.4] > > > > > >> at > > > > > >> > > > > org.apache.pulsar.zookeeper.LocalBookkeeperEnsemble.startStandalone(LocalBookkeeperEnsemble.java:445) > > > > > >> ~[org.apache.pulsar-pulsar-zookeeper-utils-2.9.4.jar:2.9.4] > > > > > >> at > > > > > >> org.apache.pulsar.PulsarStandalone.start(PulsarStandalone.java:269) > > > > > >> ~[org.apache.pulsar-pulsar-broker-2.9.4.jar:2.9.4] > > > > > >> at > > > > > >> > > > > org.apache.pulsar.PulsarStandaloneStarter.main(PulsarStandaloneStarter.jav
Re: [VOTE] Pulsar Release 2.9.4 Candidate 3
Thank you all, Close the vote with 3 bindings(PengHui, Haiting, Hang), and 4 non-bindings(Xiangying, Yunze). I will continue the release process. Thanks, Bo Hang Chen 于2022年12月28日周三 18:29写道： > > +1 (binding) > > Verified > - Checksum and signatures > - Build from source with JDK8 and maven 3.8.6 > - Checked BookKeeper so lib > - Start standalone cluster and run pulsar perf produce and consume > - Run pulsar-lakehouse-connectors based on this release > > Thanks, > Hang > > Haiting Jiang 于2022年12月25日周日 16:43写道： > > > > +1 binding > > > > - Checksum and signatures > > - Built from sources with JDK11 > > - Run Pulsar standalone > > - Validate Pub/Sub and Java Functions > > - Validate Connectors > > - Validate Stateful Functions > > - Run a simple performance check > > > > Thanks, > > Haiting > > > > On Sun, Dec 25, 2022 at 3:25 PM Haiting Jiang > > wrote: > > > > > > Hi Tison and Enrico, > > > > > > Thanks for your information. > > > > > > Hi Bo, > > > > > > > I look at some discussions, > > > > https://github.com/apache/pulsar/issues/12166. > > > > Maybe we need to upgrade the version of the bookkeeper, I am not sure > > > > whether we should upgrade the bookkeeper in branch-2.9. > > > > > > From the discussion in the issue, I think we should not add support > > > for Apple M1 in old branches. > > > We don't do major version upgrades of BK in pulsar minor versions. > > > > > > And I will continue to verify this version on the Intel chips based > > > laptops. > > > > > > Thanks, > > > Haiting > > > > > > On Sat, Dec 24, 2022 at 8:35 PM Enrico Olivelli > > > wrote: > > > > > > > > Il Sab 24 Dic 2022, 11:44 丛搏 ha scritto: > > > > > > > > > thanks for the information > > > > > > > > > > Thanks, > > > > > Bo > > > > > > > > > > tison 于2022年12月24日周六 18:25写道： > > > > > > > > > > > > Well it's easy to find: > > > > > > https://github.com/apache/pulsar/issues/12166#issuecomment-1237601981 > > > > > > > > > > > > tison 于2022年12月24日 周六18:22写道： > > > > > > > > > > > > > Hi Haiting, > > > > > > > > > > > > > > I think it's 2.11. You can search the issue on this error > > > > > > > message. I > > > > > > > remember I refer to such one previously. > > > > > > > > > > > > > > Sorry I'm outing so cannot do the search for you. > > > > > > > > > > > > > > Haiting Jiang 于2022年12月24日 周六17:20写道： > > > > > > > > > > > > > >> Hi Bo, > > > > > > >> > > > > > > >> I started standalone failed with the following errors on M1 mac, > > > > > > >> ``` > > > > > > >> 2022-12-24T17:08:58,944+0800 [main] ERROR > > > > > > >> org.apache.pulsar.PulsarStandaloneStarter - Failed to start > > > > > > >> pulsar > > > > > > >> service. > > > > > > >> java.io.IOException: Failed to load RocksDB JNI library > > > > > > >> at > > > > > > >> > > > > > org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.(KeyValueStorageRocksDB.java:98) > > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > > >> at > > > > > > >> > > > > > org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.(KeyValueStorageRocksDB.java:89) > > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > > >> at > > > > > > >> > > > > > org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.lambda$static$0(KeyValueStorageRocksDB.java:63) > > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > > >> at > > > > > > >> > > > > > org.apache.bookkeeper.bookie.storage.ldb.LedgerMetadataIndex.(LedgerMetadataIndex.java:68) > > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > > >> at > > > > > > >> > > > > > org.apache.bookkeeper.bookie.storage.ldb.SingleDirectoryDbLedgerStorage.(SingleDirectoryDbLedgerStorage.java:170) > > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > > >> at > > > > > > >> > > > > > org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage.newSingleDirectoryDbLedgerStorage(DbLedgerStorage.java:150) > > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > > >> at > > > > > > >> > > > > > org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage.initialize(DbLedgerStorage.java:129) > > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > > >> at org.apache.bookkeeper.bookie.Bookie.(Bookie.java:818) > > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > > >> at > > > > > > >> > > > > > org.apache.bookkeeper.proto.BookieServer.newBookie(BookieServer.java:152) > > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > > >> at > > > > > org.apache.bookkeeper.proto.BookieServer.(BookieServer.java:120) > > > > > > >> ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5] > > > > > > >> at > > > > > > >> > > > > > org.apache.pulsar.zookeeper.LocalBookkeeperEnsemble.runBookies(LocalBookkeeperEnsemble.java:317) > > > > > > >> ~[org.apache.pulsar-pulsar-zookeeper-utils-2.
Re: [VOTE] Pulsar Node.js Client Release 1.8.0 Candidate 2
+1 (binding) Verify following the steps from https://github.com/RobertIndie/pulsar-client-node-validation - Start standalone (master branch) - Start consumer (node ./examples/consumer.js) - Start producer (node ./examples/producer.js) - The consumer can receive the messages Regards, Penghui On Wed, Dec 28, 2022 at 1:45 PM Ran Gao wrote: > +1 (non-binding) > > - Checked npm package > https://www.npmjs.com/package/pulsar-client/v/1.8.0-rc.2 > - Checked source code + napi-darwin-unknown-arm64 work on macOS 12.6 (M1 > Pro chip) > > Thanks, > Ran Gao > > On 2022/12/27 08:29:35 Zike Yang wrote: > > Hi everyone, > > > > This is the second release candidate for the Apache Pulsar Node.js > > client, version 1.8.0. > > > > It fixes the following issues: > > https://github.com/apache/pulsar-client-node/milestone/13?closed=1 > > > > Please download the source files and review this release candidate: > > - Download the source package, verify shasum and asc > > - Follow the README.md to build and run the Pulsar Node.js client. > > > > The rc package has been published to the npm registry: > > https://www.npmjs.com/package/pulsar-client/v/1.8.0-rc.2 > > You can install it by `npm i pulsar-client@1.8.0-rc.2` and verify the > package. > > > > The vote will be open for at least 72 hours. It is adopted by majority > > approval, with at least 3 PMC affirmative votes. > > > > Source files: > > > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-client-node/pulsar-client-node-1.8.0-candidate-2/ > > > > Pulsar's KEYS file containing PGP keys we use to sign the release: > > https://dist.apache.org/repos/dist/dev/pulsar/KEYS > > > > SHA-512 checksum: > > > 80f9de8ca73b9a0bc7d083c6ba92e86721d381aa0689858270e074ff505f887533c40c5625b85a8b0943ced352e93eaafe67eb9bcb5afefadb4bd29b594620c4 > > pulsar-client-node-1.8.0.tar.gz > > > > The tag to be voted upon: > > v1.8.0-rc.2 > > https://github.com/apache/pulsar-client-node/releases/tag/v1.8.0-rc.2 > > > > Please review and vote on the release candidate #2 for the version > > 1.8.0, as follows: > > [ ] +1, Approve the release > > [ ] -1, Do not approve the release (please provide specific comments) > > > > Thanks, > > Zike Yang > > >
Re: [DISCUSS] Reject create non-existent persistent partitions.
Hi, Enrico > Please note that the new test case is about non-persistent topics I'm sorry, It's my mistake, the non-persistent already fix this problem, I need to change the test topic name to persistent. Best, Mattison On Dec 28, 2022, 16:27 +0800, Enrico Olivelli , wrote: > I agree with you. > > Please note that the new test case is about non-persistent topics > > is it expected ? > > Enrico > > Il giorno mer 28 dic 2022 alle ore 07:58 Yubiao Feng > ha scritto: > > > > Hi qiang > > > > I think this is a necessary fix, and it would be nice if more explicit > > errors were given to the client. > > > > Thanks > > Yubiao > > > > On Wed, Dec 28, 2022 at 12:43 PM wrote: > > > > > > Hi, All > > > > > > > > I'd like to start a discussion of this behaviour change as follow. > > > > > > > > The issue is described here: > > > > https://github.com/apache/pulsar/issues/19085 > > > > And the fix PR here: https://github.com/apache/pulsar/pull/19086 > > > > > > > > --- > > > > > > > > Behaviour change: > > > > > > > > Before: we can create non-existent persistent partitions. > > > > > > > > After: we will get `PulsarClientException.TopicDoesNotExistException` > > > > when > > > > we create non-existent persistent partitions. > > > > > > > > Please feel free to leave comments if you have any concerns. > > > > > > > > Best, > > > > Mattison > > > >
Re: [VOTE] Pulsar Release 2.10.3 Candidate 1
+1 (binding) Verified - Checksum and signatures - Build from source with JDK17 and maven 3.8.6 - Checked BookKeeper so lib - Start standalone cluster and run pulsar perf produce and consume - Run pulsar-lakehouse-connectors based on this release Thanks, Hang guo jiwei 于2022年12月28日周三 11:21写道： > > +1 (binding) > > maven: 3.8.6 > JDK: 17.0.3.1 > OS: 12.6 > > - Checked the signature > - Build from the source package > - Checked LICENSE > - Start standalone > - Publish and consume messages > - Verified Function and State Function > - Verified Cassandra connector > > Regards > Jiwei Guo (Tboy) > > > On Wed, Dec 28, 2022 at 9:16 AM PengHui Li wrote: > > > +1 (binding) > > > > OS: macOS 13.0.1 (22A400) > > maven: 3.8.6 > > JDK: java version "11.0.7" 2020-04-14 LTS > > > > - Checked the signature > > - Build from the source package > > - Checked LICENSE > > - Start standalone > > - Publish and consume messages > > - Verified Function and State Function > > - Verified Cassandra connector > > > > Thanks, > > Penghui > > > > On Tue, Dec 27, 2022 at 10:46 PM 丛搏 wrote: > > > > > +1 (non-binding) > > > > > > system: mac os 12.3.1, Intel > > > maven: 3.6.1 > > > java: OpenJDK 17.0.1 > > > > > > - Checked the signature > > > - Checked LICENSE > > > - Start standalone > > > - Publish and consume messages > > > - Verified Function and State Function > > > - Verified Cassandra connector > > > - Build from the source package > > > - Run a simple transaction performance check > > > > > > Thanks, > > > Bo > > > > > > Xiangying Meng 于2022年12月25日周日 20:42写道： > > > > > > > > This is the third release candidate for Apache Pulsar, version 2.10.3. > > > > > > > > This release contains 155 commits by 50 contributors. > > > > https://github.com/apache/pulsar/compare/v2.10.2...v2.10.3-candidate-1 > > > > > > > > *** Please download, test and vote on this release. This vote will stay > > > open > > > > for at least 72 hours *** > > > > > > > > Note that we are voting upon the source (tag), binaries are provided > > for > > > > convenience. > > > > > > > > Source and binary files: > > > > > > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.10.3-candidate-1/ > > > > > > > > SHA-512 checksums: > > > > > > > > > 64518096acf4c2a5ef1dcc936cd645217291254cd5c18337a743db5b4caa70a48cfc969643fd18a16ba24421952155b597e1b84be997447fe21f0b12a0555cb1 > > > > apache-pulsar-2.10.3-bin.tar.gz > > > > > > > > > ee542d64d4aa288200c06f42d71186e8797480263ab84aaeb50ac683d6ea675c298adf8207b3aa98dae378b9fc84e9ba3dc78902397a774d1756d5e1739ab475 > > > > apache-pulsar-2.10.3-src.tar.gz > > > > > > > > Maven staging repo: > > > > > > https://repository.apache.org/content/repositories/orgapachepulsar-1201/ > > > > > > > > The tag to be voted upon: > > > > v2.10.3-candidate-1 (b69f4efa6058c3f51885a61a2b3acb46f8b730f4) > > > > https://github.com/apache/pulsar/releases/tag/v2.10.3-candidate-1 > > > > > > > > Pulsar's KEYS file containing PGP keys you use to sign the release: > > > > https://dist.apache.org/repos/dist/dev/pulsar/KEYS > > > > > > > > Docker images: > > > > > > > > > > > > > > > > > https://hub.docker.com/layers/xiangyingmeng/pulsar/2.10.3/images/sha256-9034eca8a61c7edc6d5b3fb5aa214f2dbb45f89d00c4ac875931ca588645dc96?context=repo > > > > > > > > > > > > > > > > > https://hub.docker.com/layers/xiangyingmeng/pulsar-all/2.10.3/images/sha256-c7a0323bf59f20ae29b362815302e272109453b8210f130a1daaa7b38918f884?context=repo > > > > > > > > Please download the source package, and follow the README to build > > > > and run the Pulsar standalone service. > > > > >
Re: [VOTE] Pulsar Release 2.11.0 Candidate-4
-1 (binding) When checking the `apache-pulsar-shell-2.11.0-shell.tar.gz` license, I found a lot of third party jars are not included in the LICENSE file. Thanks, Hang Haiting Jiang 于2022年12月27日周二 20:26写道： > > +1 (binding) > > - Checksum and signatures > > - Built from sources using JDK 17 and maven 3.8.6 > > - Run rat check and check-binary-license on the source. > > - Run Pulsar standalone > > - Validate Pub/Sub and Java Functions > > - Validate Connectors > > - Validate Stateful Functions with `PULSAR_STANDALONE_USE_ZOOKEEPER=1` > > - Run simple performance check > > Thanks, > Haiting > > On Tue, Dec 27, 2022 at 6:53 PM Enrico Olivelli wrote: > > > > +1 (binding) > > > > - verified checksums and signatures > > - built sources > > - run some smoke tests using the binaries built from source and using > > the staged binaries > > > > great work > > > > Enrico > > > > Il giorno mar 27 dic 2022 alle ore 10:14 丛搏 ha scritto: > > > > > > +1 (non-binding) > > > > > > system: mac os 12.6, Apple M1 > > > maven: 3.8.5 > > > java: OpenJDK 17.0.3 > > > > > > - Checked the signature > > > - Checked LICENSE > > > - Start standalone with zookeeper stream storage > > > - Publish and consume messages > > > - Verified Function and State Function > > > - Verified Cassandra connector > > > - Build from the source package > > > - Run a simple transaction performance check > > > > > > Thanks, > > > Bo
Re: [DISCUSS] PIP-234: Support using shared thread pool across multiple Pulsar client instance
Hmmm
I found that if we want to provide the public API(shared event loop) in the
pulsar-client-api module.
We need to add Netty dependency to the pulsar-client-api module.
It's not a good idea to have Netty dependency in the pulsar-client-api
module.
Maybe we can add the new API to the pulsar-client-original module? like we
currently do.
```
PulsarClientImpl.builder()...
```
We only changed the default value of the thread pool. Users still have the
ability to change the threads
that they want to assign. So if they want to use multiple client instances,
they can also change the threads
options.
Thanks,
Penghui
On Tue, Dec 27, 2022 at 8:35 PM PengHui Li wrote:
> Ah, got it.
>
> It's a good idea for users to be aware of all shared resources.
> I will change the proposal as per your suggestion
>
> Thanks,
> Penghui
>
> On Tue, Dec 27, 2022 at 7:11 PM Enrico Olivelli
> wrote:
>
>> I generally support this proposal,
>> this is a problem we have in the Proxy and I have seen it on
>> applications that need to connect to multiple different tenants
>> and they need different authentication parameters, so they have to
>> create many PulsarClient instances.
>>
>> I have a suggestion:
>>
>> Exposing all the internals is a good idea for very advanced users,
>> but I believe that we should provide some simpler support.
>>
>> We should have an API to allow sharing resources among multiple
>> clients without entering the details.
>>
>> Interface PulsarClientGroup {
>>... put here all the sharable things in the current version...
>> }
>>
>> PulsarClient client = newClient().
>> .withSharedResources(pulsarClientGroup)
>> ...
>>
>>
>> I think that having a PulsarClientGroup is a good choice for future
>> evolutions
>> because the internal thread pools may change: removed/added/change the
>> purpose.
>>
>> If we require users to deal with all the possible sharable resources
>> then we have a few risks:
>> - people can "forget" to share some resources
>> - upgrading the client may lead to not taking into account some new
>> "shareable" resources
>>
>> This is way I believe that we should provide an opaque
>>
>> Enrico
>>
>> Il giorno mar 27 dic 2022 alle ore 11:07 PengHui Li
>> ha scritto:
>> >
>> > Hi all,
>> >
>> > As discussed at
>> > https://lists.apache.org/thread/5obfm17g58n3dnbzyxg57vokgmwyp6hx
>> > I have created this proposal to support shared thread pool across
>> multiple
>> > client instances
>> > Here is the proposal link https://github.com/apache/pulsar/issues/19074
>> >
>> > Please help take a look, and look forward to your suggestions.
>> >
>> > Thanks,
>> > Penghui
>> >
>> > --
>> >
>> > ### Motivation
>> >
>> > The Pulsar client mainly has three thread pools that cooperate with each
>> > other to complete the message publishing and consumption of messages.
>> >
>> > - IO threads - Used for handling network packets from the broker
>> > - Internal threads - Used for handling internal tasks such as moving the
>> > received messages to the internal receiver queue and pulling out the
>> > message from the receiver queue to return to users. And the Java client
>> is
>> > optimized by the lock-free principle; each consumer will use a pinned
>> > internal thread to reduce the lock overhead.
>> > - External threads - Used by the message listener
>> >
>> > All the above thread pools will be created automatically after a Pulsar
>> > client instance has been created.
>> >
>> > But for some cases, users need to create multiple Pulsar client
>> instances
>> > in a JVM process due to different authentications or others. Each client
>> > will have exclusive thread pools, which will cause unreasonable thread
>> > usage, waste memory, and potential performance degradation.
>> >
>> > It is not a serious problem for previous releases with the default
>> > configurations because the thread pool will only have 1 thread by
>> default.
>> > But it also doesn't make sense that we only have one thread for each
>> thread
>> > pool. We have discussed this part under this [thread](
>> > https://lists.apache.org/thread/5obfm17g58n3dnbzyxg57vokgmwyp6hx)
>> >
>> > So this proposal will provide a new possibility for users that require
>> > multiple Pulsar client instances in one JVM process to use the shared
>> > thread pools across multiple Pulsar client instances.
>> >
>> > ### Goal
>> >
>> > Provide public API to use the shared thread pool across multiple Pulsar
>> > client instances in one JVM process
>> >
>> > - IO threads
>> > - Internal threads
>> > - External threads
>> >
>> > BTW, we already have such an ability internally. It was just hidden for
>> > users. Please take a look at #12037 and #13839 to get more details.
>> >
>> > ### API Changes
>> >
>> > The following APIs will be introduced to the Java Client when creating a
>> > Client instance
>> >
>> > ```java
>> > PulsarClient.builder()
>> > .eventLoopGroup(ioEventLoopGroup)
>> > .internalExec
Re: [VOTE] PIP-224: Introduce TopicMessageId for consumer's MessageId related APIs
Hi Enrico,
Yeah, I've discussed it privately with Hang and he had the same
concern with you and. So I changed the `Schema` to a
more simple class:
```java
class TopicMessageIdSerDes {
public static byte[] serialize(TopicMessageId topicMessageId) {/* ... */}
public static TopicMessageId deserialize(byte[] bytes) {/* ... */}
}
```
You can see it and the updated Alternatives section in
https://github.com/apache/pulsar/issues/18616
Thanks,
Yunze
On Wed, Dec 28, 2022 at 4:22 PM Enrico Olivelli wrote:
>
> Schema doesn't make much sense to me.
>
> Schema is a Pulsar Client API to deal with Message serialisation, it
> is not a general purpose serde framework.
>
> I still approve the PIP overall but I don't think we should expose
> such things to the user facing APIs
>
> Il giorno mer 28 dic 2022 alle ore 09:15 Hang Chen
> ha scritto:
> >
> > +1 (binding)
> >
> > Thanks,
> > Hang
> >
> > Yunze Xu 于2022年12月28日周三 11:53写道：
> > >
> > > After discussing with @hangc0276 offline, I decided to add a
> > > `Schema` implementation in this proposal to serialize
> > > both the owner topic and the base `MessageId`. You can see the latest
> > > update on GitHub. If any of you have any concern, feel free to let me
> > > know.
> > >
> > > Thanks,
> > > Yunze
> > >
> > > On Mon, Dec 26, 2022 at 1:22 AM Enrico Olivelli
> > > wrote:
> > > >
> > > > +1 (binding)
> > > >
> > > > Enrico
> > > >
> > > > Il Ven 23 Dic 2022, 05:46 Yunze Xu ha
> > > > scritto:
> > > >
> > > > > It needs the 3rd binding +1 yet. Could anyone else take a look?
> > > > >
> > > > > Thanks,
> > > > > Yunze
> > > > >
> > > > > On Wed, Dec 21, 2022 at 3:21 PM 丛搏 wrote:
> > > > > >
> > > > > > Hi Yunze,
> > > > > >
> > > > > > add `TopicMessageId ` will couple messageID and `topic name`
> > > > > > together,
> > > > > > which is very unclear for non-partition-topic.
> > > > > >
> > > > > > ```
> > > > > > void seek(String topicName, MessageId messageId) throws
> > > > > PulsarClientException;
> > > > > > List> getLastTopicMessageId() throws
> > > > > > PulsarClientException;
> > > > > > ```
> > > > > > If the interface is designed in this way, it may be simpler, easier
> > > > > > to
> > > > > > understand, and more intuitive for users, and MessageID will not be
> > > > > > coupled with TopicName.
> > > > > >
> > > > > > Thanks,
> > > > > > Bo
> > > > > >
> > > > > > Yunze Xu 于2022年12月16日周五 15:31写道：
> > > > > > >
> > > > > > > Yeah, it's an implementation detail and I will keep the same
> > > > > > > semantics
> > > > > > > with the latest master when I push my PR.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Yunze
> > > > > > >
> > > > > > > On Fri, Dec 16, 2022 at 3:03 PM 丛搏 wrote:
> > > > > > > >
> > > > > > > > if you don't change this in PIP-229 or PIP-224, I will create a
> > > > > > > > new
> > > > > > > > PIP to handle the `BatchMessageIdImpl` and `MessageIdImpl`
> > > > > > > > `compareTo()` method, now I have no problem with this PIP
> > > > > > > > +1 (non-binding)
> > > > > > > > Sorry to bother this PIP vote.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Bo
> > > > > > > >
> > > > > > > > Yunze Xu 于2022年12月16日周五 11:58写道：
> > > > > > > > >
> > > > > > > > > If this breaking change can pass the PMC votes, I will keep
> > > > > > > > > the new
> > > > > > > > > semantics in PIP-229. Otherwise, it would not make sense to
> > > > > > > > > adopt
> > > > > the
> > > > > > > > > new semantics in PIP-229.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Yunze
> > > > > > > > >
> > > > > > > > > On Fri, Dec 16, 2022 at 11:46 AM Yunze Xu
> > > > > > > > >
> > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > I cannot find any confusing code from the proposal itself.
> > > > > > > > > > Could
> > > > > you
> > > > > > > > > > point it out? If you are mentioning the `legacyCompare` and
> > > > > `compare`
> > > > > > > > > > methods in #18890 [1], it's not related to this proposal.
> > > > > > > > > > And I
> > > > > have
> > > > > > > > > > opened PIP-229 [2] for discussion.
> > > > > > > > > >
> > > > > > > > > > BTW, the PIP-229 itself doesn't mention the compare logic.
> > > > > > > > > > But
> > > > > I'm not
> > > > > > > > > > going to adopt the new semantics because it's actually a
> > > > > > > > > > breaking
> > > > > > > > > > change, just as I've replied. You might think it's a bug,
> > > > > > > > > > but
> > > > > it's a
> > > > > > > > > > public API. Any change of the semantics in the public API
> > > > > > > > > > is a
> > > > > > > > > > breaking change.
> > > > > > > > > >
> > > > > > > > > > [1] https://github.com/apache/pulsar/pull/18890/files
> > > > > > > > > > [2]
> > > > > https://lists.apache.org/thread/x52zpwlo8pxzp81oxllh5vw82kyrzgpk
> > > > > > > > > >
> > > > > > > > > > On Fri, Dec 16, 2022 at 11:34 AM 丛搏
> > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > Although unrelated, it adds a lot of confusing code.
> > > > > > > > > > >
> > > > > > > > > > > Thank
Re: [DISCUSS] PIP-234: Support using shared thread pool across multiple Pulsar client instance
> Maybe we can add the new API to the pulsar-client-original module? like we
> currently do.
Adding stable APIs to the pulsar-client-original module that
represents the internal implementations is not a good way IMO. I'm
wondering why we need to configure a Netty EventLoop? Is there any
alternative class in the Java standard library?
Thanks,
Yunze
On Wed, Dec 28, 2022 at 10:18 PM PengHui Li wrote:
>
> Hmmm
>
> I found that if we want to provide the public API(shared event loop) in the
> pulsar-client-api module.
> We need to add Netty dependency to the pulsar-client-api module.
> It's not a good idea to have Netty dependency in the pulsar-client-api
> module.
>
> Maybe we can add the new API to the pulsar-client-original module? like we
> currently do.
>
> ```
> PulsarClientImpl.builder()...
> ```
>
> We only changed the default value of the thread pool. Users still have the
> ability to change the threads
> that they want to assign. So if they want to use multiple client instances,
> they can also change the threads
> options.
>
> Thanks,
> Penghui
>
> On Tue, Dec 27, 2022 at 8:35 PM PengHui Li wrote:
>
> > Ah, got it.
> >
> > It's a good idea for users to be aware of all shared resources.
> > I will change the proposal as per your suggestion
> >
> > Thanks,
> > Penghui
> >
> > On Tue, Dec 27, 2022 at 7:11 PM Enrico Olivelli
> > wrote:
> >
> >> I generally support this proposal,
> >> this is a problem we have in the Proxy and I have seen it on
> >> applications that need to connect to multiple different tenants
> >> and they need different authentication parameters, so they have to
> >> create many PulsarClient instances.
> >>
> >> I have a suggestion:
> >>
> >> Exposing all the internals is a good idea for very advanced users,
> >> but I believe that we should provide some simpler support.
> >>
> >> We should have an API to allow sharing resources among multiple
> >> clients without entering the details.
> >>
> >> Interface PulsarClientGroup {
> >>... put here all the sharable things in the current version...
> >> }
> >>
> >> PulsarClient client = newClient().
> >> .withSharedResources(pulsarClientGroup)
> >> ...
> >>
> >>
> >> I think that having a PulsarClientGroup is a good choice for future
> >> evolutions
> >> because the internal thread pools may change: removed/added/change the
> >> purpose.
> >>
> >> If we require users to deal with all the possible sharable resources
> >> then we have a few risks:
> >> - people can "forget" to share some resources
> >> - upgrading the client may lead to not taking into account some new
> >> "shareable" resources
> >>
> >> This is way I believe that we should provide an opaque
> >>
> >> Enrico
> >>
> >> Il giorno mar 27 dic 2022 alle ore 11:07 PengHui Li
> >> ha scritto:
> >> >
> >> > Hi all,
> >> >
> >> > As discussed at
> >> > https://lists.apache.org/thread/5obfm17g58n3dnbzyxg57vokgmwyp6hx
> >> > I have created this proposal to support shared thread pool across
> >> multiple
> >> > client instances
> >> > Here is the proposal link https://github.com/apache/pulsar/issues/19074
> >> >
> >> > Please help take a look, and look forward to your suggestions.
> >> >
> >> > Thanks,
> >> > Penghui
> >> >
> >> > --
> >> >
> >> > ### Motivation
> >> >
> >> > The Pulsar client mainly has three thread pools that cooperate with each
> >> > other to complete the message publishing and consumption of messages.
> >> >
> >> > - IO threads - Used for handling network packets from the broker
> >> > - Internal threads - Used for handling internal tasks such as moving the
> >> > received messages to the internal receiver queue and pulling out the
> >> > message from the receiver queue to return to users. And the Java client
> >> is
> >> > optimized by the lock-free principle; each consumer will use a pinned
> >> > internal thread to reduce the lock overhead.
> >> > - External threads - Used by the message listener
> >> >
> >> > All the above thread pools will be created automatically after a Pulsar
> >> > client instance has been created.
> >> >
> >> > But for some cases, users need to create multiple Pulsar client
> >> instances
> >> > in a JVM process due to different authentications or others. Each client
> >> > will have exclusive thread pools, which will cause unreasonable thread
> >> > usage, waste memory, and potential performance degradation.
> >> >
> >> > It is not a serious problem for previous releases with the default
> >> > configurations because the thread pool will only have 1 thread by
> >> default.
> >> > But it also doesn't make sense that we only have one thread for each
> >> thread
> >> > pool. We have discussed this part under this [thread](
> >> > https://lists.apache.org/thread/5obfm17g58n3dnbzyxg57vokgmwyp6hx)
> >> >
> >> > So this proposal will provide a new possibility for users that require
> >> > multiple Pulsar client instances in one JVM process to use the shared
> >> > thread poo
Re: [DISCUSS] Reject create non-existent persistent partitions.
Hi, All I have another question that needs to discuss. Should we allow the user to create the non-partitioned topic name like `persistent://tenant/namespace/localname-partition-0`? If so, this is a little confusing with the partitioned topic. e.g.: TopicName#isPartitioned method. Best, Mattison On Dec 28, 2022, 12:43 +0800, mattisonc...@gmail.com, wrote: > Hi, All > > I'd like to start a discussion of this behaviour change as follow. > > The issue is described here: https://github.com/apache/pulsar/issues/19085 > And the fix PR here: https://github.com/apache/pulsar/pull/19086 > > --- > > Behaviour change: > > Before: we can create non-existent persistent partitions. > > After: we will get `PulsarClientException.TopicDoesNotExistException` when we > create non-existent persistent partitions. > > Please feel free to leave comments if you have any concerns. > > Best, > Mattison
Re: [DISCUSS] Reject create non-existent persistent partitions.
Hi Mattison, > Should we allow the user to create the non-partitioned topic name like > `persistent://tenant/namespace/localname-partition-0`? I think we should disallow creation. This will cause the partition metadata to be incorrect. If the current behavior is to allow the creation, modifying it would be a breaking change. We include this modification in the next version, no need to cherry-pick to the old branch Thanks, Bo 于2022年12月29日周四 13:33写道： > > Hi, All > > I have another question that needs to discuss. > > Should we allow the user to create the non-partitioned topic name like > `persistent://tenant/namespace/localname-partition-0`? > > If so, this is a little confusing with the partitioned topic. > > e.g.: > TopicName#isPartitioned method. > > Best, > Mattison > On Dec 28, 2022, 12:43 +0800, mattisonc...@gmail.com, wrote: > > Hi, All > > > > I'd like to start a discussion of this behaviour change as follow. > > > > The issue is described here: https://github.com/apache/pulsar/issues/19085 > > And the fix PR here: https://github.com/apache/pulsar/pull/19086 > > > > --- > > > > Behaviour change: > > > > Before: we can create non-existent persistent partitions. > > > > After: we will get `PulsarClientException.TopicDoesNotExistException` when > > we create non-existent persistent partitions. > > > > Please feel free to leave comments if you have any concerns. > > > > Best, > > Mattison
