[GitHub] [pulsar-helm-chart] sdwerwed edited a comment on pull request #127: Antiaffinity with zone topology for high availability
sdwerwed edited a comment on pull request #127: URL: https://github.com/apache/pulsar-helm-chart/pull/127#issuecomment-859123386 Hello @sijie, can you have a look at this PR, let me know if any other action is required from my side. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [apache/pulsar] [website] website upgrade (#11766)
This is a question for the dev@pulsar list and not a blocker for infrastructure. I don’t think there is an overriding requirement to preserve docs from incubating releases. If there were then there are other ways to find and regenerate these docs. Regards, Dave Sent from my iPhone > On Dec 19, 2021, at 6:58 PM, Anonymitaet wrote: > > > Hi @dave2wave Thanks for your answer. However, > https://archive.apache.org/dist/incubator/pulsar/ hosts code files rather > than doc files. We are looking for doc files (source files) of these > versions, do you know where hosts them? Thanks again > > 2.0.1-incubating > 2.0.0-rc1-incubating > 1.22.0-incubating > 1.21.0-incubating > 1.20.0-incubating > 1.19.0-incubating > — > Reply to this email directly, view it on GitHub, or unsubscribe. > Triage notifications on the go with GitHub Mobile for iOS or Android. > You are receiving this because you were mentioned.
Re: [DISCUSSION] PIP-120: Enable client memory limit by default
+1
On 12/17/2021 14:43，mattison chao wrote：
+1
On Fri, 17 Dec 2021 at 13:56, 陳智弘 wrote:
+1
Sijie Guo 於 2021年12月17日 週五 12:38 寫道：
+1
On Tue, Dec 14, 2021 at 11:20 AM Matteo Merli wrote:
https://github.com/apache/pulsar/issues/13306
Pasted below for quoting convenience.
## Motivation
In Pulsar 2.8, we have introduced a setting to control the amount of
memory
used by a client instance.
```java
interface ClientBuilder {
ClientBuilder memoryLimit(long memoryLimit, SizeUnit unit);
}
```
By default, in 2.8 and 2.9 this setting is set to 0, meaning no limit
is
being
enforced.
I think it's a good time for 2.10 to enable this setting by default
and,
correspondingly, to disable by default the producer queue size limit.
This will simplify a lot the configuration that a producer application
will
have to come up with, when publishing with many topic/partitions or
when messages
are bigger than expected.
## Proposed changes
In 2.10 release, for the `ClientBuilder`, change
* `memoryLimit`: 0 -> 64 MB
For the `ProducerBuilder`, changes
* `maxPendingMessages`: 1000 -> 0
64MB is picked because it's a small enough memory size that will
guarantee
a very high producer throughput, irrespective of the individual
messages
size.
--
Matteo Merli
Re: [DISCUSSION] PIP-117: Change Pulsar standalone defaults
My two cents +1 , In my case, i sometimes need to run pulsar standalone in some server with limited resources. > Proposed changes The other point i can think of is maybe schema storage also need to change from using BK to using local filesystem or some else. Best Regards, Lan Liang On 12/17/2021 12:35，Sijie Guo wrote： +1 On Tue, Dec 14, 2021 at 9:18 AM Matteo Merli wrote: https://github.com/apache/pulsar/issues/13302 Copying here for quoting convenience ## Motivation Pulsar standalone is the "Pulsar in a box" version of a Pulsar cluster, where all the components are started within the context of a single JVM process. Users are using the standalone as a way to get quickly started with Pulsar or in all the cases where it makes sense to have a single node deployment. Right now, the standalone is starting by default with many components, several of which are quite complex, since they are designed to be deployed in a distributed fashion. ## Goal Simplify the components of Pulsar standalone to achieve: 1. Reduce complexity 2. Reduce startup time 3. Reduce memory and CPU footprint of running standalone ## Proposed changes The proposal here is to change some of the default implementations that are used for the Pulsar standalone. 1. **Metadata Store implementation** --> Change from ZooKeeper to RocksDB 2. **Pulsar functions package backend** --> Change from using DistributedLog to using local filesystem, storing the jars directly in the data folder instead of uploading them into BK. 3. **Pulsar functions state store implementation** --> Change the state store to be backed by a MetadataStore based backed, with the RocksDB implementation. 4. **Table Service** --> Do not start BK table service by default ## Compatibility considerations In order to avoid compatibility issues where users have existing Pulsar standalone services that they want to upgrade without conflicts, we will follow the principle of keeping the old defaults where there is existing data on the disk. We will add a file, serving the purpose as a flag, in the `data/standalone` directory, for example `new-2.10-defaults`. If the file is present, or if the data directory is completely missing, we will adopt the new set of default configuration settings. If the file is not there, we will continue to use existing defaults and we will not break the upgrade operation. -- Matteo Merli
RE: [VOTE] Pulsar Node.js Client Release 1.5.0 Candidate 1
+1 (binding) * Checked the license headers * Verified the checksum and signature * Built the source and run producer/consumer Masahiro Sakamoto Yahoo Japan Corp. E-mail: massa...@yahoo-corp.jp -Original Message- From: Nozomi Kurihara Sent: Friday, December 17, 2021 8:25 PM To: dev@pulsar.apache.org Subject: Re: [VOTE] Pulsar Node.js Client Release 1.5.0 Candidate 1 +1 (binding) * check the license headers * build the source and run producer/consumer * verify checksum and signatures Thanks, Nozomi 2021年12月16日(木) 9:29 Hiroyuki Sakai : > +1 (binding) > > * check the license headers > * build the source and run producer/consumer > * verify checksum and signatures > > == > Hiroyuki Sakai > Yahoo Japan Corp. > E-mail: hsa...@yahoo-corp.jp > > > From: Masahiro Sakamoto > Sent: Tuesday, December 14, 2021 17:55 > To: dev@pulsar.apache.org > Subject: [VOTE] Pulsar Node.js Client Release 1.5.0 Candidate 1 > > Hi everyone, > Please review and vote on the release candidate #1 for the version 1.5.0, > as follows: > [ ] +1, Approve the release > [ ] -1, Do not approve the release (please provide specific comments) > > This is the first release candidate for Apache Pulsar Node.js client, > version 1.5.0. > > It fixes the following issues: > > https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar-client-node%2Fmilestone%2F7%3Fclosed%3D1&data=04%7C01%7Cmassakam%40yahoo-corp.jp%7C49982e9f5e2c45b2eba508d9c14fe19a%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637755647515707863%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2F5mSZ%2FyDvRkqS48XLgQir1WPwFVAj2hTsDdo3mSxsOA%3D&reserved=0 > > Please download the source files and review this release candidate: > - Review release notes > - Download the source package (verify shasum and asc) and follow the > README.md to build and run the Pulsar Node.js client. > > The vote will be open for at least 72 hours. It is adopted by majority > approval, with at least 3 PMC affirmative votes. > > Source files: > > https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fpulsar%2Fpulsar-client-node%2Fpulsar-client-node-1.5.0-candidate-1%2F&data=04%7C01%7Cmassakam%40yahoo-corp.jp%7C49982e9f5e2c45b2eba508d9c14fe19a%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637755647515707863%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=s4QKkLaynsXz0ISAUxJgwQJcsJPyPtSPi7%2FHJqyhfSw%3D&reserved=0 > > Pulsar's KEYS file containing PGP keys we use to sign the release: > > https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fpulsar%2FKEYS&data=04%7C01%7Cmassakam%40yahoo-corp.jp%7C49982e9f5e2c45b2eba508d9c14fe19a%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637755647515707863%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=x2AQBOCSAaP1%2Fkhoa5JmIEUkNihQq25W%2F17hKGoM1vM%3D&reserved=0 > > SHA-512 checksum: > 5c05c5368fb822165dde5c3aa758d600eca4452e1cc160f9b53311e27436e4cd52fbc55ca6382bb56019e4b0a3a6a409f73eec7c19844fb60715e41348fd8c09 > pulsar-client-node-1.5.0.tar.gz > > The tag to be voted upon: > v1.5.0-rc.1 > > https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar-client-node%2Freleases%2Ftag%2Fv1.5.0-rc.1&data=04%7C01%7Cmassakam%40yahoo-corp.jp%7C49982e9f5e2c45b2eba508d9c14fe19a%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637755647515707863%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=fEQ79Od9HRTBsvxNEbB%2FX4eT0XexQL08GQt4yEyY%2FnE%3D&reserved=0 > > Masahiro Sakamoto > Yahoo Japan Corp. > E-mail: massa...@yahoo-corp.jp >
RE: [VOTE] Pulsar Node.js Client Release 1.5.0 Candidate 1
Closing the vote with 3 +1 and no -1. 3 +1 bindings are: * Nozomi * Hiroyuki * Masahiro Masahiro Sakamoto Yahoo Japan Corp. E-mail: massa...@yahoo-corp.jp -Original Message- From: Masahiro Sakamoto Sent: Monday, December 20, 2021 3:31 PM To: dev@pulsar.apache.org Subject: RE: [VOTE] Pulsar Node.js Client Release 1.5.0 Candidate 1 +1 (binding) * Checked the license headers * Verified the checksum and signature * Built the source and run producer/consumer Masahiro Sakamoto Yahoo Japan Corp. E-mail: massa...@yahoo-corp.jp -Original Message- From: Nozomi Kurihara Sent: Friday, December 17, 2021 8:25 PM To: dev@pulsar.apache.org Subject: Re: [VOTE] Pulsar Node.js Client Release 1.5.0 Candidate 1 +1 (binding) * check the license headers * build the source and run producer/consumer * verify checksum and signatures Thanks, Nozomi 2021年12月16日(木) 9:29 Hiroyuki Sakai : > +1 (binding) > > * check the license headers > * build the source and run producer/consumer > * verify checksum and signatures > > == > Hiroyuki Sakai > Yahoo Japan Corp. > E-mail: hsa...@yahoo-corp.jp > > > From: Masahiro Sakamoto > Sent: Tuesday, December 14, 2021 17:55 > To: dev@pulsar.apache.org > Subject: [VOTE] Pulsar Node.js Client Release 1.5.0 Candidate 1 > > Hi everyone, > Please review and vote on the release candidate #1 for the version 1.5.0, > as follows: > [ ] +1, Approve the release > [ ] -1, Do not approve the release (please provide specific comments) > > This is the first release candidate for Apache Pulsar Node.js client, > version 1.5.0. > > It fixes the following issues: > > https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar-client-node%2Fmilestone%2F7%3Fclosed%3D1&data=04%7C01%7Cmassakam%40yahoo-corp.jp%7C19e34cb6b9854c56f10308d9c38270a8%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637755787262197639%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=IDQsteV3PcQQWvWFYAMMqx0Ln2LgWhTkGxQwGaxkVYE%3D&reserved=0 > > Please download the source files and review this release candidate: > - Review release notes > - Download the source package (verify shasum and asc) and follow the > README.md to build and run the Pulsar Node.js client. > > The vote will be open for at least 72 hours. It is adopted by majority > approval, with at least 3 PMC affirmative votes. > > Source files: > > https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fpulsar%2Fpulsar-client-node%2Fpulsar-client-node-1.5.0-candidate-1%2F&data=04%7C01%7Cmassakam%40yahoo-corp.jp%7C19e34cb6b9854c56f10308d9c38270a8%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637755787262197639%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=IM%2BFJCtq1kKITimJfwcQq%2B8zREooZTxlRse9IlRRR6k%3D&reserved=0 > > Pulsar's KEYS file containing PGP keys we use to sign the release: > > https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fpulsar%2FKEYS&data=04%7C01%7Cmassakam%40yahoo-corp.jp%7C19e34cb6b9854c56f10308d9c38270a8%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637755787262197639%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=PU%2BL5rkeUZhfIOGSctUOQK0mjvHJ3tb6AzRJ1wVziq0%3D&reserved=0 > > SHA-512 checksum: > 5c05c5368fb822165dde5c3aa758d600eca4452e1cc160f9b53311e27436e4cd52fbc55ca6382bb56019e4b0a3a6a409f73eec7c19844fb60715e41348fd8c09 > pulsar-client-node-1.5.0.tar.gz > > The tag to be voted upon: > v1.5.0-rc.1 > > https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar-client-node%2Freleases%2Ftag%2Fv1.5.0-rc.1&data=04%7C01%7Cmassakam%40yahoo-corp.jp%7C19e34cb6b9854c56f10308d9c38270a8%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637755787262197639%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Xlf5yQsKuZhDCUvXn8dPZ5oA7tvCXTfVjaTBI00UPSg%3D&reserved=0 > > Masahiro Sakamoto > Yahoo Japan Corp. > E-mail: massa...@yahoo-corp.jp >
Re: [DISCUSSION] PIP-117: Change Pulsar standalone defaults
+1 On Mon, 20 Dec 2021 at 13:42, Lan Liang wrote: > My two cents +1 , In my case, i sometimes need to run pulsar standalone in > some server with limited resources. > > > > Proposed changes > > The other point i can think of is maybe schema storage also need to change > from using BK to using local filesystem or some else. > > > > > > > Best Regards, > Lan Liang > On 12/17/2021 12:35，Sijie Guo wrote： > +1 > > On Tue, Dec 14, 2021 at 9:18 AM Matteo Merli wrote: > > https://github.com/apache/pulsar/issues/13302 > > Copying here for quoting convenience > > > > > > ## Motivation > > Pulsar standalone is the "Pulsar in a box" version of a Pulsar cluster, > where > all the components are started within the context of a single JVM process. > > Users are using the standalone as a way to get quickly started with Pulsar > or > in all the cases where it makes sense to have a single node deployment. > > Right now, the standalone is starting by default with many components, > several of > which are quite complex, since they are designed to be deployed in a > distributed > fashion. > > ## Goal > > Simplify the components of Pulsar standalone to achieve: > > 1. Reduce complexity > 2. Reduce startup time > 3. Reduce memory and CPU footprint of running standalone > > ## Proposed changes > > The proposal here is to change some of the default implementations that are > used for the Pulsar standalone. > > 1. **Metadata Store implementation** --> > Change from ZooKeeper to RocksDB > > 2. **Pulsar functions package backend** --> > Change from using DistributedLog to using local filesystem, storing > the > jars directly in the data folder instead of uploading them into BK. > > 3. **Pulsar functions state store implementation** --> > Change the state store to be backed by a MetadataStore based backed, > with the RocksDB implementation. > > 4. **Table Service** --> > Do not start BK table service by default > > ## Compatibility considerations > > In order to avoid compatibility issues where users have existing Pulsar > standalone services that they want to upgrade without conflicts, we will > follow the principle of keeping the old defaults where there is existing > data on the disk. > > We will add a file, serving the purpose as a flag, in the `data/standalone` > directory, for example `new-2.10-defaults`. > > If the file is present, or if the data directory is completely missing, we > will adopt the new set of default configuration settings. > > If the file is not there, we will continue to use existing defaults and we > will > not break the upgrade operation. > > > > > > -- > Matteo Merli > > >
[RESULT] [VOTE] Apache Pulsar 2.9.1 candidate 2
Hello everyone, The VOTE passed with 5 VOTEs, 3 of them were binding. - Matteo Merli (binding) - Enrico Olivelli (binding) - Peng Hui (binding) - Nicolò Boschi - Massimiliano Mirelli I will move forward with the next steps https://github.com/apache/pulsar/wiki/Release-process Enrico Il giorno sab 18 dic 2021 alle ore 11:41 PengHui Li ha scritto: > +1 binding > > Penghui > > Enrico Olivelli 于2021年12月18日 周六18:39写道： > > > +1 (binding) > > > > - Run release validation procedure > > - CI is passing on those sources > > > > Enrico > > > > Il giorno sab 18 dic 2021 alle ore 02:51 PengHui Li > > ha > > scritto: > > > > > > Will this issue be fixed in the future releases? > > > > > > Yes, 2.8.2 and 2.9.2 will fix the problem. > > > > > > Penghui > > > > > > On Sat, Dec 18, 2021 at 3:28 AM Massimiliano Mirelli < > > > massimilianomirelli...@gmail.com> wrote: > > > > > > > Thank you for the rc! > > > > > > > > +1 (non-binding) > > > > > > > > * verify sha512 checksums > > > > * verify gpg signatures > > > > * build pulsar-all docker image > > > > * execute Fallout distributed system test (produce / receive 10k > > > messages) > > > > > > > > Building the docker image as indicated in the README: > > > > > > > > mvn clean install -DskipTests > > > > mvn package -Pdocker,-main -am -pl docker/pulsar-all -DskipTests > > > > > > > > I still get the error described in this PR#11951 ( > > > > https://github.com/apache/pulsar/pull/11951) which I suppose has > been > > > > cherry picked in 2.9.1. > > > > > > > > Giving enough permissions to /docker/pulsar/scripts/ in the src > package > > > and > > > > then building the docker image again solved the issue. > > > > > > > > I also tested the problem with the image Enrico provided, this way: > > > > > > > > docker run -it --entrypoint bash eolivelli/pulsar-all:2.9.1rc2 > > > > ls -al /pulsar/bin > > > > > > > > and that one does have the right permissions. > > > > > > > > Enrico, did you build the image using the same mvn commands (^^^), or > > is > > > > there some other way to build it? > > > > > > > > Thank you, > > > > Max > > > > > > > > On Fri, 17 Dec 2021 at 16:18, 陳智弘 wrote: > > > > > > > > > Hi PengHu, > > > > > > > > > > Will this issue be fixed in the future releases? > > > > > > > > > > PengHui Li 於 2021年12月17日 週五 21:53 寫道： > > > > > > > > > > > Hi Enrico, > > > > > > > > > > > > I'm ok, it only happens when the message is without a schema > > version. > > > > > > So I'm not giving -1. > > > > > > > > > > > > Thanks, > > > > > > Penghui > > > > > > > > > > > > > > > > > > On Fri, Dec 17, 2021 at 7:33 PM Enrico Olivelli < > > eolive...@gmail.com > > > > > > > > > > wrote: > > > > > > > > > > > > > Peng Hui, > > > > > > > > > > > > > > Il giorno ven 17 dic 2021 alle ore 08:09 PengHui Li < > > > > > peng...@apache.org> > > > > > > > ha > > > > > > > scritto: > > > > > > > > > > > > > > > Checked: > > > > > > > > > > > > > > > > - Build from the src > > > > > > > > - Check signatures > > > > > > > > - Follow the validation process > > > > > > > > > > > > > > > > But when I try to verify PulsarSQL, got following exceptions: > > > > > > > > > > > > > > > > ``` > > > > > > > > 2021-12-17T14:58:18.958+0800 ERROR remote-task-callback-3 > > > > > > > > io.prestosql.execution.StageStateMachine Stage > > > > > > > > 20211217_065818_1_cahiv.1 failed > > > > > > > > > com.google.common.util.concurrent.UncheckedExecutionException: > > > > > > > > java.nio.BufferUnderflowException > > > > > > > > at > > > > > > > > com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2051) > > > > > > > > at > > com.google.common.cache.LocalCache.get(LocalCache.java:3951) > > > > > > > > at > > > > > com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) > > > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4935) > > > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.pulsar.sql.presto.PulsarSqlSchemaInfoProvider.getSchemaByVersion(PulsarSqlSchemaInfoProvider.java:76) > > > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.pulsar.sql.presto.PulsarRecordCursor.advanceNextPosition(PulsarRecordCursor.java:485) > > > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > io.prestosql.spi.connector.RecordPageSource.getNextPage(RecordPageSource.java:90) > > > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > io.prestosql.operator.TableScanOperator.getOutput(TableScanOperator.java:302) > > > > > > > > at > > io.prestosql.operator.Driver.processInternal(Driver.java:379) > > > > > > > > at > > > > io.prestosql.operator.Driver.lambda$processFor$8(Driver.java:283) > > > > > > > > at io.prestosql.operator.Driver.tryWithLock
[DISCUSSION] Produce chunk messages failed when topic level maxMessageSize is set
Hi Pulsar Community,
I discovered a bug that chunk messages producing fails if topic level
maxMessageSize is set [1]. The root cause of this issue is because chunk
message is using broker level maxMessageSize as chunk size. And topic level
maxMessageSize is always <= broker level maxMessageSize. So once it is set, the
on-going chunk message producing fails.
## Proposed changes
I would like to fix this by just skipping topic level maxMessageSize check in
org.apache.pulsar.broker.service.AbstractTopic#isExceedMaximumMessageSize.Topic
level maxMessageSize is introduced in [2], for the purpose of "easier to plan
resource quotas for client allocation". And IMO this change will not bring
further complex into this.
## Alternative
Add a client side topic level maxMessageSize and keep it synced with broker.
Required changes:
- [client] Add a new field
org.apache.pulsar.client.impl.ProducerBase#maxMessageSize to store this client
side topic level maxMessageSize.
- [PulsarApi.proto] Add a new field maxMessageSize in the
CommandProducerSuccess for the initial value of ProducerBase#maxMessageSize
- [PulsarApi.proto] Add a new Command like
CommandUpdateClientPolicy{producerId, maxMessageSize} to update
ProducerBase#maxMessageSize when topic level maxMessageSize is updated.
Further more, some other data consistency issues need be handled very carefully
when maxMessageSize is updated.
This alternative is complex but can also solve other topic level maxMessageSize
issue [3] when batching is enabled (non-batching case is solved with PR [4]).
Any suggestions or other use cases of topic level maxMessageSize will be
appreciated.
Thanks,
Haiting Jiang
[1] https://github.com/apache/pulsar/issues/13360
[2] https://github.com/apache/pulsar/pull/8732
[3] https://github.com/apache/pulsar/issues/12958
[4] https://github.com/apache/pulsar/pull/13147
[ANNOUNCE] Apache Pulsar Node.js client 1.5.0 released
The Apache Pulsar team is proud to announce Apache Pulsar Node.js client version 1.5.0. Pulsar is a highly scalable, low latency messaging platform running on commodity hardware. It provides simple pub-sub semantics over topics, guaranteed at-least-once delivery of messages, automatic cursor management for subscribers, and cross-datacenter replication. For Pulsar Node.js client release details and downloads, visit: https://www.npmjs.com/package/pulsar-client Release Notes are at: https://github.com/apache/pulsar-client-node/releases We would like to thank the contributors that made the release possible. Regards, The Pulsar Team
