Hi, >> It can be easier to achieve when Transaction is supported. A large message can be split into a txn of smaller messages.
I would like to propose another approach "Chunking and combine" to publish large size of messages. Transaction deployment might not be convenient all the time specially for the large traffic pulsar system due to multiple reasons which I have shared in the doc. https://docs.google.com/document/d/13F2QLcOBVUJhVKQJVsV7yajYpfdp5pDL0qmMbTb5A3Q/edit?usp=sharing So, we would like to propose above approach to address publishing of large message size. Thanks, Rajan On Thu, May 9, 2019 at 7:38 PM Yong Zhang <zhangyong1025...@gmail.com> wrote: > > > > If we just let the broker/proxy decide the max size, then we can avoid > > the configuration > > in the client. That removes a lot of complexity in how to handle > > situation when client max-size > > is smaller than broker max-size and broker needs to deliver a message > > to that consumer. > > > > Thank you. Actually implement as this way. I will fix the description in > pip. > On Fri, 10 May 2019 at 01:43, Matteo Merli <matteo.me...@gmail.com> wrote: > > > Thanks Yong for starting the work on this. > > > > I think there should be more conditions specified in the PIP and we > should > > have > > a clear documentation on what the behavior will be and what kind of > errors > > an > > application will see. Especially we should cover how this will > > interact with proxy. > > > > > On client side, client should set max message size in configuration and > > it should be smaller than server support. > > > > If we just let the broker/proxy decide the max size, then we can avoid > > the configuration > > in the client. That removes a lot of complexity in how to handle > > situation when client max-size > > is smaller than broker max-size and broker needs to deliver a message > > to that consumer. > > > > Also, it becomes much easier to make the change from ops perspective, > > 1 single system instead of having to > > configure all the client applications. > > > > -- > > Matteo Merli > > <matteo.me...@gmail.com> > > > > > > On Wed, May 8, 2019 at 8:34 PM Sijie Guo <guosi...@gmail.com> wrote: > > > > > > Thanks Yong for the PIP. > > > > > > I moved your gist to Pulsar wiki page: > > > https://github.com/apache/pulsar/wiki/PIP-36%3A-Max-Message-Size > > > > > > The proposal looks good to me. +1 > > > > > > - Sijie > > > > > > On Mon, May 6, 2019 at 6:04 PM Yong Zhang <zhangyong1025...@gmail.com> > > > wrote: > > > > > > > hi all, > > > > > > > > > > > > Currently `MaxMessageSize` is hardcoded in Pulsar and it can’t be > > modified > > > > in server configuration. So there is no way when user want to modify > > the > > > > limit to transfer larger size message. > > > > > > > > Hence i propose adding a `MaxMessageSize` config in `broker.conf` to > > solve > > > > this problem. Because broker server will decide how much message size > > will > > > > be received so client need know how much message client can be sent. > > > > > > > > for details track following pip: > > > > https://gist.github.com/zymap/08ec1bb688d2da16e9cd363780480e7a > > > > > > >
