Rajan, Since this is a different topic than configuring max message size, does it make sense to move it to a separated topic?
Also I have a few questions regarding chunking. I will leave it in your doc. - Sijie On Fri, May 10, 2019 at 1:08 PM Rajan Dhabalia <rdhaba...@apache.org> wrote: > Hi, > > >> It can be easier to achieve when Transaction is supported. A large > message > can be split into a txn of smaller messages. > > I would like to propose another approach "Chunking and combine" to publish > large size of messages. Transaction deployment might not be convenient all > the time specially for the large traffic pulsar system due to multiple > reasons which I have shared in the doc. > > https://docs.google.com/document/d/13F2QLcOBVUJhVKQJVsV7yajYpfdp5pDL0qmMbTb5A3Q/edit?usp=sharing > So, we would like to propose above approach to address publishing of large > message size. > > Thanks, > Rajan > > On Thu, May 9, 2019 at 7:38 PM Yong Zhang <zhangyong1025...@gmail.com> > wrote: > > > > > > > If we just let the broker/proxy decide the max size, then we can avoid > > > the configuration > > > in the client. That removes a lot of complexity in how to handle > > > situation when client max-size > > > is smaller than broker max-size and broker needs to deliver a message > > > to that consumer. > > > > > > > Thank you. Actually implement as this way. I will fix the description in > > pip. > > On Fri, 10 May 2019 at 01:43, Matteo Merli <matteo.me...@gmail.com> > wrote: > > > > > Thanks Yong for starting the work on this. > > > > > > I think there should be more conditions specified in the PIP and we > > should > > > have > > > a clear documentation on what the behavior will be and what kind of > > errors > > > an > > > application will see. Especially we should cover how this will > > > interact with proxy. > > > > > > > On client side, client should set max message size in configuration > and > > > it should be smaller than server support. > > > > > > If we just let the broker/proxy decide the max size, then we can avoid > > > the configuration > > > in the client. That removes a lot of complexity in how to handle > > > situation when client max-size > > > is smaller than broker max-size and broker needs to deliver a message > > > to that consumer. > > > > > > Also, it becomes much easier to make the change from ops perspective, > > > 1 single system instead of having to > > > configure all the client applications. > > > > > > -- > > > Matteo Merli > > > <matteo.me...@gmail.com> > > > > > > > > > On Wed, May 8, 2019 at 8:34 PM Sijie Guo <guosi...@gmail.com> wrote: > > > > > > > > Thanks Yong for the PIP. > > > > > > > > I moved your gist to Pulsar wiki page: > > > > https://github.com/apache/pulsar/wiki/PIP-36%3A-Max-Message-Size > > > > > > > > The proposal looks good to me. +1 > > > > > > > > - Sijie > > > > > > > > On Mon, May 6, 2019 at 6:04 PM Yong Zhang < > zhangyong1025...@gmail.com> > > > > wrote: > > > > > > > > > hi all, > > > > > > > > > > > > > > > Currently `MaxMessageSize` is hardcoded in Pulsar and it can’t be > > > modified > > > > > in server configuration. So there is no way when user want to > modify > > > the > > > > > limit to transfer larger size message. > > > > > > > > > > Hence i propose adding a `MaxMessageSize` config in `broker.conf` > to > > > solve > > > > > this problem. Because broker server will decide how much message > size > > > will > > > > > be received so client need know how much message client can be > sent. > > > > > > > > > > for details track following pip: > > > > > https://gist.github.com/zymap/08ec1bb688d2da16e9cd363780480e7a > > > > > > > > > > >
