>> Also I have a few questions regarding chunking. I will leave it in your doc.
can you please use below link as previous doc-link was not allowing to comment. https://docs.google.com/document/d/13F2QLcOBVUJhVKQJVsV7yajYpfdp5pDL0qmMbTb5A3Q/edit?usp=sharing >> Since this is a different topic than configuring max message size, does it make sense to move it to a separated topic? Sure, will create a separate PIP for it. Thanks, Rajan On Tue, May 14, 2019 at 3:14 PM Sijie Guo <guosi...@gmail.com> wrote: > Rajan, > > Since this is a different topic than configuring max message size, does it > make sense to move it to a separated topic? > > Also I have a few questions regarding chunking. I will leave it in your > doc. > > - Sijie > > On Fri, May 10, 2019 at 1:08 PM Rajan Dhabalia <rdhaba...@apache.org> > wrote: > > > Hi, > > > > >> It can be easier to achieve when Transaction is supported. A large > > message > > can be split into a txn of smaller messages. > > > > I would like to propose another approach "Chunking and combine" to > publish > > large size of messages. Transaction deployment might not be convenient > all > > the time specially for the large traffic pulsar system due to multiple > > reasons which I have shared in the doc. > > > > > https://docs.google.com/document/d/13F2QLcOBVUJhVKQJVsV7yajYpfdp5pDL0qmMbTb5A3Q/edit?usp=sharing > > So, we would like to propose above approach to address publishing of > large > > message size. > > > > Thanks, > > Rajan > > > > On Thu, May 9, 2019 at 7:38 PM Yong Zhang <zhangyong1025...@gmail.com> > > wrote: > > > > > > > > > > If we just let the broker/proxy decide the max size, then we can > avoid > > > > the configuration > > > > in the client. That removes a lot of complexity in how to handle > > > > situation when client max-size > > > > is smaller than broker max-size and broker needs to deliver a message > > > > to that consumer. > > > > > > > > > > Thank you. Actually implement as this way. I will fix the description > in > > > pip. > > > On Fri, 10 May 2019 at 01:43, Matteo Merli <matteo.me...@gmail.com> > > wrote: > > > > > > > Thanks Yong for starting the work on this. > > > > > > > > I think there should be more conditions specified in the PIP and we > > > should > > > > have > > > > a clear documentation on what the behavior will be and what kind of > > > errors > > > > an > > > > application will see. Especially we should cover how this will > > > > interact with proxy. > > > > > > > > > On client side, client should set max message size in configuration > > and > > > > it should be smaller than server support. > > > > > > > > If we just let the broker/proxy decide the max size, then we can > avoid > > > > the configuration > > > > in the client. That removes a lot of complexity in how to handle > > > > situation when client max-size > > > > is smaller than broker max-size and broker needs to deliver a message > > > > to that consumer. > > > > > > > > Also, it becomes much easier to make the change from ops perspective, > > > > 1 single system instead of having to > > > > configure all the client applications. > > > > > > > > -- > > > > Matteo Merli > > > > <matteo.me...@gmail.com> > > > > > > > > > > > > On Wed, May 8, 2019 at 8:34 PM Sijie Guo <guosi...@gmail.com> wrote: > > > > > > > > > > Thanks Yong for the PIP. > > > > > > > > > > I moved your gist to Pulsar wiki page: > > > > > https://github.com/apache/pulsar/wiki/PIP-36%3A-Max-Message-Size > > > > > > > > > > The proposal looks good to me. +1 > > > > > > > > > > - Sijie > > > > > > > > > > On Mon, May 6, 2019 at 6:04 PM Yong Zhang < > > zhangyong1025...@gmail.com> > > > > > wrote: > > > > > > > > > > > hi all, > > > > > > > > > > > > > > > > > > Currently `MaxMessageSize` is hardcoded in Pulsar and it can’t be > > > > modified > > > > > > in server configuration. So there is no way when user want to > > modify > > > > the > > > > > > limit to transfer larger size message. > > > > > > > > > > > > Hence i propose adding a `MaxMessageSize` config in `broker.conf` > > to > > > > solve > > > > > > this problem. Because broker server will decide how much message > > size > > > > will > > > > > > be received so client need know how much message client can be > > sent. > > > > > > > > > > > > for details track following pip: > > > > > > https://gist.github.com/zymap/08ec1bb688d2da16e9cd363780480e7a > > > > > > > > > > > > > > > >
