On Fri, May 10, 2019 at 3:52 AM Joe F <j...@apache.org> wrote: > I think 5MB is too large and it should be reduced. :-) . I am with Kafka > on this one.. 1MB is good enough.
It seems common to see a message larger than 5MB in MySQL cdc :( > > Have we completely ruled out a split/join abstraction on the Pulsar > produce/consumer APIs? No backward compatibility, no client-server > capability mismatches, now that we have dedup, split/join should even be > resumable on an n/w interruption .. It can be easier to achieve when Transaction is supported. A large message can be split into a txn of smaller messages. > > > Joe > > On Thu, May 9, 2019 at 10:43 AM Matteo Merli <matteo.me...@gmail.com> > wrote: > > > Thanks Yong for starting the work on this. > > > > I think there should be more conditions specified in the PIP and we > should > > have > > a clear documentation on what the behavior will be and what kind of > errors > > an > > application will see. Especially we should cover how this will > > interact with proxy. > > > > > On client side, client should set max message size in configuration and > > it should be smaller than server support. > > > > If we just let the broker/proxy decide the max size, then we can avoid > > the configuration > > in the client. That removes a lot of complexity in how to handle > > situation when client max-size > > is smaller than broker max-size and broker needs to deliver a message > > to that consumer. > > > > Also, it becomes much easier to make the change from ops perspective, > > 1 single system instead of having to > > configure all the client applications. > > > > -- > > Matteo Merli > > <matteo.me...@gmail.com> > > > > > > On Wed, May 8, 2019 at 8:34 PM Sijie Guo <guosi...@gmail.com> wrote: > > > > > > Thanks Yong for the PIP. > > > > > > I moved your gist to Pulsar wiki page: > > > https://github.com/apache/pulsar/wiki/PIP-36%3A-Max-Message-Size > > > > > > The proposal looks good to me. +1 > > > > > > - Sijie > > > > > > On Mon, May 6, 2019 at 6:04 PM Yong Zhang <zhangyong1025...@gmail.com> > > > wrote: > > > > > > > hi all, > > > > > > > > > > > > Currently `MaxMessageSize` is hardcoded in Pulsar and it can’t be > > modified > > > > in server configuration. So there is no way when user want to modify > > the > > > > limit to transfer larger size message. > > > > > > > > Hence i propose adding a `MaxMessageSize` config in `broker.conf` to > > solve > > > > this problem. Because broker server will decide how much message size > > will > > > > be received so client need know how much message client can be sent. > > > > > > > > for details track following pip: > > > > https://gist.github.com/zymap/08ec1bb688d2da16e9cd363780480e7a > > > > > > >
