> I think there should be more conditions specified in the PIP and we should > have a clear documentation on what the behavior will be and what kind of errors an application will see. Especially we should cover how this will interact with proxy.
Just to clarify my thoughts here, I was thinking of what happens in these cases * Old client doesn't adapt to smaller limit from broker * Will it keep reconnecting (in a flood, possibly) when the producer retries on the same "big" message? -- Matteo Merli <matteo.me...@gmail.com> On Thu, May 9, 2019 at 1:32 PM Matteo Merli <matteo.me...@gmail.com> wrote: > > > I think 5MB is too large and it should be reduced. :-) . I am with Kafka > on this one.. 1MB is good enough. > > It would be easy to configure in such way if broker advertises 1MB and > client complies with it. > > -- > Matteo Merli > <matteo.me...@gmail.com> > > On Thu, May 9, 2019 at 12:52 PM Joe F <j...@apache.org> wrote: > > > > I think 5MB is too large and it should be reduced. :-) . I am with Kafka > > on this one.. 1MB is good enough. > > > > Have we completely ruled out a split/join abstraction on the Pulsar > > produce/consumer APIs? No backward compatibility, no client-server > > capability mismatches, now that we have dedup, split/join should even be > > resumable on an n/w interruption .. > > > > > > Joe > > > > On Thu, May 9, 2019 at 10:43 AM Matteo Merli <matteo.me...@gmail.com> wrote: > > > > > Thanks Yong for starting the work on this. > > > > > > I think there should be more conditions specified in the PIP and we should > > > have > > > a clear documentation on what the behavior will be and what kind of errors > > > an > > > application will see. Especially we should cover how this will > > > interact with proxy. > > > > > > > On client side, client should set max message size in configuration and > > > it should be smaller than server support. > > > > > > If we just let the broker/proxy decide the max size, then we can avoid > > > the configuration > > > in the client. That removes a lot of complexity in how to handle > > > situation when client max-size > > > is smaller than broker max-size and broker needs to deliver a message > > > to that consumer. > > > > > > Also, it becomes much easier to make the change from ops perspective, > > > 1 single system instead of having to > > > configure all the client applications. > > > > > > -- > > > Matteo Merli > > > <matteo.me...@gmail.com> > > > > > > > > > On Wed, May 8, 2019 at 8:34 PM Sijie Guo <guosi...@gmail.com> wrote: > > > > > > > > Thanks Yong for the PIP. > > > > > > > > I moved your gist to Pulsar wiki page: > > > > https://github.com/apache/pulsar/wiki/PIP-36%3A-Max-Message-Size > > > > > > > > The proposal looks good to me. +1 > > > > > > > > - Sijie > > > > > > > > On Mon, May 6, 2019 at 6:04 PM Yong Zhang <zhangyong1025...@gmail.com> > > > > wrote: > > > > > > > > > hi all, > > > > > > > > > > > > > > > Currently `MaxMessageSize` is hardcoded in Pulsar and it can’t be > > > modified > > > > > in server configuration. So there is no way when user want to modify > > > the > > > > > limit to transfer larger size message. > > > > > > > > > > Hence i propose adding a `MaxMessageSize` config in `broker.conf` to > > > solve > > > > > this problem. Because broker server will decide how much message size > > > will > > > > > be received so client need know how much message client can be sent. > > > > > > > > > > for details track following pip: > > > > > https://gist.github.com/zymap/08ec1bb688d2da16e9cd363780480e7a > > > > > > > >
