+1
Regards
Jiwei Guo (Tboy)
On Tue, Dec 28, 2021 at 5:44 AM 陳智弘 <thomasec...@gmail.com> wrote:
>
> +1
>
> Skip the topic level setting is better
>
> Hang Chen <chenh...@apache.org> 於 2021年12月27日 週一 22:09 寫道：
>
> > +1
> >
> > We'd better skip the topic level maxMessageSize check for chunk messages.
> >
> > Best,
> > Hang
> >
> > PengHui Li <peng...@apache.org> 于2021年12月27日周一 22:07写道：
> > >
> > > +1,
> > >
> > > We can only skip the topic level messages size check for the chunk
> > message.
> > >
> > > Regards,
> > > Penghui
> > >
> > > On Mon, Dec 20, 2021 at 3:37 PM Haiting Jiang <jianghait...@apache.org>
> > > wrote:
> > >
> > > > Hi Pulsar Community,
> > > >
> > > > I discovered a bug that chunk messages producing fails if topic level
> > > > maxMessageSize is set [1]. The root cause of this issue is because
> > chunk
> > > > message is using broker level maxMessageSize as chunk size. And topic
> > level
> > > > maxMessageSize is always <= broker level maxMessageSize. So once it is
> > set,
> > > > the on-going chunk message producing fails.
> > > >
> > > > ## Proposed changes
> > > > I would like to fix this by just skipping topic level maxMessageSize
> > check
> > > > in
> > > >
> > org.apache.pulsar.broker.service.AbstractTopic#isExceedMaximumMessageSize.Topic
> > > > level maxMessageSize is introduced in [2], for the purpose of "easier
> > to
> > > > plan resource quotas for client allocation". And IMO this change will
> > not
> > > > bring further complex into this.
> > > >
> > > > ## Alternative
> > > > Add a client side topic level maxMessageSize and keep it synced with
> > > > broker.
> > > >
> > > > Required changes:
> > > > - [client] Add a new field
> > > > org.apache.pulsar.client.impl.ProducerBase#maxMessageSize to store this
> > > > client side topic level maxMessageSize.
> > > > - [PulsarApi.proto] Add a new field maxMessageSize in the
> > > > CommandProducerSuccess for the initial value of
> > ProducerBase#maxMessageSize
> > > > - [PulsarApi.proto] Add a new Command like
> > > > CommandUpdateClientPolicy{producerId, maxMessageSize} to update
> > > > ProducerBase#maxMessageSize when topic level maxMessageSize is updated.
> > > > Further more, some other data consistency issues need be handled very
> > > > carefully when maxMessageSize is updated.
> > > > This alternative is complex but can also solve other topic level
> > > > maxMessageSize issue [3] when batching is enabled (non-batching case is
> > > > solved with PR [4]).
> > > >
> > > > Any suggestions or other use cases of topic level maxMessageSize will
> > be
> > > > appreciated.
> > > >
> > > > Thanks,
> > > > Haiting Jiang
> > > >
> > > > [1] https://github.com/apache/pulsar/issues/13360
> > > > [2] https://github.com/apache/pulsar/pull/8732
> > > > [3] https://github.com/apache/pulsar/issues/12958
> > > > [4] https://github.com/apache/pulsar/pull/13147
> > > >
> >
