+1
Skip the topic level setting is better
Hang Chen <chenh...@apache.org> 於 2021年12月27日 週一 22:09 寫道：
> +1
>
> We'd better skip the topic level maxMessageSize check for chunk messages.
>
> Best,
> Hang
>
> PengHui Li <peng...@apache.org> 于2021年12月27日周一 22:07写道：
> >
> > +1,
> >
> > We can only skip the topic level messages size check for the chunk
> message.
> >
> > Regards,
> > Penghui
> >
> > On Mon, Dec 20, 2021 at 3:37 PM Haiting Jiang <jianghait...@apache.org>
> > wrote:
> >
> > > Hi Pulsar Community,
> > >
> > > I discovered a bug that chunk messages producing fails if topic level
> > > maxMessageSize is set [1]. The root cause of this issue is because
> chunk
> > > message is using broker level maxMessageSize as chunk size. And topic
> level
> > > maxMessageSize is always <= broker level maxMessageSize. So once it is
> set,
> > > the on-going chunk message producing fails.
> > >
> > > ## Proposed changes
> > > I would like to fix this by just skipping topic level maxMessageSize
> check
> > > in
> > >
> org.apache.pulsar.broker.service.AbstractTopic#isExceedMaximumMessageSize.Topic
> > > level maxMessageSize is introduced in [2], for the purpose of "easier
> to
> > > plan resource quotas for client allocation". And IMO this change will
> not
> > > bring further complex into this.
> > >
> > > ## Alternative
> > > Add a client side topic level maxMessageSize and keep it synced with
> > > broker.
> > >
> > > Required changes:
> > > - [client] Add a new field
> > > org.apache.pulsar.client.impl.ProducerBase#maxMessageSize to store this
> > > client side topic level maxMessageSize.
> > > - [PulsarApi.proto] Add a new field maxMessageSize in the
> > > CommandProducerSuccess for the initial value of
> ProducerBase#maxMessageSize
> > > - [PulsarApi.proto] Add a new Command like
> > > CommandUpdateClientPolicy{producerId, maxMessageSize} to update
> > > ProducerBase#maxMessageSize when topic level maxMessageSize is updated.
> > > Further more, some other data consistency issues need be handled very
> > > carefully when maxMessageSize is updated.
> > > This alternative is complex but can also solve other topic level
> > > maxMessageSize issue [3] when batching is enabled (non-batching case is
> > > solved with PR [4]).
> > >
> > > Any suggestions or other use cases of topic level maxMessageSize will
> be
> > > appreciated.
> > >
> > > Thanks,
> > > Haiting Jiang
> > >
> > > [1] https://github.com/apache/pulsar/issues/13360
> > > [2] https://github.com/apache/pulsar/pull/8732
> > > [3] https://github.com/apache/pulsar/issues/12958
> > > [4] https://github.com/apache/pulsar/pull/13147
> > >
>
