It looks loke you already prepared a awesome PIP!
Please copy your proposal on a PIP.
I am +1
Enrico
Il Mar 28 Dic 2021, 04:27 Zike Yang <zky...@streamnative.io.invalid> ha
scritto:
> + 1 for skipping the topic level message size check for the chunk message
> in the broker.
>
> On Tue, Dec 28, 2021 at 9:14 AM guo jiwei <techno...@apache.org> wrote:
>
> > +1
> >
> >
> > Regards
> > Jiwei Guo (Tboy)
> >
> > On Tue, Dec 28, 2021 at 5:44 AM 陳智弘 <thomasec...@gmail.com> wrote:
> > >
> > > +1
> > >
> > > Skip the topic level setting is better
> > >
> > > Hang Chen <chenh...@apache.org> 於 2021年12月27日 週一 22:09 寫道：
> > >
> > > > +1
> > > >
> > > > We'd better skip the topic level maxMessageSize check for chunk
> > messages.
> > > >
> > > > Best,
> > > > Hang
> > > >
> > > > PengHui Li <peng...@apache.org> 于2021年12月27日周一 22:07写道：
> > > > >
> > > > > +1,
> > > > >
> > > > > We can only skip the topic level messages size check for the chunk
> > > > message.
> > > > >
> > > > > Regards,
> > > > > Penghui
> > > > >
> > > > > On Mon, Dec 20, 2021 at 3:37 PM Haiting Jiang <
> > jianghait...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Hi Pulsar Community,
> > > > > >
> > > > > > I discovered a bug that chunk messages producing fails if topic
> > level
> > > > > > maxMessageSize is set [1]. The root cause of this issue is
> because
> > > > chunk
> > > > > > message is using broker level maxMessageSize as chunk size. And
> > topic
> > > > level
> > > > > > maxMessageSize is always <= broker level maxMessageSize. So once
> > it is
> > > > set,
> > > > > > the on-going chunk message producing fails.
> > > > > >
> > > > > > ## Proposed changes
> > > > > > I would like to fix this by just skipping topic level
> > maxMessageSize
> > > > check
> > > > > > in
> > > > > >
> > > >
> >
> org.apache.pulsar.broker.service.AbstractTopic#isExceedMaximumMessageSize.Topic
> > > > > > level maxMessageSize is introduced in [2], for the purpose of
> > "easier
> > > > to
> > > > > > plan resource quotas for client allocation". And IMO this change
> > will
> > > > not
> > > > > > bring further complex into this.
> > > > > >
> > > > > > ## Alternative
> > > > > > Add a client side topic level maxMessageSize and keep it synced
> > with
> > > > > > broker.
> > > > > >
> > > > > > Required changes:
> > > > > > - [client] Add a new field
> > > > > > org.apache.pulsar.client.impl.ProducerBase#maxMessageSize to
> store
> > this
> > > > > > client side topic level maxMessageSize.
> > > > > > - [PulsarApi.proto] Add a new field maxMessageSize in the
> > > > > > CommandProducerSuccess for the initial value of
> > > > ProducerBase#maxMessageSize
> > > > > > - [PulsarApi.proto] Add a new Command like
> > > > > > CommandUpdateClientPolicy{producerId, maxMessageSize} to update
> > > > > > ProducerBase#maxMessageSize when topic level maxMessageSize is
> > updated.
> > > > > > Further more, some other data consistency issues need be handled
> > very
> > > > > > carefully when maxMessageSize is updated.
> > > > > > This alternative is complex but can also solve other topic level
> > > > > > maxMessageSize issue [3] when batching is enabled (non-batching
> > case is
> > > > > > solved with PR [4]).
> > > > > >
> > > > > > Any suggestions or other use cases of topic level maxMessageSize
> > will
> > > > be
> > > > > > appreciated.
> > > > > >
> > > > > > Thanks,
> > > > > > Haiting Jiang
> > > > > >
> > > > > > [1] https://github.com/apache/pulsar/issues/13360
> > > > > > [2] https://github.com/apache/pulsar/pull/8732
> > > > > > [3] https://github.com/apache/pulsar/issues/12958
> > > > > > [4] https://github.com/apache/pulsar/pull/13147
> > > > > >
> > > >
> >
>
>
> --
> Zike Yang
>
