This is the voting thread for PIP-131. It will stay open for at least 48h. https://github.com/apache/pulsar/issues/13544
The discussion thread is https://lists.apache.org/thread/c63d9s73j9x1m3dkqr3r38gyp8s7cwzf ## Motivation Currently, chunk messages producing fails if topic level maxMessageSize is set [1]. The root cause of this issue is because chunk message is using broker level maxMessageSize as chunk size. And topic level maxMessageSize is always <= broker level maxMessageSize. So once it is set, the on-going chunk message producing fails. ## Goal Resolve topic level maxMessageSize compatibility issue with chunking messages. ## Implementation Current best solution would be just skipping topic level maxMessageSize check in org.apache.pulsar.broker.service.AbstractTopic#isExceedMaximumMessageSize. Topic level maxMessageSize is introduced in [2], for the purpose of "easier to plan resource quotas for client allocation". And IMO this change will not bring further complex into this. ## Reject Alternatives Add a client side topic level maxMessageSize and keep it synced with broker. Required changes: - [client] Add a new field org.apache.pulsar.client.impl.ProducerBase#maxMessageSize to store this client side topic level maxMessageSize. - [PulsarApi.proto] Add a new field maxMessageSize in the CommandProducerSuccess for the initial value of ProducerBase#maxMessageSize - [PulsarApi.proto] Add a new Command like CommandUpdateClientPolicy{producerId, maxMessageSize} to update ProducerBase#maxMessageSize when topic level maxMessageSize is updated. Further more, some other data consistency issues need be handled very carefully when maxMessageSize is updated. This alternative is complex but can also solve other topic level maxMessageSize issue [3] when batching is enabled (non-batching case is solved with PR [4]). [1] https://github.com/apache/pulsar/issues/13360 [2] https://github.com/apache/pulsar/pull/8732 [3] https://github.com/apache/pulsar/issues/12958 [4] https://github.com/apache/pulsar/pull/13147 Thanks, Haiting Jiang
