> I think it's a good time for 2.10 to enable this setting by default and, > correspondingly, to disable by default the producer queue size limit.
+1 > 64MB is picked because it's a small enough memory size that will guarantee > a very high producer throughput, irrespective of the individual messages size. Did you consider determining the default limit by inspecting the JVM's runtime? For example, in the broker, we set `maxMessagePublishBufferSizeInMB` to 1/2 of direct memory, by default. I concede that this config is more complicated than a broker config since we're talking about a client application. However, it seems like applications with 1 GB memory would probably want a different setting than one with 32 GB memory, and a fractional value could make the client work better "out of the box." We cannot make everyone happy with a default value, so if this dynamic default is too complicated to implement, I'm good with 64 MB. :) Thanks, Michael On Sun, Dec 19, 2021 at 11:17 PM Lan Liang <liangyuanpen...@163.com> wrote: > > +1 > > > On 12/17/2021 14:43，mattison chao<mattisonc...@gmail.com> wrote： > +1 > > On Fri, 17 Dec 2021 at 13:56, 陳智弘 <thomasec...@gmail.com> wrote: > > +1 > > Sijie Guo <guosi...@gmail.com> 於 2021年12月17日 週五 12:38 寫道： > > +1 > > On Tue, Dec 14, 2021 at 11:20 AM Matteo Merli <mme...@apache.org> wrote: > > https://github.com/apache/pulsar/issues/13306 > > > Pasted below for quoting convenience. > > > -------- > > ## Motivation > > In Pulsar 2.8, we have introduced a setting to control the amount of > memory > used by a client instance. > > ```java > interface ClientBuilder { > ClientBuilder memoryLimit(long memoryLimit, SizeUnit unit); > } > ``` > > By default, in 2.8 and 2.9 this setting is set to 0, meaning no limit > is > being > enforced. > > I think it's a good time for 2.10 to enable this setting by default > and, > correspondingly, to disable by default the producer queue size limit. > > This will simplify a lot the configuration that a producer application > will > have to come up with, when publishing with many topic/partitions or > when messages > are bigger than expected. > > ## Proposed changes > > In 2.10 release, for the `ClientBuilder`, change > * `memoryLimit`: 0 -> 64 MB > > For the `ProducerBuilder`, changes > * `maxPendingMessages`: 1000 -> 0 > > 64MB is picked because it's a small enough memory size that will > guarantee > a very high producer throughput, irrespective of the individual > messages > size. > > > > -- > Matteo Merli > <mme...@apache.org> > > >
