> My concern is whether this change will affect some users who are creating > many clients.
This is a good point. There's an internal API to share the thread pools across clients. There's a code example in the Pulsar code base, https://github.com/apache/pulsar/blob/f029757ecb9653a97710f2bac563d247d04feb63/pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java#L1463-L1473 . This change was made by https://github.com/apache/pulsar/pull/12037 and https://github.com/apache/pulsar/pull/13839 . Perhaps it would be useful to expose the solution to share thread pools across clients. A better abstraction and end user API for doing that would be useful so that the details of individual thread pools would be hidden. The current code for sharing the thread pools is very verbose as you can see from the referenced code example. -Lari On 2022/12/20 04:25:06 mattisonc...@gmail.com wrote: > +1 > > My concern is whether this change will affect some users who are creating > many clients. I think we can wait for other users to confirm it. (If this > will be affected, maybe we can give it a max_io_thread_num and then expand > the size from 1 to max_io_thread_num when adding a new consumer or producer) > > > Best, > Mattison > On Dec 20, 2022, 11:17 +0800, PengHui Li <peng...@apache.org>, wrote: > > Hi all, > > > > I noticed the Java Client (I haven't checked other clients) uses 1 IO > > thread and 1 listener > > thread by default. It will require users to update the thread configuration > > if they have > > multiple cores and desired high throughput. > > > > Here is the example that we change to 16 IO threads in > > openmessaging benchmark > > https://github.com/openmessaging/benchmark/blob/master/driver-pulsar/pulsar.yaml#L22 > > > > We can apply the configuration of the threads based on the CPU cores. So > > that for the > > most common cases, users don't need to touch the thread configuration. > > > > ``` > > private int numIoThreads = Runtime.getRuntime().availableProcessors(); > > private int numListenerThreads = Runtime.getRuntime().availableProcessors(); > > ``` > > > > WDYT? > > > > Thanks, > > Penghui >
