Hmmm
I found that if we want to provide the public API(shared event loop) in the
pulsar-client-api module.
We need to add Netty dependency to the pulsar-client-api module.
It's not a good idea to have Netty dependency in the pulsar-client-api
module.
Maybe we can add the new API to the pulsar-client-original module? like we
currently do.
```
PulsarClientImpl.builder()...
```
We only changed the default value of the thread pool. Users still have the
ability to change the threads
that they want to assign. So if they want to use multiple client instances,
they can also change the threads
options.
Thanks,
Penghui
On Tue, Dec 27, 2022 at 8:35 PM PengHui Li <peng...@apache.org> wrote:
> Ah, got it.
>
> It's a good idea for users to be aware of all shared resources.
> I will change the proposal as per your suggestion
>
> Thanks,
> Penghui
>
> On Tue, Dec 27, 2022 at 7:11 PM Enrico Olivelli <eolive...@gmail.com>
> wrote:
>
>> I generally support this proposal,
>> this is a problem we have in the Proxy and I have seen it on
>> applications that need to connect to multiple different tenants
>> and they need different authentication parameters, so they have to
>> create many PulsarClient instances.
>>
>> I have a suggestion:
>>
>> Exposing all the internals is a good idea for very advanced users,
>> but I believe that we should provide some simpler support.
>>
>> We should have an API to allow sharing resources among multiple
>> clients without entering the details.
>>
>> Interface PulsarClientGroup {
>> ... put here all the sharable things in the current version...
>> }
>>
>> PulsarClient client = ....newClient().
>> .withSharedResources(pulsarClientGroup)
>> ...
>>
>>
>> I think that having a PulsarClientGroup is a good choice for future
>> evolutions
>> because the internal thread pools may change: removed/added/change the
>> purpose.
>>
>> If we require users to deal with all the possible sharable resources
>> then we have a few risks:
>> - people can "forget" to share some resources
>> - upgrading the client may lead to not taking into account some new
>> "shareable" resources
>>
>> This is way I believe that we should provide an opaque
>>
>> Enrico
>>
>> Il giorno mar 27 dic 2022 alle ore 11:07 PengHui Li
>> <peng...@apache.org> ha scritto:
>> >
>> > Hi all,
>> >
>> > As discussed at
>> > https://lists.apache.org/thread/5obfm17g58n3dnbzyxg57vokgmwyp6hx
>> > I have created this proposal to support shared thread pool across
>> multiple
>> > client instances
>> > Here is the proposal link https://github.com/apache/pulsar/issues/19074
>> >
>> > Please help take a look, and look forward to your suggestions.
>> >
>> > Thanks,
>> > Penghui
>> >
>> > --------------------------------------------------
>> >
>> > ### Motivation
>> >
>> > The Pulsar client mainly has three thread pools that cooperate with each
>> > other to complete the message publishing and consumption of messages.
>> >
>> > - IO threads - Used for handling network packets from the broker
>> > - Internal threads - Used for handling internal tasks such as moving the
>> > received messages to the internal receiver queue and pulling out the
>> > message from the receiver queue to return to users. And the Java client
>> is
>> > optimized by the lock-free principle; each consumer will use a pinned
>> > internal thread to reduce the lock overhead.
>> > - External threads - Used by the message listener
>> >
>> > All the above thread pools will be created automatically after a Pulsar
>> > client instance has been created.
>> >
>> > But for some cases, users need to create multiple Pulsar client
>> instances
>> > in a JVM process due to different authentications or others. Each client
>> > will have exclusive thread pools, which will cause unreasonable thread
>> > usage, waste memory, and potential performance degradation.
>> >
>> > It is not a serious problem for previous releases with the default
>> > configurations because the thread pool will only have 1 thread by
>> default.
>> > But it also doesn't make sense that we only have one thread for each
>> thread
>> > pool. We have discussed this part under this [thread](
>> > https://lists.apache.org/thread/5obfm17g58n3dnbzyxg57vokgmwyp6hx)
>> >
>> > So this proposal will provide a new possibility for users that require
>> > multiple Pulsar client instances in one JVM process to use the shared
>> > thread pools across multiple Pulsar client instances.
>> >
>> > ### Goal
>> >
>> > Provide public API to use the shared thread pool across multiple Pulsar
>> > client instances in one JVM process
>> >
>> > - IO threads
>> > - Internal threads
>> > - External threads
>> >
>> > BTW, we already have such an ability internally. It was just hidden for
>> > users. Please take a look at #12037 and #13839 to get more details.
>> >
>> > ### API Changes
>> >
>> > The following APIs will be introduced to the Java Client when creating a
>> > Client instance
>> >
>> > ```java
>> > PulsarClient.builder()
>> > .eventLoopGroup(ioEventLoopGroup)
>> > .internalExecutorProvider(sharedInternalExecutorProvider)
>> > .externalExecutorProvider(sharedExternalExecutorProvider)
>> > .scheduledExecutorProvider(sharedScheduledExecutorProvider)
>> > ```
>>
>
