Ah, got it.
It's a good idea for users to be aware of all shared resources.
I will change the proposal as per your suggestion
Thanks,
Penghui
On Tue, Dec 27, 2022 at 7:11 PM Enrico Olivelli <eolive...@gmail.com> wrote:
> I generally support this proposal,
> this is a problem we have in the Proxy and I have seen it on
> applications that need to connect to multiple different tenants
> and they need different authentication parameters, so they have to
> create many PulsarClient instances.
>
> I have a suggestion:
>
> Exposing all the internals is a good idea for very advanced users,
> but I believe that we should provide some simpler support.
>
> We should have an API to allow sharing resources among multiple
> clients without entering the details.
>
> Interface PulsarClientGroup {
> ... put here all the sharable things in the current version...
> }
>
> PulsarClient client = ....newClient().
> .withSharedResources(pulsarClientGroup)
> ...
>
>
> I think that having a PulsarClientGroup is a good choice for future
> evolutions
> because the internal thread pools may change: removed/added/change the
> purpose.
>
> If we require users to deal with all the possible sharable resources
> then we have a few risks:
> - people can "forget" to share some resources
> - upgrading the client may lead to not taking into account some new
> "shareable" resources
>
> This is way I believe that we should provide an opaque
>
> Enrico
>
> Il giorno mar 27 dic 2022 alle ore 11:07 PengHui Li
> <peng...@apache.org> ha scritto:
> >
> > Hi all,
> >
> > As discussed at
> > https://lists.apache.org/thread/5obfm17g58n3dnbzyxg57vokgmwyp6hx
> > I have created this proposal to support shared thread pool across
> multiple
> > client instances
> > Here is the proposal link https://github.com/apache/pulsar/issues/19074
> >
> > Please help take a look, and look forward to your suggestions.
> >
> > Thanks,
> > Penghui
> >
> > --------------------------------------------------
> >
> > ### Motivation
> >
> > The Pulsar client mainly has three thread pools that cooperate with each
> > other to complete the message publishing and consumption of messages.
> >
> > - IO threads - Used for handling network packets from the broker
> > - Internal threads - Used for handling internal tasks such as moving the
> > received messages to the internal receiver queue and pulling out the
> > message from the receiver queue to return to users. And the Java client
> is
> > optimized by the lock-free principle; each consumer will use a pinned
> > internal thread to reduce the lock overhead.
> > - External threads - Used by the message listener
> >
> > All the above thread pools will be created automatically after a Pulsar
> > client instance has been created.
> >
> > But for some cases, users need to create multiple Pulsar client instances
> > in a JVM process due to different authentications or others. Each client
> > will have exclusive thread pools, which will cause unreasonable thread
> > usage, waste memory, and potential performance degradation.
> >
> > It is not a serious problem for previous releases with the default
> > configurations because the thread pool will only have 1 thread by
> default.
> > But it also doesn't make sense that we only have one thread for each
> thread
> > pool. We have discussed this part under this [thread](
> > https://lists.apache.org/thread/5obfm17g58n3dnbzyxg57vokgmwyp6hx)
> >
> > So this proposal will provide a new possibility for users that require
> > multiple Pulsar client instances in one JVM process to use the shared
> > thread pools across multiple Pulsar client instances.
> >
> > ### Goal
> >
> > Provide public API to use the shared thread pool across multiple Pulsar
> > client instances in one JVM process
> >
> > - IO threads
> > - Internal threads
> > - External threads
> >
> > BTW, we already have such an ability internally. It was just hidden for
> > users. Please take a look at #12037 and #13839 to get more details.
> >
> > ### API Changes
> >
> > The following APIs will be introduced to the Java Client when creating a
> > Client instance
> >
> > ```java
> > PulsarClient.builder()
> > .eventLoopGroup(ioEventLoopGroup)
> > .internalExecutorProvider(sharedInternalExecutorProvider)
> > .externalExecutorProvider(sharedExternalExecutorProvider)
> > .scheduledExecutorProvider(sharedScheduledExecutorProvider)
> > ```
>
