> Maybe we can add the new API to the pulsar-client-original module? like we
> currently do.
Adding stable APIs to the pulsar-client-original module that
represents the internal implementations is not a good way IMO. I'm
wondering why we need to configure a Netty EventLoop? Is there any
alternative class in the Java standard library?
Thanks,
Yunze
On Wed, Dec 28, 2022 at 10:18 PM PengHui Li <peng...@apache.org> wrote:
>
> Hmmm
>
> I found that if we want to provide the public API(shared event loop) in the
> pulsar-client-api module.
> We need to add Netty dependency to the pulsar-client-api module.
> It's not a good idea to have Netty dependency in the pulsar-client-api
> module.
>
> Maybe we can add the new API to the pulsar-client-original module? like we
> currently do.
>
> ```
> PulsarClientImpl.builder()...
> ```
>
> We only changed the default value of the thread pool. Users still have the
> ability to change the threads
> that they want to assign. So if they want to use multiple client instances,
> they can also change the threads
> options.
>
> Thanks,
> Penghui
>
> On Tue, Dec 27, 2022 at 8:35 PM PengHui Li <peng...@apache.org> wrote:
>
> > Ah, got it.
> >
> > It's a good idea for users to be aware of all shared resources.
> > I will change the proposal as per your suggestion
> >
> > Thanks,
> > Penghui
> >
> > On Tue, Dec 27, 2022 at 7:11 PM Enrico Olivelli <eolive...@gmail.com>
> > wrote:
> >
> >> I generally support this proposal,
> >> this is a problem we have in the Proxy and I have seen it on
> >> applications that need to connect to multiple different tenants
> >> and they need different authentication parameters, so they have to
> >> create many PulsarClient instances.
> >>
> >> I have a suggestion:
> >>
> >> Exposing all the internals is a good idea for very advanced users,
> >> but I believe that we should provide some simpler support.
> >>
> >> We should have an API to allow sharing resources among multiple
> >> clients without entering the details.
> >>
> >> Interface PulsarClientGroup {
> >> ... put here all the sharable things in the current version...
> >> }
> >>
> >> PulsarClient client = ....newClient().
> >> .withSharedResources(pulsarClientGroup)
> >> ...
> >>
> >>
> >> I think that having a PulsarClientGroup is a good choice for future
> >> evolutions
> >> because the internal thread pools may change: removed/added/change the
> >> purpose.
> >>
> >> If we require users to deal with all the possible sharable resources
> >> then we have a few risks:
> >> - people can "forget" to share some resources
> >> - upgrading the client may lead to not taking into account some new
> >> "shareable" resources
> >>
> >> This is way I believe that we should provide an opaque
> >>
> >> Enrico
> >>
> >> Il giorno mar 27 dic 2022 alle ore 11:07 PengHui Li
> >> <peng...@apache.org> ha scritto:
> >> >
> >> > Hi all,
> >> >
> >> > As discussed at
> >> > https://lists.apache.org/thread/5obfm17g58n3dnbzyxg57vokgmwyp6hx
> >> > I have created this proposal to support shared thread pool across
> >> multiple
> >> > client instances
> >> > Here is the proposal link https://github.com/apache/pulsar/issues/19074
> >> >
> >> > Please help take a look, and look forward to your suggestions.
> >> >
> >> > Thanks,
> >> > Penghui
> >> >
> >> > --------------------------------------------------
> >> >
> >> > ### Motivation
> >> >
> >> > The Pulsar client mainly has three thread pools that cooperate with each
> >> > other to complete the message publishing and consumption of messages.
> >> >
> >> > - IO threads - Used for handling network packets from the broker
> >> > - Internal threads - Used for handling internal tasks such as moving the
> >> > received messages to the internal receiver queue and pulling out the
> >> > message from the receiver queue to return to users. And the Java client
> >> is
> >> > optimized by the lock-free principle; each consumer will use a pinned
> >> > internal thread to reduce the lock overhead.
> >> > - External threads - Used by the message listener
> >> >
> >> > All the above thread pools will be created automatically after a Pulsar
> >> > client instance has been created.
> >> >
> >> > But for some cases, users need to create multiple Pulsar client
> >> instances
> >> > in a JVM process due to different authentications or others. Each client
> >> > will have exclusive thread pools, which will cause unreasonable thread
> >> > usage, waste memory, and potential performance degradation.
> >> >
> >> > It is not a serious problem for previous releases with the default
> >> > configurations because the thread pool will only have 1 thread by
> >> default.
> >> > But it also doesn't make sense that we only have one thread for each
> >> thread
> >> > pool. We have discussed this part under this [thread](
> >> > https://lists.apache.org/thread/5obfm17g58n3dnbzyxg57vokgmwyp6hx)
> >> >
> >> > So this proposal will provide a new possibility for users that require
> >> > multiple Pulsar client instances in one JVM process to use the shared
> >> > thread pools across multiple Pulsar client instances.
> >> >
> >> > ### Goal
> >> >
> >> > Provide public API to use the shared thread pool across multiple Pulsar
> >> > client instances in one JVM process
> >> >
> >> > - IO threads
> >> > - Internal threads
> >> > - External threads
> >> >
> >> > BTW, we already have such an ability internally. It was just hidden for
> >> > users. Please take a look at #12037 and #13839 to get more details.
> >> >
> >> > ### API Changes
> >> >
> >> > The following APIs will be introduced to the Java Client when creating a
> >> > Client instance
> >> >
> >> > ```java
> >> > PulsarClient.builder()
> >> > .eventLoopGroup(ioEventLoopGroup)
> >> > .internalExecutorProvider(sharedInternalExecutorProvider)
> >> > .externalExecutorProvider(sharedExternalExecutorProvider)
> >> > .scheduledExecutorProvider(sharedScheduledExecutorProvider)
> >> > ```
> >>
> >
