lhotari opened a new pull request, #24790:
URL: https://github.com/apache/pulsar/pull/24790
PIP: #19074
### Motivation
When many PulsarClient instances run in the same JVM, creating separate
thread pools, timers, event loops, and DNS resolvers for each wastes resources
and increases DNS load. Some applications also require separate client
connections while still sharing infrastructure (e.g., multiple Pulsar
Sinks/Sources in a Flink pipeline). This change introduces a way to share core
client resources across clients to significantly reduce memory usage, thread
counts, and DNS lookups, while keeping connection pools isolated per client.
Example usage where 1000 client instances are created:
```
@Test
public void testSharedResources() throws PulsarClientException {
PulsarClientSharedResources sharedResources =
PulsarClientSharedResources.builder().build();
List<PulsarClient> clients = new ArrayList<>();
for (int i = 0; i < 1000; i++) {
clients.add(PulsarClient.builder()
.serviceUrl("pulsar://localhost:6650")
.sharedResources(sharedResources)
.build());
}
for (PulsarClient client : clients) {
client.close();
}
sharedResources.close();
}
```
This PR builds upon #24784 changes for the DNS resolver and cache sharing.
### Modifications
* Introduce PulsarClientSharedResources and
PulsarClientSharedResourcesBuilder APIs to construct and manage shared
components (event loop, executors, scheduled executor, timer, lookup executor,
DNS resolver/cache).
* Add ClientBuilder.sharedResources(...) to attach a shared resources
instance to a client.
* Implement shared resources in the client: accept externally provided
executors/timer/event loop/DNS resolver and use factory helpers for defaults
when not shared.
### Documentation
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [x] `doc` <!-- Your PR contains doc changes. -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update
later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
