Hi Pulsar community, I created a PIP to use the system topic to store the bundle load data in the load manager.
The proposal can be found: https://github.com/apache/pulsar/issues/15037 --------- ## Motivation Currently, Pulsar LoadManager is using Zookeeper to store the bundle load data, when we have many bundles, this might put a lot of pressure on Zookeeper. ## Goal This PIP proposes storing the load manager's bundle load data to a system topic and using TableView to read it. ## Implementation Since the bundle load data is stats data, it no needs a strong consistent guarantee. So we can use the system topic to store the load data and use TableView to read it. ### System topic client Add a new SystemTopicClient calls `LoadBalanceBundleDataSystemTopicClient` , the topic name is `persistent://pulsar/system/__load_balance_bundle_data`, and the key is the bundle name, value is BundleData. ```java public class LoadBalanceBundleDataSystemTopicClient extends SystemTopicClientBase<LoadBalanceBundleDataEvent> { // ... } ``` Add new Event calls `LoadBalanceBundleDataEvent` ``` @Data public class LoadBalanceBundleDataEvent { private String bundle; private BundleData bundleData; } ``` ### ModularLoadManagerImpl Add a TableView in `ModularLoadManagerImpl` to replace the bundle cache ``` private TableView<String, BundleData> bundlesTableView; ``` ## Compatibility This feature can have both backward and forward compatibility since the bundle data is stats data. --------- Thanks, Kai
