Re: [VOTE] PIP-136: Sync Pulsar policies across multiple clouds

2022-07-12 Thread Rajan Dhabalia
Hi, After multiple discussions during Community meetings: I have updated PIP with the changes and approach we agreed on. This PIP allows any metadata-store to generate change-events using plugable MetadataSynchronizer to publish and handle change events. PIP also uses change-event topic as a WAL t

Re: [VOTE] PIP-136: Sync Pulsar policies across multiple clouds

2022-03-22 Thread PengHui Li
> This feature allows users to sync two separate clusters which are having independent global-zookeeper (metadata-store) instances. So, this feature will not be limited to sync a few fields of policies but think in terms of auto creation and syncing namespaces/tenants to entirely different metadata

Re: [VOTE] PIP-136: Sync Pulsar policies across multiple clouds

2022-03-22 Thread Rajan Dhabalia
>> Do we need to provide the ability for users to decide to replicate the ACLs and replication cluster or not? This feature allows users to sync two separate clusters which are having independent global-zookeeper (metadata-store) instances. So, this feature will not be limited to sync a few fields

Re: [VOTE] PIP-136: Sync Pulsar policies across multiple clouds

2022-03-21 Thread PengHui Li
Thanks for the explanation. > yes, local policies doesn't need to be replicate to other clusters and it will only replicate global policies which is shared across multiple clusters such tenant/namespace's identity-creation, ACLs, replication clusters, etc. As described in this blog [1], section "

Re: [VOTE] PIP-136: Sync Pulsar policies across multiple clouds

2022-03-21 Thread Rajan Dhabalia
>> If it contains namespace policy replication, There are some policies no need to replicate to another cluster yes, local policies doesn't need to be replicate to other clusters and it will only replicate global policies which is shared across multiple clusters such tenant/namespace's identity-cre

Re: [VOTE] PIP-136: Sync Pulsar policies across multiple clouds

2022-03-18 Thread PengHui Li
Hi Rajan, Thanks for the great proposal. Will all the namespace policies be replicated to the remote cluster? I noticed the PIP title mentioned policies, but looks like from the `MetadataChangeEvent`, no namespaces policies defined. If it contains namespace policy replication, There are some poli

Re: [VOTE] PIP-136: Sync Pulsar policies across multiple clouds

2022-03-18 Thread Joe F
+1 On Thu, Mar 17, 2022 at 12:07 PM Rajan Dhabalia wrote: > Hi, > > I would like to start VOTE on PIP-136: > https://github.com/apache/pulsar/issues/13728 > > Thanks, > Rajan > > On Tue, Feb 8, 2022 at 4:58 PM Rajan Dhabalia > wrote: > > > > > >> How do we designate the host broker? Is it manua

[VOTE] PIP-136: Sync Pulsar policies across multiple clouds

2022-03-17 Thread Rajan Dhabalia
Hi, I would like to start VOTE on PIP-136: https://github.com/apache/pulsar/issues/13728 Thanks, Rajan On Tue, Feb 8, 2022 at 4:58 PM Rajan Dhabalia wrote: > > >> How do we designate the host broker? Is it manual? How does it work > when the host broker is removed from the cluster? > No, it wi