Hi Xiangying
In my test, the topic-level replication works as expected when using a
different configuration metadata store but can not work as expected when
using the same configuration metadata store.
I have two questions:
- Do any exact users use the global config metadata store and still
need
Hi Penghui
>I'm sorry, I don't fully understand your point here. What is the "support
replication on message and topic level"?
>As I understand, are the `allowed-clusters` and `replication-clusters` more
concise options?
Pulsar support set replication-cluster for per message.
After this proposal
Hi Jiwei,
> I think we can rename this PIP to: *Introduce `allowed-clusters` and
`topic-policy-synchronized-clusters` to fully support replication on
message and topic level*
I'm sorry, I don't fully understand your point here. What is the "support
replication on message and topic level"?
As
As I understand, I think the topic is about the granularity and
flexibility of the resource hierarchy design.
- The namespace level granularity is good for most cases from the start.
But the business is changing, and the requirement for flexibility will be
engaged.
- Moving to another namespace f
Hi Jiwei
Great advice. Thanks for your suggestions and additions.
Thanks,
Xiangying
On Fri, Dec 15, 2023 at 9:41 AM guo jiwei wrote:
> Hi Xiangying,
>I think we can rename this PIP to: *Introduce `allowed-clusters` and
> `topic-policy-synchronized-clusters` to fully support replication
Hi Xiangying,
I think we can rename this PIP to: *Introduce `allowed-clusters` and
`topic-policy-synchronized-clusters` to fully support replication on
message and topic level*
Currently, we can set replication clusters on the message and topic
level, but the replication clusters should be
Hi Girish,
I'm very pleased that we have reached some consensus now. Pulsar already
supports geo-replication at the topic level, but the existing
implementation of this topic level replication does not match our
expectations.
At the moment, I can think of three directions to solve this problem:
Hello Xiangying,
On Thu, Dec 7, 2023 at 6:32 AM Xiangying Meng wrote:
> Hi Girish,
>
> What you are actually opposing is the implementation of true topic-level
> geo-replication. You believe that topics should be divided into different
> namespaces based on replication. Following this line of t
Hi Girish,
What you are actually opposing is the implementation of true topic-level
geo-replication. You believe that topics should be divided into different
namespaces based on replication. Following this line of thought, what we
should do is restrict the current topic-level replication settings,
Hello, replies inline.
On Wed, Dec 6, 2023 at 5:28 PM Xiangying Meng wrote:
> Hi Girish,
>
> Thank you for your explanation. Because Joe's email referenced the current
> implementation of Pulsar, I misunderstood him to be saying that this
> current implementation is not good.
>
> A possible use
Hi Girish,
>But the reason why you've raised this PIP is to bring down the actual
replication semantics at a topic level. Yes, namespace level still exists
as per your PIP as well, but is basically left only to be a "default in
case topic level is missing".
I'm afraid there's some misunderstandin
Hi Girish,
Thank you for your explanation. Because Joe's email referenced the current
implementation of Pulsar, I misunderstood him to be saying that this
current implementation is not good.
A possible use case is where there is one or a small number of topics in
the namespace that store importan
Girish,
Thank you for making my point much better than I did ..
-Joe
On Tue, Dec 5, 2023 at 1:45 AM Girish Sharma
wrote:
> Hello Xiangying,
>
> I believe what Joe here is referring to as "application design" is not the
> design of pulsar or namespace level replication but the design of your
>
Hello Xiangying,
I believe what Joe here is referring to as "application design" is not the
design of pulsar or namespace level replication but the design of your
application and the dependency that you've put on topic level replication.
In general, I am aligned with Joe from an application desig
Hi Joe,
You're correct. The initial design of the replication policy leaves room
for improvement. To address this, we aim to refine the cluster settings at
the namespace level in a way that won't impact the existing system. The
replication clusters should solely be used to establish full mesh
repl
>if users want to change the replication policy for
topic-n and do not change the replication policy of other topics, they need
to change all the topic policy under this namespace.
This PIP unfortunately flows from attempting to solve bad application
design
A namespace is supposed to represent
Hi Rajan and Girish,
Thanks for your reply. About the question you mentioned, there is some
information I want to share with you.
>If anyone wants to setup different replication clusters then either
>those topics can be created under different namespaces or defined at topic
>level policy.
>And use
Hi Xiangying,
Shouldn't the solution to the issue mentioned in #21564 [0] mostly be
around validating that topic level replication clusters are subset of
namespace level replication clusters?
It would be a completely compatible change as even today the case where a
topic has a cluster not defined
Hi,
I don't agree with your assumptions about allowed clusters and
replication-cluster definition. Allowed clusters defined at tenant level
will restrict tenants to create namespaces in regions/clusters where they
are not allowed. and replication clusters will help tenant to set up full
mesh repli
Hi, Pulsar Community
I drafted a proposal to make the configuration of clusters at the namespace
level clearer. This helps solve the problem of geo-replication not working
correctly at the topic level.
https://github.com/apache/pulsar/pull/21648
I'm looking forward to hearing from you.
BR
Xiang
20 matches
Mail list logo
