Very interesting. Overall I support this proposal
A couple of questions:
- What about monitoring the status of the migration?
- Should we block all maintenance operations on the "blue" cluster ?
like deleting/creating stuff
- Should we stop ledger trimming and offloading ?
- What about authentication/authorization between the two clusters ?
should we provide a dedicated set of credentials for the "migration"?
thanks
Enrico
Il giorno mer 13 lug 2022 alle ore 10:55 Asaf Mesika
<asaf.mes...@gmail.com> ha scritto:
>
> Few questions
>
> "CompletableFuture<Position> asyncMigrate();"
> Does this method only change the status of the managed ledger?
>
> "message ManagedLedgerInfo {
>
> // Flag to check if topic is terminated and migrated to different cluster
> optional bool migrated = 4;
>
> }"
>
> This flag then is only changed to true when it has finished migration: i.e.
> no new messages were written, all existing consumers finished reading all
> messages and disconnected and the topic can now be deleted?
>
> "Broker sends topic migration message to client so, producer/consumer at
> client side can handle redirection accordingly"
>
> For producers, the message will be sent the moment the status of the topic
> has changed, so all messages from there on will be written to the new
> cluster?
> For consumers, the message will be sent when there are no more messages to
> read?
>
>
>
> On Tue, Jul 12, 2022 at 8:23 PM Rajan Dhabalia <rdhaba...@apache.org> wrote:
>
> > Hi,
> >
> > We have created PIP-184 which helps users to perform cluster migration with
> > Apache Pulsar. Cluster migration or Blue-Green cluster deployment is one of
> > the proven solutions to migrate live traffic from one cluster to another.
> > One of the examples is applications running on Kubernetes sometimes require
> > a Kubernetes cluster upgrade which can cause downtime for the entire
> > application during a Kubernetes cluster upgrade. Blue-green deployment is
> > an application release model that gradually transfers user traffic from a
> > previous version of an app or microservice to a nearly identical new
> > release—both of which are running in production.
> >
> > The old version can be called the blue environment while the new version
> > can be known as the green environment. Once production traffic is fully
> > transferred from blue to green, blue can standby in case of rollback or be
> > pulled from production and updated to become the template upon which the
> > next update is made. We need such capability in Apache pulsar to migrate
> > live traffic from the blue cluster to the green cluster so, eventually, the
> > entire traffic moves from the blue cluster to the green cluster without
> > causing downtime for the topics.
> >
> > PIP: https://github.com/apache/pulsar/issues/16551
> >
> > Thanks,
> > Rajan
> >
