2nd/restore issue  ( I think I need to solve the offsets topic issue before
I go with the scale up and down issue )

As you had indicated, I went ahead and created the offsets topic. The
status of the cluster  ( destination ) is thus

opic# Partitions# BrokersBrokers Spread %Brokers Skew %Brokers Leader Skew %#
ReplicasUnder Replicated %Leader SizeProducer Message/SecSummed Recent
Offsets
s8k.checkpoints.internal
<https://kafka-manager.bf2.tumblr.net/clusters/grete_test/topics/s8k.checkpoints.internal>
1 3 60 0 0 3 0 0.00 0
s8k.act_search_page
<https://kafka-manager.bf2.tumblr.net/clusters/grete_test/topics/s8k.act_search_page>
1 3 60 0 0 3 0 6675.30 4,166,842
s8k.act_reach
<https://kafka-manager.bf2.tumblr.net/clusters/grete_test/topics/s8k.act_reach>
1 3 60 0 0 3 0 20657.92 11,579,529
mm2-status.s8k.internal
<https://kafka-manager.bf2.tumblr.net/clusters/grete_test/topics/mm2-status.s8k.internal>
5 5 100 0 0 3 0 0.00 10
mm2-offsets.s8k_test.internal
<https://kafka-manager.bf2.tumblr.net/clusters/grete_test/topics/mm2-offsets.s8k_test.internal>
1 3 60 0 0 3 0 0.00 0
mm2-offsets.s8k.internal
<https://kafka-manager.bf2.tumblr.net/clusters/grete_test/topics/mm2-offsets.s8k.internal>
1 3 60 0 0 3 0 0.00 0
mm2-configs.s8k.internal
<https://kafka-manager.bf2.tumblr.net/clusters/grete_test/topics/mm2-configs.s8k.internal>
1 3 60 0 0 3 0 0.00 13

You can see . that we have the  5 ( I created bot the offsets, to be safe
for the below )

*clusters = s8k, s8k_test*



*s8k.bootstrap.servers = .....*

*s8k_test.bootstrap.servers = ......*


*# only allow replication dr1 -> dr2*

*s8k->s8k_test.enabled = true*

*s8k->s8k_test.topics = act_search_page|act_reach*

*s8k->s8k_test.emit.heartbeats.enabled = false*




*s8k_test->s8k.enabled = false*

*s8k_test->s8k.emit.heartbeats.enabled = false*

*s8k_test.replication.factor = 3*

*s8k.replication.factor = 3*

*offsets.storage.replication.factor = 3*

*replication.factor = 3*

*replication.policy.separator = .*

*tasks.max = 4*




What seems strange is that I do not have a single record in the offsets
topic.. Is that normal ?   I would imagine that without a record, there is
no way that a restore would happen.... And that is obvious when I restart
the mm2 instance... Find the screenshot attached. In essence the latency
avg lag is reset \when the mm2 instance is reset indicating no restore but
restart from EARLIEST... I must be missing some thing simple














On Sun, Oct 13, 2019 at 7:41 PM Ryanne Dolan <ryannedo...@gmail.com> wrote:

> Vishal, the first issue is easy: you must set tasks.max to something above
> 1 (the default) in order to achieve any parallelism. This property is
> passed along to the internal Connect workers. It's unfortunate that Connect
> is not smart enough to default this property to the number of workers. I
> suspect that will improve before long.
>
> For the second issue, is it possible you are missing the offsets topic? It
> should exist alongside the config and status topics. Connect should create
> this topic, but there are various reasons this can fail, e.g. if the
> replication factor is misconfigured. You can try creating this topic
> manually or changing offsets.storage.replication.factor.
>
> Ryanne
>
> On Sun, Oct 13, 2019, 5:13 PM Vishal Santoshi <vishal.santo...@gmail.com>
> wrote:
>
> > Using https://github.com/apache/kafka/tree/trunk/connect/mirror as a
> > guide,
> > I have build from source the origin/KIP-382 of
> > https://github.com/apache/kafka.git.
> >
> > I am seeing 2 issues
> >
> > * I brought up 2 processes on 2 different nodes ( they are actually pods
> on
> > k8s but that should not matter ). They share the mm2.properties file and
> > are replicating ( 1-way ) 3 topics with 8 partitions in total.  That
> seems
> > to be the way to create a standalone mm2 cluster. I do not however see(
> at
> > least the mbeans do not show ) any attempt to rebalance.
> >
> >
> https://github.com/apache/kafka/tree/trunk/connect/mirror#monitoring-an-mm2-process
> > mbeans
> > are all on a single node
> >
> >
> >
> > * I restart the processes on the 2 nodes ( hard stop ans start ). The
> > offsets for replication seem to be reset to the earliest, as if it is a
> > brand new mirroring. It is also obvious from the
> > "record-age-ms-avg|replication-latency-ms-avg"
> > which I track through the restart.
> >
> >
> >
> >
> > This implies that
> >
> >
> > 1. Load balancing by rebalancing is not working. I cannot scale up or
> down
> > by adding nodes to the mm2 cluster or removing them.
> >
> > 2. Restore on a mirror is not working. If the MM2 cluster is brought
> down,
> > it does not start mirroring from the last known state. I see the,
> > state/config topics etc created as expected..
> >
> >
> >
> >
> >
> > The mm2.properties is pretty mimimal
> >
> >
> > *clusters = a , b*
> >
> >
> >
> > *a.bootstrap.servers = k.....*
> >
> > *b.bootstrap.servers = k.....*
> >
> >
> > *# only allow replication dr1 -> dr2*
> >
> > *a->b.enabled = true*
> >
> > *a->b.topics = act_search_page*
> >
> > *a->b.emit.heartbeats.enabled = false*
> >
> >
> > *b->a..enabled = false*
> >
> > *b->a.emit.heartbeats.enabled = false*
> >
> >
> >
> >
> >
> >
> > What do you think is the issue ?
> >
> >
> > Thanks
> >
>

Reply via email to