Hey all,
I'd like to understand why I can't find the metrics for mm2 [1]. By using
jconsole, I can find the mbeans for kafka.connect, kafka.consumer and
kafka.producer but I can't find anything on kafka.connect.mirror. Attached
is the image of the jconsole listing the mbeans.
What am I doing wron
Nvm. This was an issue on my side.
On Mon, Sep 7, 2020 at 3:03 PM Samuel Cantero wrote:
> Hey all,
>
> I'd like to understand why I can't find the metrics for mm2 [1]. By using
> jconsole, I can find the mbeans for kafka.connect, kafka.consumer and
> kafka.producer but
Hello,
I'd like to know whether starting mirroring from latest is doable with
mirrormaker2. I tried setting the
target-cluster-alias.consumer.auto.offset.reset to latest but it doesn't
work (for an active/passive setup). Any help would be appreciated.
Best,
>
> If MM2 mirrors a topic that has been previously mirrored and the consumer
> offsets have preserved, `auto.offset.reset`` should not matter, because MM2
> always start mirroring from the last known offsets
>
> On 2020/09/10 23:03:16, Samuel Cantero wrote:
> > Hello,
&
onfig and see if they work
>
> On 2020/09/15 20:53:12, Samuel Cantero wrote:
> > I tried this but even the override does not work for me as I can still
> see
> > "earliest" on the ConsumerConfig properties. Did you manage actually to
> > make this work or are you just
will try to understand why it is failing for other properties now.
On Wed, Sep 16, 2020 at 12:20 PM Ning Zhang wrote:
> just a possibility that `source-cluster-alias.` does not align with the
> used in `clusters:`
>
> On 2020/09/16 14:24:39, Samuel Cantero wrote:
> > Definite
od luck and feel free to ping me if you want to try and discuss this.
>
> Iftach
>
>
> On Wed, Sep 16, 2020, 19:59 Samuel Cantero wrote:
>
> > So the problem is that this is hardcoded for 2.5.1 and 2.6:
> >
> >
> https://github.com/apache/kafka/blob/62abe01bee03
Hi all,
It looks like the offsets returned by the
RemoteClusterUtils.translateOffsets are always the same as the
source/primary cluster, i.e, there's no actual translation.
The upstream offsets are fine but the downstream are not (it is just the
same as the upstream one). After a quick look, it s
It seems the problem is just between kafka 0.10.x --> kafka 2.x. It is
working fine for kafka 2.x --> 2.x.
On Sat, Oct 3, 2020 at 4:50 PM Samuel Cantero wrote:
> Hi all,
>
> It looks like the offsets returned by the
> RemoteClusterUtils.translateOffsets are always the sa
oh no, actually I was wrong. We have the same problem.
On Mon, Oct 5, 2020 at 5:17 PM Samuel Cantero wrote:
> It seems the problem is just between kafka 0.10.x --> kafka 2.x. It is
> working fine for kafka 2.x --> 2.x.
>
> On Sat, Oct 3, 2020 at 4:50 PM Samuel Cantero
This is working for 2.5.1 but broken for trunk (or master).
On Mon, Oct 5, 2020 at 5:19 PM Samuel Cantero wrote:
> oh no, actually I was wrong. We have the same problem.
>
> On Mon, Oct 5, 2020 at 5:17 PM Samuel Cantero wrote:
>
>> It seems the problem is just between kafka
s not
working and hence offset translation was working. So this feature works
from "trunk" too.
Sharing it here in case someone comes across this in the future.
Best,
On Mon, Oct 5, 2020 at 6:02 PM Samuel Cantero wrote:
> This is working for 2.5.1 but broken for trunk (or master).
I've seen this before. I've found that consumer offsets sync does not work
with `consumer.auto.offset.reset:latest`. If you set this to earliest, then
it should work. One way to workaround the need to start from earliest is by
starting with latest and once mirroring is ongoing swap to earliest. Thi
>
> ... Alan
>
> On Wed, Mar 17, 2021 at 4:50 PM Samuel Cantero
> wrote:
>
> > I've seen this before. I've found that consumer offsets sync does not
> work
> > with `consumer.auto.offset.reset:latest`. If you set this to earliest,
> then
> > i
I've found that bug the hard way. FWIW I've migrated several clusters from
kafka 0.10 to kafka 2.x using mm2. So offsets sync work fine for kafka 0.10.
Best,
On Wed, Mar 17, 2021 at 6:43 PM Samuel Cantero wrote:
> No, what I meant is that offsets sync
You could check whether the producers on the source cluster are using some
compression algo. Use the same one on mm2 as currently it will decompress
and then write again to the target cluster uncompressed unless you
configure it.
On Sun, Feb 6, 2022 at 05:20 Kafka Life wrote:
> Dear Kafka Expert
Which kafka version are you running? Which replication factor is being
used? are producers using acks=all?
On Wed, Aug 9, 2023 at 9:14 AM Tiansu Yu
wrote:
> 1. We use cruise-control to actively balance the partitions across all
> brokers. So point 1 could be ruled out.
> 2. I am not sure how muc
17 matches
Mail list logo
