Qin
Partition problem is solved by passing "--new.producer true" option in
command line,  but adding auto.offset.rese=smallest config, every time i
restart the Mirror tool it copies from starting ends up having lot of
duplicate messages in destination cluster.
Could you please tell me how do i configure to make sure that destination
cluster is always insync with source cluster.

SunilKalva

On Sun, Mar 8, 2015 at 12:54 AM, Jiangjie Qin <j...@linkedin.com.invalid>
wrote:

> For data not showing up, you need to make sure mirror maker consumer
> auto.offset.reset is set to smallest, otherwise when you run mirror maker
> for the first time, all the pre-existing messages won¹t be consumed.
> For partition sticking, can you verify if your messages are keyed messages
> or not? If they are not keyed messages, can you check if you are using old
> producer or new producer? For old producer, the default behavior is
> sticking to one partition for 10 min and then move to the next partition.
> So if you wait for more than 10 min, you should see messages in two
> different partitions.
>
> Jiangjie (Becket) Qin
>
> On 3/7/15, 8:28 AM, "sunil kalva" <sambarc...@gmail.com> wrote:
>
> >And i also observed ,all the data is moving to one partition in
> >destination
> >cluster though i have multiple partitions for that topic in source and
> >destination clusters.
> >
> >SunilKalva
> >
> >On Sat, Mar 7, 2015 at 9:54 PM, sunil kalva <sambarc...@gmail.com> wrote:
> >
> >> I ran kafka mirroring tool after producing data in source cluster, and
> >> this is not copied to destination cluster. If i produce data after
> >>running
> >> tool those data are copied to destination cluster. Am i missing
> >>something ?
> >>
> >> --
> >> SunilKalva
> >>
> >
> >
> >
> >--
> >SunilKalva
>
>


-- 
SunilKalva

Reply via email to