Hello,
Hope everyone is doing well. I was hoping to get some assistance with a
strange issue we're experiencing while using the MirrorMaker to pull data
down from an 8 node Kafka cluster in AWS into our data center. Both Kafka
clusters and the mirror are using version 0.8.1.1 with dedicated Zookee
;m not entirely sure
how the MirrorMaker cycles through it's topics/partitions and why it would
read very slowly or bypass reading from certain partitions.
Craig J. Swift
Software Engineer - Data Pipeline
ReturnPath Inc.
Work: 303-999-3220 Cell: 720-560-7038
On Wed, Sep 9, 2015 at 9:29 AM
;d need to be doing a capture all the
> time, and then figure out when MirrorMaker stops fetching from that broker,
> and then stare at a ton of data to find whatever protocol thing has
> happened -- all of which is ugly enough that I'd been reluctant to mention
> it until now).
&g
Hello,
We've done a variety of different mirror configurations in the past (we
mirror from AWS into several different data centers) including the first
one you describe. In fact for some high volume / large message topics we
actually found splitting it up and placing a dedicated mirror for the lar
+1
Craig J. Swift
Principal Software Engineer - Data Pipeline
ReturnPath Inc.
Work: 303-999-3220 Cell: 720-560-7038
On Thu, Jun 16, 2016 at 2:50 PM, Henry Cai
wrote:
> +1 for Lambda expression.
>
> On Thu, Jun 16, 2016 at 1:48 PM, Rajiv Kurian wrote:
>
> > +1
> >
> > On Thu, Jun 16, 2016 at 1:
Hello,
I was hoping someone might know this off the top of their head. I haven't
been able to find any documentation on it so I was curious if it was still
available. We're upgrading to Kafka 8.2 from 8.1 (ya, we're a bit behind)
and we used to have the following in jmxtrans that would give us mon
Hello,
Initially we had our setup with both ZK and Kafka on the same nodes as
well. Over time though this proved problematic as we both increased the
usage of Kafka and had to scale out the cluster. ZK specifically can be
very touchy if it gets IO bound - so when the Kafka cluster would get under
Hello,
We're in the process of upgrading several of our clusters to Kafka 10. I
was wondering if it's possible to use the Kafka 10 client code (old or new)
to read from a source Kafka 8 cluster and then use the new 10 producer to
write to a destination Kafka 10 cluster? I know there's a recommende
iewpage.action?pageId=27846330
>
> -David
>
> On 10/6/16, 1:32 PM, "Craig Swift"
> wrote:
>
> Hello,
>
> We're in the process of upgrading several of our clusters to Kafka 10.
> I
> was wondering if it's possible to use the Kafka 10 clien
Ok great - thanks for the clarification. Exactly what I needed. :)
Craig
On Thu, Oct 6, 2016 at 2:09 PM, Scott Reynolds wrote:
> you cannot use a k10 client with a k8 cluster. The protocol changed
>
> You CAN use a k8 client with a k10 cluster.
>
> On Thu, Oct 6, 2016 at 12:00
Hello,
I think we're misunderstanding the docs on some level and I need a little
clarification. We have the following setup:
1) 0.8.2 producer -> writing to Kafka 0.10.0.1 cluster w/ version 10
message format (source cluster).
2) 0.10.0.1 mirror using the 'new consumer' reading from the source cl
Hello,
Just to close this issue out. The 8 producer going to the 10 cluster was
the root issue. The mirror maker by default was unable to produce the
message to the destination cluster. The work around was to include a
MirrorMakerMessageHandler
that did nothing but repackage the message again. In
12 matches
Mail list logo
