Hi All,
What is the lightest way to check if the connection to Kafka is up or not?
Currently I'm having the producer attempt to send a message to a topic, and
then catching a TimeoutException and handling logic inside of the catch
statement. This doesn't seem like the right way to do this, though
Hi Ivan,
Sorry to take so long in getting back to you. We made our own changes to
the MirrorMaker before we saw Jiangjie Qin say that the trunk had exact
partition mapping. I think it's best to use the current trunk's MM as
opposed to our's because there's probably has more test coverage. Let me
What does a "clean shutdown" of the MM entail? So far I've just been using
Ctrl + C to send an interrupt to kill it.
Alex
On Sat, Mar 7, 2015 at 10:59 PM, Jiangjie Qin
wrote:
> If auto.offset.reset is set to smallest, it does not mean the consumer
> will always consume from the smallest. It me
Hello,
Note I am using the new 0.8.2 version of Kafka and so I'm using the new
KafkaProducer class.
I have a special type of message data that I need to push to every
partition in a topic. Can that be done with custom partitioner that
implements Partitioner when Partitioner expects you to return
that's fine to me , you can open a separate thread , But the original
> question when the consumerconnector got connected to a separate topic ,
> Whether KafkaStream will have all the information of the partitions for
> that corresponding topic , Please confirm
>
> Thanks
>
I am writing a custom producer that needs to know information about the
topic it's about to produce to. In particular it needs to know the number
of partitions on the topic. Is there some utility method that returns such
data? I am using scala v2.9.2 and kafka v8.2.0.
Alex
I was going to make a separate email thread for this question but this
thread's topic echoes what my own would have been.
How can I query a broker or zookeeper for the number of partitions in a
given topic? I'm trying to write a custom partitioner that sends a message
to every partition within a
l as this exact
mirroring process seems like a common use case, and it's not clear how one
should go about doing this.
Thanks Guozhang,
Alex Melville
On Sun, Feb 22, 2015 at 3:53 PM, Guozhang Wang wrote:
> Hi Alex,
>
> What I originally meant is that you probably need to ma
^^ I would really appreciate this as well. It's unclear how to get log4j
working with Kafka when you have no prior experience with log4j.
On Mon, Feb 23, 2015 at 4:39 AM, Scott Chapman wrote:
> Thanks. But we're using log4j. I tried setting the type to async but it
> generated a warning of no su
SimpleConsumer that iterates through the logs of source and target
partition, comparing them to each other as the iteration ensues, but it'd
be nice if there was an existing tool for doing this, or if could have some
guarantee that the MM will retain partition ordering by default.
Cheers,
Alex Melville
we're
inclined to choose the Java one).
Functionally are both the scala and java codebases the same?
Thanks in advance,
Alex Melville
behind the "[B cannot be cast to java.lang.String" error?
Thank you in advance Neha, Guozhang, and the rest of the Kafka community!
Alex Melville
On Fri, Dec 5, 2014 at 5:30 PM, Neha Narkhede wrote:
> Going back to your previous requirement of ensuring that the data in the
&
as no initial offset in zookeeper.
>
> You can have MM print more log by changing the log level in
> config/tools-log4j.properties
>
> On Sun, Feb 15, 2015 at 8:39 AM, Alex Melville
> wrote:
>
> > Hi Kafka'ers,
> >
> >
> > I am trying to get the Mi
Hi Kafka'ers,
I am trying to get the Mirrormaker working with two separate clusters, one
as the source and the other as the target. The topic I'm trying to copy
over exists on both the source and target clusters. Here are the relevant
entries in my consumer and producer properties files, which I'
7;s
> wireless interface. You can try with setting advertised.host.name=kafka's
> server hostname in the server.properties and run it again.
>
> On Sun, Feb 8, 2015 at 8:38 AM, Alex Melville wrote:
>
> > Howdy all,
> >
> > I recently upgraded to Kafka 0.8.2.0 and am
ception.
Any help here is appreciated.
Cheers,
Alex Melville
my
producer.properties config file?
On Sat, Dec 13, 2014 at 8:11 PM, Alex Melville wrote:
>
> Dear Kafka Devs,
>
>
>
Dear Kafka Devs,
ecify the partition id; so each producer
> will only send to a single partition.
>
> Guozhang
>
>
> On Tue, Nov 25, 2014 at 8:19 PM, Alex Melville
> wrote:
>
> > Howdy friends,
> >
> >
> > I'd like to mirror the topics on several clusters to a ce
target cluster, but I'm not 100% sure if I'll be able to verify my
guarantee if I do it this way. Can anyone here verify that partitions
copied over to the target cluster by the default Mirrormaker are an exact
copy of those on the source cluster?
Thanks in advance,
Alex Melville
the cause of the error.
>
> Thanks,
>
> Jun
>
> On Tue, Nov 4, 2014 at 11:25 PM, Alex Melville
> wrote:
>
> > Background:
> >
> > I have searched for a while online, and through the files located in the
> > kafka/logs directory, trying to fi
Background:
I have searched for a while online, and through the files located in the
kafka/logs directory, trying to find where kafka writes log output to in
order to debug the SimpleProducer I wrote. My producer is almost identical
to the simple producer located here
https://cwiki.apache.org/con
Hi Apache Community,
My company has the following use case. We have multiple geographically
disparate data centers each with their own Kafka cluster, and we want to
aggregate all of these center's data to one central Kafka cluster located
in a data center distinct from the rest using MirrorMaker.
23 matches
Mail list logo
