Hi Philip.
Noted and really appreciate for your inputs. There is no problem to
patch the code.
I just didn't want to be coupled with forked version of the storm-kafka
spout. Since my patch will not be in the main branch of the code.
Thanks
Oleg.
On Tue, Nov 19, 2013 at 9:44 PM, Philip O'Toole
https://www.google.com/search?q=jmx+to+graphite
On Tue, Nov 19, 2013 at 5:24 PM, David Montgomery wrote:
> Hi,
> On the Kafka cluster I would like like to get monitoring stats. I do not
> know java per this page.
>
> Monitoring
>
> Our monitoring is done though a centralized monitoring system
Hi,
On the Kafka cluster I would like like to get monitoring stats. I do not
know java per this page.
Monitoring
Our monitoring is done though a centralized monitoring system custom to
LinkedIn, but it keys off the JMX stats exposed from Kafka. To see what is
available the easiest thing is just
Are you sure? I think it gives the count of messages in all topics that have
been received by the broker since the broker process started. If I restart the
broker process, the counts all reset to 0 and then start incrementing again.
On Nov 19, 2013, at 12:56 PM, Monika Garg wrote:
> Hi,
> T
Ohh...that is the case.I never think that way that in a running cluster
there can be no active controller.Yes you are right,it will be a bug then.
On Nov 19, 2013 6:02 PM, "Neha Narkhede" wrote:
> Sure, but this is to alert if there is no active controller when all
> brokers are live, which point
Sure, but this is to alert if there is no active controller when all
brokers are live, which points to a bug.
-Neha
On Tue, Nov 19, 2013 at 9:53 AM, Monika Garg wrote:
> Thanks Neha,Now got your point completely.
>
> One more small thing that I want to understand:
>
> The case when mbean value
Thanks Neha,Now got your point completely.
One more small thing that I want to understand:
The case when mbean values for ActiveControllerCount
across all brokers in a cluster != 1,will be only if all the brokers of the
cluster are down and in that case it will not be possible to get the value
of
Hi,
The count in the Mbean
"kafka.server":type="BrokerTopicMetrics",name="AllTopicsMessagesInPerSec"
gives the total no of messages for all the topics on the broker.
On Tue, Nov 19, 2013 at 8:13 PM, Andrew Otto wrote:
> Would
>
> "kafka.server":type="BrokerTopicMetrics",name="AllTopicsMessagesI
I think the max 50Mbps is almost the disk bottleneck
My guess is IO is the bottle neck for kafka if you set to same type(async
without ack)
I got throughput at about 30Mb
Try to increase if you don't care about latency very much
log.flush.interval.messages=1
log.flush.interval.ms=3000
On Tue
Hi Neha,
I thought request.required.acks has a default value of 0. I have not
modified it and running with the same default value. At the same time what
is the max throughput expected in 0.8 ?
On Tue, Nov 19, 2013 at 8:43 PM, Wendy Bartlett <
wendy.bartl...@threattrack.com> wrote:
> Will Kafka
Will Kafka 09 be backward compatible with 08?
From: Neha Narkhede
Sent: Tuesday, November 19, 2013 9:27 AM
To: users@kafka.apache.org
Subject: Re: Producer reaches a max of 7Mbps
I went through the performance page where it can reach a speed of 50MBps.
I
Would
"kafka.server":type="BrokerTopicMetrics",name="AllTopicsMessagesInPerSec" count
be easier?
Also, correct me if I am wrong, but I believe that these count values are the
total number of messages seen for a topic (or all topics) since the Broker was
started, not the total number of message
I went through the performance page where it can reach a speed of 50MBps.
I think that number is true for 07, not 08. If you want higher producer
throughout in 08, you can set request . required.acks=0. Note that it means
that the producer does not receive server side acknowledgements if you use
You might find this useful -
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+mirroring+(MirrorMaker)
Thanks,
Neha
On Nov 18, 2013 8:32 PM, "David Montgomery"
wrote:
> Hi,
>
> My endpoint for kafka is in Europe where the a consumers gets the final
> messages. But I want to write to kafka
Like I mentioned above, it is due to ease of alerting -
" alert if the addition of the mbean values for ActiveControllerCount
across all brokers in a cluster != 1."
It can be done with boolean as well, we found the addition rule to be
easier.
Thanks,
Neha
On Nov 19, 2013 1:48 AM, "Monika Garg"
The Storm mailing list is probably a better place for this thread. If I
understand the issue, it is not a ZK issue, nor a Kafka config issue. We run
multiple topos draining the same topics all the time.
In any event, you just need to patch the Kafka Spout code to catch this
exception, and ask
Thanks for replying Neha.
Yes I got the same thing as u written:There should only be one controller
at any point of time in a Kafka
cluster. If that controller broker is bounced, then the controller moves to
another broker.
But my doubt is this value will fluctuate always between 0 and 1 ,so it
mus
This information helps me a lot.
Thank you very much, Monika.
Andreas Maier
Am 18.11.13 08:28 schrieb "Monika Garg" unter :
>Yes,in jconsole the Mbean
>"kafka.server":type="BrokerTopicMetrics",contains
>all the details for all the topics of each machine/broker.
>
>To sum up the number of messa
18 matches
Mail list logo
