Thanks for the fast response. I did a quick test and initial results look
promising. When I swapped in the patched version, CPU usage dropped from
~150% to ~65%. Still a bit higher than what I see with 0.8.1.1 but much
more reasonable.
I'll do more testing on Monday but wanted to get you some quic
Hello,
I recently upgraded a java producer from the 0.8.2-beta client library to
the 0.8.2. The producer is writing to a 0.8.1.1 kafka cluster. It was
running without issues with the 0.8.2-beta version, but when i switched to
the new one i get snappy exceptions on kafka logs.
Logs: http://paste.u
Mathias, Solon,
We did identify a CPU issue and patched it in
https://issues.apache.org/jira/browse/KAFKA-1952. Could you apply the patch
in the 0.8.2 branch and see if that addresses the issue?
Thanks,
Jun
On Fri, Feb 13, 2015 at 3:26 PM, Jay Kreps wrote:
> We can reproduce this issue, have
As Manikumar mentioned the code in 0.8.2 is not released and that class is
just a stub (it doesn't do anything yet). If you would like to try out the
new consumer you can try it on trunk. However be aware that it doesn't yet
do partition balancing among topics as that is pending server side work.
H
Hey all,
We found an issue in 0.8.2 that can lead to high CPU usage on brokers with
lots of partitions. We are working on a fix for this. You can track
progress here:
https://issues.apache.org/jira/browse/KAFKA-1952
I would recommend holding off on upgrading to 0.8.2 until we have a fix for
this
We can reproduce this issue, have a theory as to the cause, and are working
on a fix. Here is the ticket to track it:
https://issues.apache.org/jira/browse/KAFKA-1952
I would recommend people hold off on 0.8.2 upgrades until we have a handle
on this.
-Jay
On Fri, Feb 13, 2015 at 1:47 PM, Solon G
There are FetcherLagMetrics that you can take a look at. However, it
is probably easiest to just monitor MaxLag as that reports the maximum
of all the lag metrics.
On Fri, Feb 13, 2015 at 05:03:28PM +0800, tao xiao wrote:
> Hi team,
>
> Is there a metric that shows the consumer lag of a particula
Thanks for looking into that!
On Fri, Feb 13, 2015 at 05:31:39AM +, Jiangjie Qin wrote:
> I think this is the offset checker bug.
> The offset checker will
> 1. first check if the offset exists in offset topic on broker or not.
> 2. If it is on broker then it will just return that offset.
> 3.
The partitions nearly all have replication factor 2 (a few stray ones have
1), and our producers use request.required.acks=-1. However, I should note
there were hardly any messages being produced when I did the upgrade and
observed the high CPU load.
I should have time to do some profiling on Mond
I created KAFKA-1951 to improve the offset checker to let it print offsets
stored in both zookeeper and Kafka.
I don¹t think we have a way to wipe out offset topic in Kafka for a
consumer.
‹Jiangjie (Becket) Qin
On 2/12/15, 9:59 PM, "tao xiao" wrote:
>Thanks for the explanation. It there a way
Paul,
There is ongoing work to move to Kafka API instead of making
calls to zookeeper.
Here is the JIRA https://issues.apache.org/jira/browse/STORM-650 .
-Harsha
On Fri, Feb 13, 2015, at 01:02 PM, Paul Mackles wrote:
> I noticed that the standard Kafka storm spout gets topic met
I noticed that the standard Kafka storm spout gets topic metadata from
zookeeper (under "/brokers/topics/") instead of issuing MetadataRequests to one
of the brokers. Aside from possible encapsulation issues, are there any other
downsides to using ZK this way? Are there significant cases where Z
Mathias,
What's the ack mode you used in the producer? Could you share the command
you used to run kafka-producer-perf-test.sh?
Thanks,
Jun
On Thu, Feb 12, 2015 at 1:17 PM, Mathias Söderberg <
mathias.soederb...@gmail.com> wrote:
> Jun,
>
> Pardon the radio silence. I booted up a new broker, c
I'm checking into this on our side. The version we're working on jumping to
right now is not the 0.8.2 release version, but it is significantly ahead
of 0.8.1.1. We've got it deployed on one cluster and I'm making sure it's
balanced right now before I take a look at all the metrics. I'll fill in
mo
Solon,
What's the replication factor you used for those partitions? What's the
producer ack that you used? Also, could you do a bit of profiling on the
broker to see which methods used the most CPU?
Thanks,
Jun
On Thu, Feb 12, 2015 at 3:19 PM, Solon Gordon wrote:
> I saw a very similar jump i
Hi team,
Is there a metric that shows the consumer lag of a particular consumer
group? similar to what offset checker provides
--
Regards,
Tao
16 matches
Mail list logo
