Rajiv,
Upgrading from 0.8.2.1 to 0.9.0.0 should also be fine. If you can reproduce
this issue in a test environment, that would be great. The following may be
helpful in figuring out the issue.
1. Use ack=1 instead of ack=0 will allow the producer to see the error code
when the send fails.
2. If
I was mistaken about the version. We were actually using 0.8.2.1 before
upgrading to 0.9.
On Thu, Dec 17, 2015 at 6:13 PM, Dana Powers wrote:
> I don't have much to add on this, but q: what is version 0.8.2.3? I thought
> the latest in 0.8 series was 0.8.2.2?
>
> -Dana
> On Dec 17, 2015 5:56 PM,
I don't have much to add on this, but q: what is version 0.8.2.3? I thought
the latest in 0.8 series was 0.8.2.2?
-Dana
On Dec 17, 2015 5:56 PM, "Rajiv Kurian" wrote:
> Yes we are in the process of upgrading to the new producers. But the
> problem seems deeper than a compatibility issue. We have
Yes we are in the process of upgrading to the new producers. But the
problem seems deeper than a compatibility issue. We have one environment
where the old producers work with the new 0.9 broker. Further when we
reverted our messed up 0.9 environment to 0.8.2.3 the problem with those
topics didn't
Yes, the new java producer is available in 0.8.2.x and we recommend people
use that.
Also, when those producers had the issue, were there any other things weird
in the broker (e.g., broker's ZK session expires)?
Thanks,
Jun
On Thu, Dec 17, 2015 at 2:37 PM, Rajiv Kurian wrote:
> I can't think
I can't think of anything special about the topics besides the clients
being very old (Java wrappers over Scala).
I do think it was using ack=0. But my guess is that the logging was done by
the Kafka producer thread. My application itself was not getting exceptions
from Kafka.
On Thu, Dec 17, 201
Hmm, anything special with those 3 topics? Also, the broker log shows that
the producer uses ack=0, which means the producer shouldn't get errors like
leader not found. Could you clarify on the ack used by the producer?
Thanks,
Jun
On Thu, Dec 17, 2015 at 12:41 PM, Rajiv Kurian wrote:
> The to
The topic which stopped working had clients that were only using the old
Java producer that is a wrapper over the Scala producer. Again it seemed to
work perfectly in another of our realms where we have the same topics, same
producers/consumers etc but with less traffic.
On Thu, Dec 17, 2015 at 12
Are you using the new java producer?
Thanks,
Jun
On Thu, Dec 17, 2015 at 9:58 AM, Rajiv Kurian wrote:
> Hi Jun,
> Answers inline:
>
> On Thu, Dec 17, 2015 at 9:41 AM, Jun Rao wrote:
>
> > Rajiv,
> >
> > Thanks for reporting this.
> >
> > 1. How did you verify that 3 of the topics are corrupte
Hi Jun,
Answers inline:
On Thu, Dec 17, 2015 at 9:41 AM, Jun Rao wrote:
> Rajiv,
>
> Thanks for reporting this.
>
> 1. How did you verify that 3 of the topics are corrupted? Did you use
> DumpLogSegments tool? Also, is there a simple way to reproduce the
> corruption?
>
No I did not. The only re
Rajiv,
Thanks for reporting this.
1. How did you verify that 3 of the topics are corrupted? Did you use
DumpLogSegments tool? Also, is there a simple way to reproduce the
corruption?
2. As Lance mentioned, if you are using snappy, make sure that you include
the right snappy jar (1.1.1.7).
3. For
Hey Rajiv,
Are you using snappy compression?
On Tue, Dec 15, 2015 at 12:52 PM, Rajiv Kurian wrote:
> We had to revert to 0.8.3 because three of our topics seem to have gotten
> corrupted during the upgrade. As soon as we did the upgrade producers to
> the three topics I mentioned stopped being
We had to revert to 0.8.3 because three of our topics seem to have gotten
corrupted during the upgrade. As soon as we did the upgrade producers to
the three topics I mentioned stopped being able to do writes. The clients
complained (occasionally) about leader not found exceptions. We restarted
our
Now that it has been a bit longer, the spikes I was seeing are gone but the
CPU and network in/out on the three brokers that were showing the spikes
are still much higher than before the upgrade. Their CPUs have increased
from around 1-2% to 12-20%. The network in on the same brokers has gone up
fr
I upgraded one of our Kafka clusters (9 nodes) from 0.8.2.3 to 0.9
following the instructions at
http://kafka.apache.org/documentation.html#upgrade
Most things seem to work fine based on our metrics. Something I noticed is
that the network out on 3 of the nodes goes up every 5-6 minutes. I see a
c
15 matches
Mail list logo
