Hi Ismael,
I'm using bruce (https://github.com/ifwe/bruce) to send the produce
requests, with a RequiredAcks value of 1. Everything works fine when
all brokers are running 0.8.2.1. Also if I set up a new 0.9.0.0
cluster from scratch rather than trying to upgrade, everything works
fine. The problem only occurs after upgrading one broker in the
3-broker cluster.
The topic I am sending to has 8 partitions numbered 0-7. Doing
further experimentation I see that the ACK error 2 occurs only when
I send to partition 7. No problems occur when sending to partitions
0-6. If it helps I can send output from "kafka-topics.sh --describe"
as well as tcpdump output showing the produce requests and responses.
For comparison I tried using the 0.9.0.0 version of
kafka-console-producer.sh to send messages. With the default
RequiredAcks value of 0, it worked although I don't know which
partition it sent to. With a RequiredAcks value of 1 I get the
output shown below.
Thanks,
Dave
[2016-01-15 10:33:28,843] ERROR Uncaught error in kafka producer I/O
thread: (org.apache.kafka.clients.producer.internals.Sender)
org.apache.kafka.common.protocol.types.SchemaException: Error reading field
'throttle_time_ms': java.nio.BufferUnderflowException
at
org.apache.kafka.common.protocol.types.Schema.read(Schema.java:71)
at
org.apache.kafka.clients.NetworkClient.handleCompletedReceives(NetworkClient.java:464)
at
org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:279)
at
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:216)
at
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:128)
at java.lang.Thread.run(Thread.java:745)
On Fri, Jan 15, 2016 at 1:06 AM, Ismael Juma <ism...@juma.me.uk> wrote:
> Hi Dave,
>
> On Fri, Jan 15, 2016 at 2:04 AM, Dave Peterson <d...@academia.edu> wrote:
>
> > I was trying to upgrade an 0.8.2.1 broker cluster to 0.9.0.0 by following
> > the instructions here:
> >
> > http://kafka.apache.org/documentation.html#upgrade
> >
> > After upgrading one broker, with inter.broker.protocol.version=0.8.2.X
> > set, I get ACK error 2 (InvalidMessage) when I try to send produce
> > requests.
>
>
> I haven't seen other reports of this issue yet. Also, we have a system test
> that covers this scenario:
>
>
> https://github.com/apache/kafka/blob/trunk/tests/kafkatest/tests/upgrade_test.py
>
> Just to double-check, what is the version of the producer that you are
> using to send produce requests to the 0.9.0.0 broker when you get the
> error?
>
> Ismael
>
