I am in the process of updating to 0.9 and had another question. The docs at http://kafka.apache.org/documentation.html#upgrade say that to do a smooth upgrade from 0.8.2.X to 0.9 we can use the inter.broker.protocol.version config to control what protocol to use. After upgrading how can we tell that we are running the right inter broker protocol? Is there any jmx metric I could look at to confirm that I started the broker correctly? Or maybe something in the logs? I grepped for inter.broker.protocol.version in the logs and didn't find anything.
How about when I change the protocol version back to 0.9.0.0? I have followed the steps outlined in the guide on a test environment and my broker and client metrics look fine. But it would be great to get some kind of confirmation that each step (upgrade to 0.9 with 0.8.2.X protocol and then restart with 0.9.0.0 protocol) worked. Thanks, Rajiv On Tue, Dec 1, 2015 at 11:39 AM, Guozhang Wang <wangg...@gmail.com> wrote: > Old clients should work well with the new servers, while vice versa is not > generally true mainly because the new consumer client uses a few new > request types that only the new brokers can recognize. So the normal > upgrade path is server-first, clients later. > > Filed https://issues.apache.org/jira/browse/KAFKA-2923 to update the > upgrade doc to include it. > > Guozhang > > On Tue, Dec 1, 2015 at 11:14 AM, Rajiv Kurian <ra...@signalfx.com> wrote: > > > Thanks folks. Anywhere I can read about client compatibility i.e. old > > clients working with new servers and new clients working with old > servers? > > > > Thanks, > > Rajiv > > > > On Tue, Dec 1, 2015 at 10:54 AM, Jay Kreps <j...@confluent.io> wrote: > > > > > I think the point is that we should ideally try to cover all these in > the > > > "upgrade" notes. > > > > > > -Jay > > > > > > On Tue, Dec 1, 2015 at 10:37 AM, Aditya Auradkar < > aaurad...@linkedin.com > > > > > > wrote: > > > > > > > Rajiv, > > > > > > > > By default, the quota is unlimited until you decide to configure them > > > > explicitly. > > > > And yes, we did get rid of "replica.lag.max.messages", so that > > > > configuration will no longer apply. > > > > > > > > Aditya > > > > > > > > On Tue, Dec 1, 2015 at 10:24 AM, Todd Snyder <tsny...@blackberry.com > > > > > > wrote: > > > > > > > > > The quota page is here: > > > > > http://kafka.apache.org/documentation.html#design_quotas > > > > > > > > > > "By default, each unique client-id receives a fixed quota in > > bytes/sec > > > as > > > > > configured by the cluster (quota.producer.default, > > > > quota.consumer.default)" > > > > > > > > > > > > > > > I also noticed there's been a change in the replication > configuration > > > > > while reading: > > > > > > > > > > > > > > > http://www.confluent.io/blog/hands-free-kafka-replication-a-lesson-in-operational-simplicity/ > > > > > > > > > > It may not break anything, but it may impact how you decide to > > > configure > > > > > and monitor replication > > > > > > > > > > "Now there is only one value you need to configure on the server > > which > > > is > > > > > replica.lag.time.max.ms. The interpretation of this has changed to > > be > > > > the > > > > > time for which a replica has been out-of-sync with the leader. > Stuck > > or > > > > > failed replicas are detected the same way as before - if a replica > > > fails > > > > to > > > > > send a fetch request for longer than replica.lag.time.max.ms, it > is > > > > > considered dead and is removed from the ISR. The mechanism of > > detecting > > > > > slow replicas has changed - if a replica starts lagging behind the > > > leader > > > > > for longer than replica.lag.time.max.ms, then it is considered too > > > slow > > > > > and is removed from the ISR. So even if there is a spike in traffic > > and > > > > > large batches of messages are written on the leader, unless the > > replica > > > > > consistently remains behind the leader for replica.lag.time.max.ms > , > > it > > > > > will not shuffle in and out of the ISR." > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: Rajiv Kurian [mailto:ra...@signalfx.com] > > > > > Sent: Tuesday, December 01, 2015 11:57 > > > > > To: users@kafka.apache.org > > > > > Subject: Re: Any gotchas upgrading to 0.9? > > > > > > > > > > Also I remember reading (can't find now) something about default > > > traffic > > > > > quotas. I'd hope the default quotas are very large (infinite?) and > > not > > > > > small so that compatibility is maintained. It would be very > > unfortunate > > > > if > > > > > some of our traffic was throttled because of the upgrade because of > > > magic > > > > > defaults. For example we have a certain cluster dedicated to > serving > > a > > > > > single important topic and we'd hate for it to be throttled because > > of > > > > > incorrect defaults. > > > > > > > > > > Thanks, > > > > > Rajiv > > > > > > > > > > On Tue, Dec 1, 2015 at 8:54 AM, Rajiv Kurian <ra...@signalfx.com> > > > wrote: > > > > > > > > > > > I saw the upgrade path documentation at > > > > > > http://kafka.apache.org/documentation.html and that kind of > > answers > > > > (1). > > > > > > Not sure if there is anything about client compatibility though. > > > > > > > > > > > > > > > > > > On Tue, Dec 1, 2015 at 8:51 AM, Rajiv Kurian <ra...@signalfx.com > > > > > > wrote: > > > > > > > > > > > >> I plan to upgrade both the server and clients to 0.9. Had a few > > > > > questions > > > > > >> before I went ahead with the upgrade: > > > > > >> > > > > > >> 1. Do all brokers need to be on 0.9? Currently we are running > > 0.8.2. > > > > > We'd > > > > > >> ideally like to convert only a few brokers to 0.9 and only if we > > > don't > > > > > see > > > > > >> problems convert the rest. > > > > > >> > > > > > >> 2. Is it possible to run Kafka 0.9 clients (specifically the > > > consumer) > > > > > >> with Kafka 0.8.2 brokers? > > > > > >> > > > > > >> Any link to the upgrade path would be really useful. > > > > > >> > > > > > >> Thanks, > > > > > >> Rajiv > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > -- Guozhang >
