The zookeeper connection URL with namespace can be zkhost1:123,zkhost2:123/newnamespace
The wiki is up to date for Kafka 0.7.2. There is no officially supported feature to do that sort of migration, I suggested one approach that I could think of :-) Thanks, Neha On Tuesday, March 19, 2013, Jason Rosenberg wrote: > I can do most of that I presume. > > It looks like to set up a separate namespace for zk, I can add /path at the > end of each node:port in my zkconnect string, e.g.: > zkhost1:123/newnamespace,zkhost2:123/newnamespace > right? > > For mirroring, there's some vague documentation here: > https://cwiki.apache.org/KAFKA/kafka-mirroring-mirrormaker.html > Is this the most up to date approach for 0.7.2? Set up a MirrorMaker > intermediate process that consumes from the old and produces to the new? > > I am not able to restart producers one by one (as there are many, on a > rather asynchronous update/restart cycle). But I can eventually get them > migrated over, etc. > > Jason > > On Tue, Mar 19, 2013 at 7:07 PM, Neha Narkhede > <neha.narkh...@gmail.com<javascript:;> > >wrote: > > > Can you do the following - > > > > 1. Start a mirror Kafka cluster with the new version on a separate > > zookeeper namespace. Configure this to mirror data from the existing > kafka > > cluster. > > 2. Move your consumers to pull data from the mirror > > 3. For each producer, one at a time, change the zookeeper namespace to > > point to the mirror and restart the producer. > > 4. Once the producers have moved to mirror cluster, shutdown mirroring > and > > old cluster. > > > > Thanks, > > Neha > > > > On Tuesday, March 19, 2013, Jason Rosenberg wrote: > > > > > I need to upgrade some kafka broker servers. So I need to seamlessly > > > migrate traffic from the old brokers to the new ones, without losing > > data, > > > and without stopping producers. I can temporarily stop consumers, etc. > > > > > > Is there a strategy for this? > > > > > > Also, because of the way we are embedding kafka in our framework, our > > > brokerId's are auto-generated (based on hostname, etc.), so I can't > > simply > > > copy over broker log files, etc., by transferring an old brokerId to a > > new > > > host. > > > > > > Is there a way to change the view of the cluster from the producer's > > > standpoint, without doing so from the consumers standpoint? That way, > > the > > > producers can start writing to the new brokers, while the consumers > drain > > > all data from the old brokers before switching to the new brokers. > > > > > > I don't actually care about ordering of messages, since the consumers > are > > > publishing them to a store that will index them properly based on > source > > > timestamp, etc. > > > > > > We are using zk for both producers and consumers connections. > > > > > > This is using 0.7.2. I assume in 0.8 it will be easier, since with > > > replication, you can phase in the new servers gradually, etc., no? > > > > > > Thanks, > > > > > > Jason > > > > > >
