That's exactly what partition reassignment does. It may take a bit of time
for the reassignment to complete.
For example, if you initiate the process to change the replica assignment
from {1, 2, 3} to {4,5,6}, the values in the assigned replica (AR) and
leader/isr path in ZK may go through the following transition.
AR leader/isr
1. {1,2,3} 1/{1,2,3}
2. {1,2,3,4,5,6} 1/{1,2,3}
3. {1,2,3,4,5,6} 1/{1,2,3,4,5,6}
4. {1,2,3,4,5,6} 4/{1,2,3,4,5,6}
5. {1,2,3,4,5,6} 4/{4,5,6}
6. {4,5,6} 4/{4,5,6}
Going from step 2 to step 3 can take some time since we have to wait for
the new replicas to fully catch up. The reassignment process completes when
you are in step 6. The reassignment tool has a mode that you can use to
check the status.
Thanks,
Jun
On Wed, May 21, 2014 at 10:09 PM, David Birdsong
<david.birds...@gmail.com>wrote:
> On Wed, May 21, 2014 at 9:06 PM, Jun Rao <jun...@gmail.com> wrote:
>
> > Expansion can be done by following
> > http://kafka.apache.org/documentation.html#basic_ops_cluster_expansion
> >
> > If you just want to free up a server, you can stop the broker there and
> > start a broker using the same broker id on a new server. Data should be
> > automatically replicated to the new server.
> >
> >
> The thing is, I don't want to fully free up the server, I just want it to
> stop hosting all the topic,partition sets it originally did.
>
> Here's the flow:
>
> broker_1 hosts topic,{1,2,3,4}
> over time load for the partitions overwhelms broker_1
>
> spin up broker_2
> migrate topic,{3,4} to broker_2 successfully
>
> broker_1 is only useful if partitions topic,{3,4} are dropped by broker_1
>
> how does one get broker_1 to disassociate, drop, forget, evict...(not sure
> the verb) topic,{3,4} and let broker_2 own it.
>
> Thanks,
> >
> > Jun
> >
> >
> > On Wed, May 21, 2014 at 3:23 PM, David Birdsong <
> david.birds...@gmail.com
> > >wrote:
> >
> > > Any suggestions? I'm kind of in a bind in that I don't understand how
> to
> > > grow the cluster when more capacity is needed--which happens to be
> right
> > > now for me.
> > >
> > > The only thing I can think that might work is to create new brokers,
> > > cherry-pick topic/partition pairs and move them, then turn off the old
> > ones
> > > and forever retire their IDs freeing up my old hardware to come back
> > online
> > > as a new kafka broker ID.
> > >
> > >
> > > On Wed, May 21, 2014 at 9:16 AM, David Birdsong <
> > david.birds...@gmail.com
> > > >wrote:
> > >
> > > >
> > > >
> > > >
> > > > On Wed, May 21, 2014 at 9:11 AM, David Birdsong <
> > > david.birds...@gmail.com>wrote:
> > > >
> > > >> Here's the reassignment json and current "describe" output:
> > > >> https://gist.github.com/davidbirdsong/32cd0c4f49496a6a32e5
> > > >>
> > > >>
> > > >> In my re-assignment json, I tried to re-assign to 2 when the repl is
> > set
> > > >> to 3. Once I noticed the the completely new node "133" had appeared
> in
> > > the
> > > >> ISR, I tried stopping 224, wiping kafka completely and then brought
> > 224
> > > >> back up as 224 again. It promptly replicated the topic, but never
> > joined
> > > >> the ISR.
> > > >>
> > > >
> > > > to be clear, i don't want it to join the ISR. i'm curious how to make
> > 224
> > > > forget about the partitions for that topic since i have other plans
> for
> > > it.
> > > >
> > > >
> > > >>
> > > >> How does one move a replica? This is exactly what I'm trying to do.
> > > >>
> > > >> My pattern is a common one. I started with a set of 3 kafka brokers.
> > The
> > > >> load and space is overwhelming them. I'm trying to add new brokers
> and
> > > >> spread the partitions to new nodes while removing some of the
> > > partitions on
> > > >> the old nodes so as to make room. It's the latter that I don't get
> how
> > > to
> > > >> do.
> > > >>
> > > >> I've conflated two issues here mostly due to needing to get this
> > cluster
> > > >> stable again.
> > > >> - reduce replication
> > > >> - remove a partition from a broker, ie. remove the replica
> > > >>
> > > >> they're very distinct actions, but both would help me in the moment
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> On Wed, May 21, 2014 at 8:56 AM, Jun Rao <jun...@gmail.com> wrote:
> > > >>
> > > >>> During the re-assignment, did you move the replica off the old
> > broker?
> > > >>>
> > > >>> Thanks,
> > > >>>
> > > >>> Jun
> > > >>>
> > > >>>
> > > >>> On Wed, May 21, 2014 at 8:21 AM, David Birdsong <
> > > >>> david.birds...@gmail.com>wrote:
> > > >>>
> > > >>> > I did that and so now the topic has 4 replicas for a repl count
> of
> > 3,
> > > >>> but
> > > >>> > only the 'new' replicas exist in the ISR.
> > > >>> >
> > > >>> > The old broker that I want to clear disk space and generally free
> > up
> > > >>> > resources has fully synced a topic that I want to disassociate
> from
> > > it.
> > > >>> >
> > > >>> > Is there a way to do this?
> > > >>> >
> > > >>> >
> > > >>> > On Wed, May 21, 2014 at 7:48 AM, Jun Rao <jun...@gmail.com>
> wrote:
> > > >>> >
> > > >>> > > We don't have an exact tool for doing this. You may be able to
> do
> > > >>> that
> > > >>> > > through
> > > >>> > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
> http://kafka.apache.org/documentation.html#basic_ops_increase_replication_factorby
> > > >>> > > specifying fewer replicas.
> > > >>> > >
> > > >>> > > Thanks,
> > > >>> > >
> > > >>> > > Jun
> > > >>> > >
> > > >>> > >
> > > >>> > > On Wed, May 21, 2014 at 1:23 AM, David Birdsong <
> > > >>> > david.birds...@gmail.com
> > > >>> > > >wrote:
> > > >>> > >
> > > >>> > > > Is there a way to reduce the replication count? I'm trying to
> > > >>> spread
> > > >>> > > > existing partitions across more brokers, but it's hard to
> > decomm
> > > a
> > > >>> > > broker.
> > > >>> > > > Reducing repl count would suffice for now.
> > > >>> > > >
> > > >>> > > > Any tips?
> > > >>> > > >
> > > >>> > > > I'm running a mix of 0.8.1.1 and 0.8.1 (I'm upgrading now.)
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > > >>
> > > >>
> > > >
> > >
> >
>
