No problem. Delete the reassign_partitions znode. Then do the controller again.
The controller znode gets instantly re-created but with a different broker as the controller. If you do a get /controller you can see the values change -Clark Sent from my iPhone > On May 17, 2015, at 11:48 AM, Chris Neal <cwn...@gmail.com> wrote: > > Gah. Sorry for the spamming replies. > > The reassign_partitions znode is also still there, and still empty....For > what that's worth. :) > Back to thinking that deleting the reassign_partitions znode is the right > way to go. > > Chris > >> On Sun, May 17, 2015 at 1:46 PM, Chris Neal <cwn...@gmail.com> wrote: >> >> Ok, I might have lied. Timezone got me. I do see this in the Kafka logs, >> so maybe it did work? >> >> [2015-05-17 14:36:57,924] INFO 0 successfully elected as leader >> (kafka.server.ZookeeperLeaderElector) >> [2015-05-17 14:36:57,924] INFO 0 successfully elected as leader >> (kafka.server.ZookeeperLeaderElector) >> [2015-05-17 14:36:58,975] INFO New leader is 0 >> (kafka.server.ZookeeperLeaderElector$LeaderChangeListener) >> [2015-05-17 14:36:58,975] INFO New leader is 0 >> (kafka.server.ZookeeperLeaderElector$LeaderChangeListener) >> >> Chris >> >>> On Sun, May 17, 2015 at 1:43 PM, Chris Neal <cwn...@gmail.com> wrote: >>> >>> Thank you again. :) >>> >>> I ran the delete, but I'm not sure if it did anything: >>> >>> root> >>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh >>> -server myhost.mydomain.com:2181 delete /controller >>> Connecting to myhost.mydomain.com:2181 >>> >>> WATCHER:: >>> >>> WatchedEvent state:SyncConnected type:None path:null >>> root> >>> >>> I followed that up immediately by an "ls /", and the controller znode was >>> there. Either still there, or had already been re-created. There was >>> nothing in the Kafka logs indicating that anything had happened either, so >>> I'm thinking maybe I need to try and delete the znode partition manually? >>> >>> Or, should I have seen the controller znode disappear for a time? >>> >>> Very much appreciate your help! >>> Chris >>> >>>> On Sun, May 17, 2015 at 1:33 PM, Clark Haskins <cl...@kafka.guru> wrote: >>>> >>>> Yeah it's empty. So I would try and force a new controller election to >>>> see if that cleans up the znode. >>>> >>>> Delete /controller >>>> >>>> If that doesn't work you will have to delete the reassign_partitions >>>> znode manually >>>> >>>> -Clark >>>> >>>> Sent from my iPhone >>>> >>>>> On May 17, 2015, at 11:16 AM, Chris Neal <cwn...@gmail.com> wrote: >>>>> >>>>> Sorry :) >>>>> Here you go: >>>>> >>>>> root> >>>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh >>>>> -server myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181 >>>> get >>>>> /admin/reassign_partitions >>>>> Connecting to myhost.mydomain.com <http://myhost.mydomain.com:2181/ >>>>> :2181 >>>>> <snip> >>>>> WATCHER:: >>>>> >>>>> WatchedEvent state:SyncConnected type:None path:null >>>>> {"version":1,"partitions":[]} >>>>> cZxid = 0x6001afde9 >>>>> ctime = Wed Jan 21 18:37:40 UTC 2015 >>>>> mZxid = 0x6001afde9 >>>>> mtime = Wed Jan 21 18:37:40 UTC 2015 >>>>> pZxid = 0x6001afde9 >>>>> cversion = 0 >>>>> dataVersion = 0 >>>>> aclVersion = 0 >>>>> ephemeralOwner = 0x0 >>>>> dataLength = 29 >>>>> numChildren = 0 >>>>> >>>>> Thank you! >>>>> Chris >>>>> >>>>>> On Sun, May 17, 2015 at 12:58 PM, Clark Haskins <cl...@kafka.guru> >>>> wrote: >>>>>> >>>>>> Do a get /admin/reassign_partitions >>>>>> >>>>>> -Clark >>>>>> >>>>>> Sent from my iPhone >>>>>> >>>>>>> On May 17, 2015, at 10:20 AM, Chris Neal <cwn...@gmail.com> wrote: >>>>>>> >>>>>>> Sure thing :) >>>>>>> Hopefully I did this right. Somewhat of a Zookeeper noob. >>>>>>> >>>>>>> root> >>>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh >>>>>>> -server myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181 >>>> ls >>>>>>> /admin/reassign_partitions >>>>>>> Connecting to myhost.mydomain.com <http://myhost.mydomain.com:2181/ >>>>>>> :2181 >>>>>>> <snip> >>>>>>> WATCHER:: >>>>>>> >>>>>>> WatchedEvent state:SyncConnected type:None path:null >>>>>>> [] >>>>>>> >>>>>>> Hope that is helpful :) >>>>>>> If this is not what you were asking for, please just let me know. >>>>>>> Thank you! >>>>>>> Chris >>>>>>> >>>>>>>> On Sun, May 17, 2015 at 12:17 PM, Clark Haskins <cl...@kafka.guru> >>>>>> wrote: >>>>>>>> >>>>>>>> The reassign_partitions znode is the important one. Please paste the >>>>>>>> contents of it. That node should only exist while there is a >>>>>> reassignment >>>>>>>> in progress. >>>>>>>> >>>>>>>> You can probably fix this up by forcing a new controller to come >>>> online >>>>>> by >>>>>>>> deleting /controller >>>>>>>> >>>>>>>> -Clark >>>>>>>> >>>>>>>> Sent from my iPhone >>>>>>>> >>>>>>>>> On May 17, 2015, at 10:14 AM, Chris Neal <cwn...@gmail.com> wrote: >>>>>>>>> >>>>>>>>> Hi Clark, >>>>>>>>> >>>>>>>>> Thank you for your reply! I do see that znode under /admin: >>>>>>>>> >>>>>>>>> root> >>>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh >>>>>>>>> -server myhost.mydomain.com:2181 ls /admin >>>>>>>>> Connecting to myhost.mydomain.com:2181 >>>>>>>>> <snip> >>>>>>>>> WATCHER:: >>>>>>>>> >>>>>>>>> WatchedEvent state:SyncConnected type:None path:null >>>>>>>>> [reassign_partitions, delete_topics] >>>>>>>>> >>>>>>>>> I'm not sure what this tells me though :) >>>>>>>>> Again, thanks for your time. >>>>>>>>> Chris >>>>>>>>> >>>>>>>>>> On Sun, May 17, 2015 at 12:20 AM, Clark Haskins <cl...@kafka.guru >>>>> >>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Does the partition reassignment znode exist under /admin in >>>> zookeeper? >>>>>>>>>> >>>>>>>>>> -Clark >>>>>>>>>> >>>>>>>>>> Sent from my iPhone >>>>>>>>>> >>>>>>>>>>> On May 16, 2015, at 7:16 PM, Chris Neal <cwn...@gmail.com> >>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Sorry for bumping my own thread. :S Just wanted to get it in >>>> front >>>>>> of >>>>>>>>>> some >>>>>>>>>>> eyes again! >>>>>>>>>>> >>>>>>>>>>> Thanks for your time and help. >>>>>>>>>>> Chris >>>>>>>>>>> >>>>>>>>>>>> On Thu, May 14, 2015 at 10:24 AM, Chris Neal <cwn...@gmail.com> >>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi All, >>>>>>>>>>>> >>>>>>>>>>>> I am running: kafka_2.10-0.8.1.1, and when I run the >>>>>>>>>>>> reassign-partitions.sh script, I get this: >>>>>>>>>>>> >>>>>>>>>>>> Partitions reassignment failed due to Partition reassignment >>>>>> currently >>>>>>>>>> in >>>>>>>>>>>> progress for Map(). Aborting operation >>>>>>>>>>>> kafka.common.AdminCommandFailedException: Partition reassignment >>>>>>>>>> currently >>>>>>>>>>>> in progress for Map(). Aborting operation >>>>>>>>>>>> at >>>> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:204) >>>>>>>>>>>> at >>>> kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:124) >>>>>>>>>>>> at >>>> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:49) >>>>>>>>>>>> at >>>> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala) >>>>>>>>>>>> >>>>>>>>>>>> I'm running it as: >>>>>>>>>>>> /opt/kafka/kafka/bin/kafka-reassign-partitions.sh --zookeeper >>>>>>>> $ZOOKEEPER >>>>>>>>>>>> --reassignment-json-file ./$1.json --execute >>>>>>>>>>>> >>>>>>>>>>>> and my json file looks like: >>>>>>>>>>>> {"version":1, >>>>>>>>>>>> "partitions":[ >>>>>>>>>>>> {"topic":"perf_mytopic_raw", "partition":0,"replicas":[0,1]}, >>>>>>>>>>>> {"topic":"perf_mytopic_raw","partition":1,"replicas":[0,1]}]} >>>>>>>>>>>> >>>>>>>>>>>> The empty Map() looks suspicious, and I found another post from >>>>>> about >>>>>>>> a >>>>>>>>>>>> year ago where it was thought that this might be a bug? I >>>> looked >>>>>>>>>> through >>>>>>>>>>>> JIRA and didn't see anything that looked like this issue was >>>> created >>>>>>>> or >>>>>>>>>>>> fixed, so I wanted to ask again about it. >>>>>>>>>>>> >>>>>>>>>>>> Any ideas why this might be failing? There is definitely not >>>>>> another >>>>>>>>>>>> reassignment in progress. :) >>>>>>>>>>>> >>>>>>>>>>>> Thanks very much for your time. >>>>>>>>>>>> Chris >>
