Hi Todd , Steps to reproduce : 1) Configure n>1 ,say 3 name dirs 2) Upgrade Namenode - After upgrade is over for 1st dir , stop/kill the namenode 3) Start namenode and write some files (delta) & then stop 4) Rollback namenode Now , after (4) , namenode will start normally , but it will contain the delta , since the unrollbacked 2nd dir will be the latest one (with the greatest checkpointtime ) & during savenamespace during loadFSImage() , the 2nd dir image will be saved to all the 3 dirs .. The system could 've gone back to the state before rollback - a FAILED Rolback which i think would 've been fine rather than the current behaviour .. I'am using 20.1 , but I verified in latest trunk also .. -- Sreehari
HUAWEI TECHNOLOGIES CO.,LTD. huawei_logo Solitaire Domlur Bangalore www.huawei.com ---------------------------------------------------------------------------- --------------------------------------------------------- This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! _____ From: Todd Lipcon [mailto:t...@cloudera.com] Sent: Monday, May 30, 2011 9:56 PM To: hdfs-dev@hadoop.apache.org; sreehar...@huawei.com Subject: Re: Expectation during Rollback after a partially failed Upgrade .. Should DELTA be visible ? Hi Sreehari, Which versions are you using for this test? In step 1 below, how does the upgrade fail? Can you provide specific steps to reproduce? Smells like a bug to me, if I understand the scenario correctly. -Todd On Mon, May 30, 2011 at 5:29 AM, Sreehari G <sreehar...@huawei.com> wrote: Hi all , What is the expectation in the following scenario : Namenode has 3 name dirs configured .. 1) Namenode upgrade starts - Upgrade fails after 1st directory is upgraded (2nd and 3rd dir is left unchanged ..) 2) Namenode starts 3) Namenode shutdown and rollbacked Are the new changes done meant to be visible ? With current implementation , after a rollback new changes are visible .. Is this expected ?? As per my observation , since Namenode is saving the latest image dir(the upgraded 1st dir since checkpointtime is incremented during upgrade for this dir) will be loaded and saved to all dirs during loadfsimage .. But if a ROLLBACK is done , the 1st dir will be rolled back (the older copy becomes current and its checkpointtime is now LESS than other dirs ..) and others left behind since they dont contain previous .. Now during loadfsimage , the 2nd dir will be selected since it has the highest checkpoint time and saved to all dirs (including 1st ) .. Now due to this , the new changes b/w UPGRADE and ROLLBACK DELTA present in 2nd dir gets reflected even after ROLLBACK .. Is this expected ? I ask this since : 1) this is not the case with a SUCCESSFULL Upgrade/Rollback (New changes are lost after rollback).. & 2) In Hadoop-702 (DFS Upgrade Proposal) , FSStateTransition7.htm says { r0. if previous directory does not exist then fail rollback } , but it continues in the current implementation .. Regards , Sreehari HUAWEI TECHNOLOGIES CO.,LTD. huawei_logo Solitaire Domlur Bangalore www.huawei.com ---------------------------------------------------------------------------- --------------------------------------------------------- This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -- Todd Lipcon Software Engineer, Cloudera
