** Description changed: [Impact] - * If a user detaches a volume before unmount a race is hit (kernel + * If a user detaches a volume before unmount a race is hit (kernel stuck detaching the volume) and new volumes are not recognized - * Stefan Bader suggested the following patch set to resolve the issue: - * 0e34582699392d67910bd3919bc8fd9bedce115e - blkfront: fixes for 'xm block-detach ... --force' - * 5d7ed20e822ef82117a4d9928b030fa0247b789d - blkfront: don't access freed struct xenbus_device - * a66b5aebb7dc9e695dcb4b528906fd398b63f3d9 - blkfront: Clean up vbd release - * b70f5fa043b318659c936d8c3c696250e6528944 - blkfront: Lock blkfront_info when closing + * Stefan Bader suggested the following patch set to resolve the issue: + * 0e34582699392d67910bd3919bc8fd9bedce115e + blkfront: fixes for 'xm block-detach ... --force' + * 5d7ed20e822ef82117a4d9928b030fa0247b789d + blkfront: don't access freed struct xenbus_device + * a66b5aebb7dc9e695dcb4b528906fd398b63f3d9 + blkfront: Clean up vbd release + * b70f5fa043b318659c936d8c3c696250e6528944 + blkfront: Lock blkfront_info when closing [Test Case] The was originally seen with AMI ami-bffa6fd6[0] doing the following: 1. Launch an instance. 2. Attach a new volume to the instance using the API. 3. Mount the volume on the instance. 4. Detach the volume using the API. 5. Wait a few seconds (30 seconds? 60 seconds?). 6. Unmount the volume on the instance. 7. Wait for volume to become available. 8. Delete the volume once it is available and go to step 2. With about 135 iterations of these steps this problem can be reproduced. [0] That AMI is "lucid server release 20130124 instance-store amd64 us-east-1 ami-bffa6fd6 aki-88aa75e1 paravirtual" bffa6fd6 In the ubuntu instance with a self-compiled 2.6.32 kernel with these patches applied the behavior of the kernel is as expected even with the user error. [Regression Potential] - * tbd, looking to see if Stefan Bader can help here. + * Since EC2 images in Lucid are based on a separate branch, + we can rule out regressions on the generic/server images. + * Code changes are limited to the xen-blkfront driver and + to hot-adding/-removing disk images. Only Xen guests + using PV disks can be affected. + * So there is potential for introducing new bugs into the + process but then should be detected while testing for + verification. + * I would consider the risk of regressions as low. [Other Info] Root cause of the problem are: - 1. User error: The user first does force detach then unmount. - Correct usage is: first unmount then detach. - 2. This 2.6.32 kernel has a race bug in the blkfront driver. + 1. User error: The user first does force detach then unmount. + Correct usage is: first unmount then detach. + 2. This 2.6.32 kernel has a race bug in the blkfront driver. When the race bug is hit then the instance kernel is stuck in the detaching code and hence does not recognize the new attached volume. In the ubuntu instance with a self-compiled 2.6.32 kernel with these patches applied the behavior of the kernel is as expected even with the user error. $ lsb_release -rd Description: Ubuntu 10.04.4 LTS Release: 10.04 $ apt-cache policy linux-ec2 linux-ec2: - Installed: 2.6.32.350.31 - Candidate: 2.6.32.364.45 - Version table: - 2.6.32.364.45 0 - 500 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ lucid-updates/main Packages - 500 http://security.ubuntu.com/ubuntu/ lucid-security/main Packages - *** 2.6.32.350.31 0 - 100 /var/lib/dpkg/status - 2.6.32.305.6 0 - 500 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ lucid/main Packages + Installed: 2.6.32.350.31 + Candidate: 2.6.32.364.45 + Version table: + 2.6.32.364.45 0 + 500 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ lucid-updates/main Packages + 500 http://security.ubuntu.com/ubuntu/ lucid-security/main Packages + *** 2.6.32.350.31 0 + 100 /var/lib/dpkg/status + 2.6.32.305.6 0 + 500 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ lucid/main Packages
-- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1326870 Title: blkfront driver race, can not attach new volumes To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/1326870/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
