On Tue, Jan 04, 2011 at 07:32:06PM +0100, Olaf van der Spek wrote: > On Tue, Jan 4, 2011 at 7:29 PM, Steve Langasek <vor...@debian.org> wrote: > > On Tue, Jan 04, 2011 at 06:51:13PM +0100, Olaf van der Spek wrote: > >> On Tue, Jan 4, 2011 at 6:49 PM, Sven Joachim <svenj...@gmx.de> wrote: > >> > Maybe we're talking at cross-purposes here; I was speaking about the > >> > case of turning a directory into a symlink on upgrades, which cannot > >> > safely be done while there are still files under it. > > > >> > Thinking more about it, this cannot be done even if all packages move > >> > their files away from /usr, since an unknown amount of stuff exists > >> > under /usr/local, so /usr can ever only be made a symlink to / for new > >> > installs (or by the sysadmin). > > > >> I think it's about /usr/lib being a symlink to /lib. > > > > No, it isn't. > > Then what is it about? > I thought the problem was having both /lib and /usr/lib.
Well, that's the issue at hand. The reason I mentioned this is because I believe that the / and /usr separation is a case where we should stop to consider the "bigger picture" rather than just the immediate problem. Solving that would solve the /usr/lib and /lib issue as a natural consequence. > Linking /usr to / would also solve that but is more complex than > linking /usr/lib to /lib. Only minimally; the same issues apply to unifying both. A bit of background can be found in this thread from 2009: http://lists.debian.org/debian-devel/2009/05/msg00075.html This is a long thread, but there's a lot of useful detail in it, the most important bits I've summarised below. Note that this is concerning not allowing /usr as a separate filesystem, but eliminating /usr entirely is a logical consequence of it and so is mostly relevant here. Historically, we have always had a separate / and /usr, with the possibility of /usr being a separately-mounted filesystem. So, the points to consider are • what are the benefits of the separation? • what are the costs of the separation? • do the benefits and costs of the separation as originally conceived apply today in 2011? I was previously a proponent of keeping /usr separate, but after spending some time considering it I've revised my opinion of the matter. I previously thought it had a useful purpose in some cases; I'm now of the opinion that it's essentially pointless. Most of the historical reasons for keeping it separate are today no longer relevant; they are either superseded by better methods, or are not compatible with modern practices. To provide some context, I'll run through a few of the use cases and try to provide some insight into the rationale behind them, and some opinion on whether it's actually useful. Most of these were discussed in the above thread. 1) /usr is shared by many machines over NFS This is a "traditional" use of a separate /usr. It may have even been common back when SVR3 was popular and disc space was at a premium, and modern package managers were absent. But, it's completely insane to do this today. Pros: • Saves disc space by sharing one filesystem amongst many hosts • Allows centralised management of software updates; updates only need doing once for all hosts Cons: • Debian uses dpkg for package management; you can't separate / and /usr and expect it to work since packages need to update both locations. If you have a / per host and a shared /usr, a disaster is inevitable. • /bin, /sbin, /lib are not shared, so you need an alternative mechanism to propagate changes to the hosts; and /etc is even more complex. Also, how do you run dpkg maintainer scripts on each host? • Debian does not support a *shared* /usr over NFS. It supports mounting a *non-shared* /usr over NFS, but that's not the same thing. If you're doing this, you're doing it wrong. You should be NFS mounting the root filesystem in its entirety or else dpkg will break. I use debian-live to manage such images, and NFS mount the shared root (containing /usr) with a writable unionfs overlay to allow host-specific configuration and customisation. It's robust and supported. 2) /usr is mounted read-only for security and safety Mounting /usr read-only is common practice; I even do this myself with apt-get configured to remount read-write when changes are made and then remount read-only afterwards. Pros: • You have read-write / and read-only /usr which prevents changes to /usr during normal system operation. • Allows mounting with nodev Cons: • Does not protect /bin, /sbin, /lib etc. • Can be done on read-write / with read-only bind mounts • Can be done with read-only / • With udev, nodev can be used on / If /usr is eliminated, then we can just mount / read-only instead; it's arguably even safer. Quite a lot of work has gone into having a read-only / such as /lib/init/rw. It works today, though a few small issues remain (/etc/mtab, LVM backup/cache/archive). 3) A minimal root filesystem makes recovery easier Having / without /usr and other filesystems mounted makes it possible to troubleshoot and recover from problems such as filesystem corruption. Pros: • /etc will be recoverable if /usr gets corrupted Cons: • The system is unusable without /usr mounted; do we really care about the small chance that one filesystem will be corrupted over the other. i.e. does having them separate really reduce the probability that we will lose /etc in any meaningful way? • You also need /var (for dpkg status etc.) for recovery • Given that / is typically read-write and /usr might be read-only, the chances are that / will be corrupted rather than /usr unless you have hardware failure, in which case both will likely be screwed I've occasionally used single user mode when LVM problems or hardware changes prevented filesystems being mounted etc. But it's been more common to use the initramfs shell or a rescue CD. Overall, I can see that this mitigates recovery, but I'm not convinced that it's giving that much extra robustness. I'll be keeping backups for recovery/reinstallation rather than playing the probability game. 3) Different backup policies for / and /usr Pros: • Alleged performance gains Cons: • You don't need them on separate filesystems for separate backup policies I don't buy this one. While you might have a different backup policy for /etc and /var, there's no reason to treat any dpkg-managed location such as /bin, /lib and /usr/* differently from each other since they exist as a managed whole. 4) / is encrypted and /usr is unencrypted Pros: • Performance Cons: • Do you really need the entire / encrypted, or just specific bits? This looks like it's for encrypting things in /etc. I'm not sure that this solution is necessarily the best one. The / vs /usr distinction is one of convenience rather than actually encrypting only the bits actually required. 5) I have /usr on LVM and / on a partition This is a fairly common setup; I used it myself in the past. Pros: • It is possible to mount the root filesystem with no LVM in case there are LVM problems Cons: • / is of fixed size; you can't online resize it Nowadays this is a non-issue. I've had my root filesystems on LVM for years now; with initramfs and separate /boot; there's no need for these workarounds and recovery on LVM vs a normal partition is just as easy. A rescue CD can deal with both just as effectively. If you're using LVM, you should have your / on LVM, which would allow it to be dynamically resized as for all other partitions. 6) Libraries are separated in /lib and /usr/lib Programs in /bin and /sbin use /lib; programs in /usr/bin and /usr/sbin use /lib and /usr/lib. Programs on / must work prior to /usr being mounted. Libraries on / must only depend upon other libraries in /. Cons: • It's difficult to draw the line of what should be in / and what should be in /usr • Special care needs to be taken before /usr is mounted since important pieces such as locales are not available This is the problem initially brought up here. As we increase the complexity of what is required prior to mounting /usr such as having NSS modules with dependencies on other libraries, those other libraries need moving to / as well. The same applies to crypto support as mentioned initially. If / and /usr are unified, the problems introduced by having them separate are immediately resolved: everything becomes available from / being mounted except /var. 7) A separate /usr is required to boot the system Many years ago, a minimal root filesystem was required to bring the system up to a certain level of operational readiness in order to mount /usr and other filesystems. Examples included: • NFS mounts which required networking and portmap/rpc • RAID requiring array reconstruction • LVM which required discovery of available volumes • Loading additional kernel modules to access hardware • And other things For quite some time now, Debian uses an initramfs to boot. This does all the setup for you. This means you can mount / from NFS or from LVM or MD (or both stacked) etc. There's no requirement for a separate /usr for booting. 8) Filling up /usr won't fill up / Installing software won't use up all the space on the root filesystem Nice idea in priciple… but you can still fill up your root filesystem with kernel images and initrds if /boot isn't separate. This is a preventative measure, but not a particularly effective one. And if filling up / occurs, then it's not exactly hard to reverse the procedure. 9) / and /usr are using different filesystem types This is a rather hard one to take seriously. While there are times when you would choose one filesystem type over another for various reasons, using different types for / and /usr makes little sense given their similar requirements. It provides zero benefit. I think many instances of confusion and misunderstanding over the / and /usr separation, particularly with regard to NFS mounts, but also covering read-only mounts and recovery are due to not fully considering the implications of a modern package manager on the traditional UNIX filesystem hierarchy. We can consider that there are just two different types of directory in the file system: • those managed by dpkg • those containing user data which are not under dpkg control All locations managed by dpkg must be considered a unified whole; it does not make any sense to share one part and not another. They must be updated together or else the system will be left in a broken and inconsistent state. A separate /usr is no longer required to boot the system now we have initramfs. In consequence, there's no reason to have /bin separate from /usr/bin, /lib from /usr/lib etc. It makes sense to have /var separate (writable). It makes no sense to have /usr separate. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `- GPG Public Key: 0x25BFB848 Please GPG sign your mail.
signature.asc
Description: Digital signature
