Hello fellow developers, I have bad news about /usr-move. The primary mitigation (M18) for aliased diversions (P3) as implemented by me in quite a few packages is broken. This was brought to my attention by independent reports of Emil Södergren and Colin Watson on live-build and debian-installer-utils respectively. I now have a better understanding of dpkg-divert (when it is faced with aliasing) and need to redo most patches (probably about 20). Affected packages shall receive updated patches during January. I'm sorry for the inconvenience caused.
If your have a lot of time and are interested in the underlying technical details, I invite you to continue reading. Otherwise, consider doing something more pleasant. Fundamentally, I assumed that dpkg-divert --rename would only move a file if it was considered installed according to the dpkg database. What really happens is that --rename moves a file as long as it exists. In many cases that I tested, this would happen to work by chance, but important corner cases would be wrong. We start with an aliased file being diverted. M18 says that the diversion should be duplicated to the canonical location. Unfortunately, we cannot translate the diversion target in the same way. Say we move /bin/foo to /usr/bin/foo in a package upgrade. dpkg will first put all the new files into temporary locations, then move them all into place (including /usr/bin/foo) and then unlink any removed files (including /bin/foo). Due to a sanity check, it notices that /bin/foo was just unpacked and skips its deletion. This sanity check is not in effect when a file is diverted. If we were to alias the diversion targets, the diverted file would be reliably lost. This is why all of the duplicated diversions use a .usr-is-merged suffix. This brings another consequence. When diverting, we must rename the file depending on whether it is aliased or canonical, but that's not what --rename does. No matter how we order the dpkg-divert calls it will wrongly rename either the aliased or the canonical case. And that's what we've seen in debian-installer-utils and live-build. As a result, I now propose using dpkg-divert with --no-rename for duplicated diversions and doing the move manually consulting dpkg -S to figure out where to move to. Beware that dpkg-divert is normally idempotent and skips a rename when a file already is diverted, so as we implement the rename externally, it must properly be conditional to the diversion being newly added to retain idempotency. Since I was failing to get this right earlier, I figured I could as well write some tests. As a result, I'm attaching a shell script. It constructs a few demo packages and attempts a hopefully exhaustive combination of using them and dpkg-divert by writing its experiments to the working directory. Running it should be fairly boring. If you happen to review it, I appreciate a response. The newly proposed logic becomes fairly elaborate. As there is no place that could share it and we want to get rid of it fairly soon, I intend to copy the code into the relevant users. A slight simplification can be achieved if the diverting package depends on a version of the diverted package that has canonicalized the diverted file. Then we may use --rename on the canonical path and --no-rename on the aliased path. As a result, adding the diversion in preinst may wrongly rename the file to the canonical diversion target, but the dependency ensures that it'll be clobbered with an updated version. Once the dependency is unpacked, this error is papered over. codesearching DEP17.*M18 (and a few other patterns) suggest the following packages needing an update: * bfh-metapackages * clonezilla * cryptsetup-nuke-password (and cryptsetup) * debian-installer-utils * isc-dhcp * molly-guard * live-build * open-infrastructure-system-tools * opensysusers * piuparts * progress-linux-metapackages * systemd * zutils (and gzip) I've not reviewed them yet. debian-installer-utils has a fixed MR and live-build definitely is broken. Please expect me to follow up on each package within January. Helmut
