That doesn't explain why they show up sometimes, but not all of
the time.
There are 3 devices in play here.
* The backing device, let's say /dev/vda; this is where we want
to store the data.
* The caching device, let's say /dev/vdb; this holds the cache.
* The bcache device; this only appears when a backing device is
detected; via bcache-probe which reads the header on the disk
and indicates it's a bcache device.
Now, there are a few sequences that we see:
1) make-bcache -b /dev/vda -c /deb/vdb will create a bcache0
with the cache and backing dev at the same time and emit the
CACHED_UUID change event which runs *after*
60-persistent-storage, and adds the bcache/by-uuid links
2) After a reboot (or if you tear down a bcache0 by stopping them
in sysfs) we have several scenarios where the 2 real block
devices are found:
2a) order: backing, bcacheN, caching
In this scenario, vda will run through
60-persistent-storage.rules and blkid will discover
ID_FS_TYPE=bcache, and generates only by-path, by-id links
for the device, 69-bcache.rules will run bcache-register
/dev/vda which informs the kernel that a backing device has
been found.
If the cache device has not yet been handled by udev (we
don't know if a cache device exists, bcache can work
without one) then the kernel will create a bcacheN
device, but without the cache device being present, it will
*NOT* emit a CHANGE event with CACHED_UUID value.
Now when the cache device (vdb) is going through,
60-persistent-storage.rules, blkid will discover the same
settings, TYPE=bcache, generate by-path, by-id symlinks.
Then 69-bcache.rules runs, triggers a bcache-register on vdb
registering the cache device in the kernel and because
the superblock on the backing device specifies the
cache device UUID it used the kernel can bind the cache
device into the correct bcacheN device.
A mainline kernel (no sauce patch) does not emit a CHANGE
event with CACHED_UUID value. Our sauce patch changes
the kernel to emit the CHANGE event with CACHED_UUID
whenever we bind a cache device to an existing bcacheN
precisely so the bcache/by-* links get generated.
2b) order: cache, backing, bcacheN
Here, when vdb is plugged in, the cache device is registered
in the kernel, but without a backing device, no bcacheN device
is created yet.
Then the backing is probed, as in 2a; the bcache-register command
will register it with the kernel, and create a bcache0, and
since the cache device is present, mainline kernels (and ours)
will emit the CHANGE event with CACHED_UUID
With this in-mind; it appears to me that in the focal case
we're seeing the bcache driver CHANGE event with CACHED_UUID
and then additional change events from the block layer around
the bcache0 device (but not emitted from the bcache driver code)
So I'd like to understand why ... I expect those events to occur
but not before the driver change event.
And maybe that's the race here
The happy path:
1. udev processing CHANGE on backing device, it's bcache so we register it
2. kernel bcache driver creates a bcache0 block device (udev ADD event)
3. udev processing CHANGE on bcache0 device
4. udev processing CHANGE on cache device, it's bcache so we register it
5. kernel bcache driver emits CHANGE event with CACHED_UUID
6. udev processing CHANGE on bcache0 device with CACHED_UUID
The bad path
1. udev processing CHANGE on cache device, it's bcache so we register it
2. udev processing CHANGE on backing device, it's bcache so we register it
3. kernel bcache driver creates a bcache0 block device (udev ADD event)
4. kernel bcache driver emits CHANGE event with CACHED_UUID
5. udev processing CHANGE on bcache0 device with CACHED_UUID
6. udev processing CHANGE on bcache0 device without CACHED_UUID
if the bcache change event occurs
before the block layer change events, then the database does
get clobbered ... but we've an incomplete trace; we should
see pairs of events, one from KERNEL, and then one from UDEV.
We should definitely get a complete capture for this bug.
w.r.t the fix; I've a few concerns:
1) bcache-keep-symlinks relies on /dev/bcache/by-* links to
exist, if they are removed, then the keep won't work.
2) The removal of the /dev/bcache/by-uuid/ happens during
processing of 60-persistent-storage.rules, which will remove the
link before bcache-keep-symlinks can run (it's in
69-bcache.rules)
3) since we cannot rely on the existing links, instead we should
read and parse the bcache superblock devices, much like the one
I put together (and you reminded me I wrote)
https://github.com/g2p/bcache-tools/pull/29/files
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux-signed in Ubuntu.
https://bugs.launchpad.net/bugs/1861941
Title:
bcache by-uuid links disappear after mounting bcache0
Status in bcache-tools package in Ubuntu:
Fix Released
Status in linux package in Ubuntu:
Invalid
Status in linux-signed package in Ubuntu:
Invalid
Status in systemd package in Ubuntu:
Invalid
Status in bcache-tools source package in Focal:
New
Status in linux source package in Focal:
Invalid
Status in linux-signed source package in Focal:
New
Status in systemd source package in Focal:
Invalid
Status in bcache-tools source package in Groovy:
Fix Released
Status in linux source package in Groovy:
Invalid
Status in linux-signed source package in Groovy:
Invalid
Status in systemd source package in Groovy:
Invalid
Bug description:
1.
root@ubuntu:~# lsb_release -rd
Description: Ubuntu Focal Fossa (development branch)
Release: 20.04
2.
root@ubuntu:~# lsb_release -rd
Description: Ubuntu Focal Fossa (development branch)
Release: 20.04
root@ubuntu:~# apt-cache policy linux-image-virtual
linux-image-virtual:
Installed: 5.4.0.12.15
Candidate: 5.4.0.12.15
Version table:
*** 5.4.0.12.15 500
500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages
100 /var/lib/dpkg/status
root@ubuntu:~# apt-cache policy linux-image-5.4.0-12-generic
linux-image-5.4.0-12-generic:
Installed: 5.4.0-12.15
Candidate: 5.4.0-12.15
Version table:
*** 5.4.0-12.15 500
500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages
100 /var/lib/dpkg/status
3. mount /dev/bcache0 && ls -al /dev/bcache/by-uuid/
+ ls -al /dev/bcache/by-uuid/
total 0
drwxr-xr-x 2 root root 60 Feb 4 23:31 .
drwxr-xr-x 3 root root 60 Feb 4 23:31 ..
lrwxrwxrwx 1 root root 13 Feb 4 23:31 abdfd1f6-44ce-4266-91db-24667b9ae51a
-> ../../bcache0
4.
root@ubuntu:~# ls -al /dev/bcache/by-uuid
ls: cannot access '/dev/bcache/by-uuid': No such file or directory
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.4.0-12-generic 5.4.0-12.15
ProcVersionSignature: Ubuntu 5.4.0-12.15-generic 5.4.8
Uname: Linux 5.4.0-12-generic x86_64
ApportVersion: 2.20.11-0ubuntu16
Architecture: amd64
Date: Tue Feb 4 23:31:52 2020
ProcEnviron:
TERM=xterm-256color
PATH=(custom, no user)
LANG=C.UTF-8
SHELL=/bin/bash
SourcePackage: linux-signed-5.4
UpgradeStatus: No upgrade log present (probably fresh install)
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/bcache-tools/+bug/1861941/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
