Quoting Craig Sanders ([email protected]):
> І've had it happen to me on several occasions - both disks and NICs.
I of course believe you, and am reading the details you provide below
this with interest. Thank you.
> The most common cause in my experience wasn't adding or removing drives/NICs,
> it was:
>
> a) upgrading the kernel, which changed the order that devices were detected,
> resulting in eth0 and eth1 being swapped, or sda & sdb being swapped, etc.
This I can readily believe, and I _think_ I once observed it during a
(very) major kernel transition -- and I considered this small beer in
context, FWIW.
> and
>
> b) driver modules being loaded in a different order (same cause, different
> incarnation) - this could be partially solved by listing the modules you want
> loaded in /etc/modules, they'll load in the order listed (unless something
> else triggers them being loaded earlier).
FWIW, my own preference is to locally compile a kernel with needed
drivers monolithically included and not building unneeded ones at all.
(On non-server systems, I do the same except compile as modules drivers
I might reasonable expect to some day want but not initially.)
I would be interested to know the circumstances in which 'modules loaded
in a different order', specifically if any were _other than...
> Adding drives and controller cards and USB **also** changes the order....
This, again, is exactly what I would expect and spoke of upthread, and
what I would consider small beer.
That is, _of course_ adding/removing drives and controller cards may
change device order. When you do so, you expect that and expect to
update one or two relevant system rc files.
USB? Yes, indeed, notorious agent of chaos that it is -- which is one
of multiple reasons why you don't leave casual-use hotplugged
mass-storage devices plugged in during system reboots, and why I'd be
adverse to relying on USB-connected network interfaces if I had any
alternative at all.
So, to recap, unless you can (please!) detail instances where 'driver
modules loaded in a different order' _without_ the above obvious and
well-understood causative factors, I think you've just reiterated
exactly what I said upthread.
> e.g. on my my system, I have four Crucial SSDs plugged into the motherboard
> (my boot drives and root zpool), a variety of WD & Seagate drives plugged into
> a SAS controller card (my main storage and backup zpools). Also a USB card
> reader, and an android phone (which sits in its USB connected dock recharging
> and displaying a clock whenever I'm at home, also running a sip client to
> connect to my asterisk server over my local wifi).
>
> On most reboots, the SAS ports are detected first, then the USB card reader
> (which doesn't even have any cards inserted), then the phone, then finally my
> motherboard SATA ports. I can't remember the details, but this order changed
> when I made some changes to BIOS settings...motherboard ports used to come
> first, now they don't. I don't care enough to find out why or change it
> because it doesn't matter at all if i use UUIDs or LABELs or /dev/disk/by-*
USB _is_ a notorious agent of chaos. (My surmise, admittedly.) But you
might also indeed be getting the several HBA drivers loaded in variable
order.
I'll bet that the device node instability would vanish if you compile in
the drivers monolithically. That's what I'd try, anyway -- might put an
end to that nonsense, and good riddance.
> The SAS port drives aren't even detected in any predictable order. All of the
> 4TB ST4000DX drives (my "backup" zpool) are plugged into one SFF-8087 socket
> on the SAS card (which goes to one of my 4-drive hot swap bays), and the 1TB
> WDs and STs are plugged into the other (which goes into another 4-drive bay).
> You'd expect them to be detected in that order, but...nope.
But (and my apologies if you clarify this; I'm a bit pressed for time),
I'm betting that the devices within each _set_ of ports, the motherboard
SATA set, the PCI-E SAS set, and the set of any block devices on USB,
each are assigned devices contiguously. So, see above.
> If I happen to reboot without my phone plugged in to a USB port, then the
> Crucial drives would probably be /dev/sdm to /dev/sdp rather thn n to q.
As mentioned, I would always carefully avoid having a casual-use
hotpluggable (e.g., USB) mass-storage device plugged in _at all_ during
boot time. Your Mileage May Differ.{tm}
You might not want to do that, but this basically explains my experience
that shifting nodes, if at all, happens only very rarely during major
kernel transitions or hardware addition/removal. Making sure hotplug
mass storage is gone during reboots helps prevent that outcome.
Monolithically compiled drivers may help, too, but I don't _always_ do
it, yet I don't observe what you say happens, so I'm not sure it's
strictly necessary. If I -did- see that, though, monolithically
compiled drivers would be the first thing I'd try.
> I also have a second SAS controller connected to a third 4-drive hot swap bay
> for adding and removing replacement drives and temporary drives. I can't
> remember if I've ever rebooted with drives inserted in that but I have no idea
> whether they would be detected immediately after the first SAS controller or
> at some other time.
Well, there you go talking about adding/removing drives, again -- and of
course that changes device nodes. That's what updating /etc/fstab is
for.
> So, when the kernel devs say that you can't rely on the order of naming for
> drives and other devices, I believe them.
What kernel devs?
Surely you aren't talking about the Freedesktop.org weenies.
I of course agree that device nodes for drives and other devices can
change, but the question was: Under which cirumstances? What you've
just described is pretty much exactly the situation I detailed upthread.
> and, yeah, this is an unusual setup for a home system. It's not all that
> unusual for anyone running a file server for a business or other organisation,
> or anyone who doesn't want to pay for a ridiculously overpriced NAS box when
> they can DIY with linux's built-in features.
I would never recommend for a business as file server with simultaneous
use of motherboard SATA ports, a PCI-E SAS card, and USB things on an
ongoing basis. That seems like poor component selection, IMVAO. [0]
> > And ifrename is cool.
>
> It was cool. I installed it on every machine for several years. Then it became
> unnecessary when the same capability (renaming NIC interfaces according MAC
> address) was standard in udev.
Or, to put it a different way, udev becomes unnecessary the moment you
remember ifrename.
> Given a choice between using a standard feature that's in every linux system
> (well, possibly excluding some embedded linux devices) and using a relatively
> obscure, "non-standard" tool that does the same thing, the decision to switch
> to using udev for that was easy.
MS-Windows is 'standard', too. ;->
Personally, I rather like being in charge of my own software. In fact,
I rather insist, thanks-very-much-I'm-sure.
> udev also has the advantage over ifrename of being useful for a lot more
> device-related stuff than just NIC device renaming.
It's a floor wax _and_ a dessert-topping! ;-> [1]
> PNIN often results in ugly NIC device names, but that's easily fixed with
> udev rules...
Or ifrename. Or other features that nail device node assignment to
MAC addres, as in RHEL/CentOS.
> devtmpfs doesn't solve the device order or naming problem.
No, but it goes a long way towards eliminating the alleged necessity of
udev, which I am pretty sure is what motivated Torvalds and co. to
introduce it. IIRC, it was after that notorious incident when Sievers
attempted to strongarm kdbus into the kernel so that the systemd/dbus
people could overwhelm the kernel with messge traffic.
devtmpfs, among other things, was a statement that 'Actually, it turns
out we don't need your code to recognise hardware and autoload firmware
BLOBs, so pray don't motivate us to make even more of what you do
irrelevant.'
> > https://wiki.gentoo.org/wiki/Mdev
> > https://github.com/slashbeast/mdev-like-a-boss
>
> whether it's called udev or mdev or some other clone of udev, it still does
> the same thing.
No, you are mistaken. In _no way_ is mdev a clone of udev. Not hardly.
Thank Ghu it is _not_ the least bit like that, because that would have
made it nearly pointless.
mdev is a greatly less baroque, more modestly scoped, more manageable
design, with a greatly smaller attack surface, and generally far, far
less to go wrong. It's not for everyone, but what is?
I started losing interest in udev at a rapid pitch the day I found that
the system no longer permitted me to use mknod to create a needed device
node in /dev. This is not tolerable, sorry: Software that tries to
tell the sysadmin he may not take necessary steps to administer his
system gets scrapped at the next convenient opportunity.
> > ifrename is cool. ;->
>
> was :)
Unchanged temperature. I just measured it!
> I'd rather NOT have that shit but since it's impossible to buy the CPU (or
> any Intel equivalent, and many/most ARM etc CPUs too) without it, my choices
> are: 1. don't buy it, 2. buy it with that shit.
I do empathise.
Possible help may soon emerge from a new-ish initiative with Raptor
Computing's Talos II series using IBM POWER9 CPUs, where there is,
refreshingly, none of that shit anywhere in the SoC or surrounding
circuitry. More speculatively, the J-Core project has been reviving the
Hitachi SuperH CPU architecture now that all of the patents have expired,
with hardware designs that are open all the way from top to bottom.
Much depends on completion of their roadmap, particularly the 64-bit
version of SH-4 & support circuitry. We shall see.
> I expect that the AMD PSP will also eventually be cracked, or someone will
> discover the NSA backdoor (or, as you mentioned, their disabling method as
> recently happened with the Intel Management Engine)
Inshallah.
[0] http://linuxmafia.com/~rick/lexicon.html#imvao
[1] Sorry, obscure pop-culture reference. I hope this can be streamed in Oz:
http://www.nbc.com/saturday-night-live/video/shimmer-floor-wax/n8625?snl=1
_______________________________________________
luv-main mailing list
[email protected]
https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main
