Very nice, Luke.
Does the code that lets you custom-name your interfaces live in github or
puppet-forge anywhere ?
If not, would you be willing to share ? I can bring brownies and/or beer to
the collaboration :)
Dan White | d_e_wh...@icloud.com
------------------------------------------------
“Sometimes I think the surest sign that intelligent life exists elsewhere in the
universe is that none of it has tried to contact us.” (Bill Waterson: Calvin &
Hobbes)
On Aug 24, 2016, at 11:36 AM, Luke Bigum <luke.bi...@lmax.com> wrote:
Here we have very strict control over our hardware and what interface goes
where. We keep CentOS 6's naming scheme on Dell hardware, so p2p1 is PCI slot
2, Port 1, and don't try rename it. We have a 3rd party patch manager tool
(patchmanager.com), LLDP on our switches, and a Nagios check that tells me if
an interface is not plugged into the switch port it is supposed to be plugged
into (according to patchmanager). This works perfectly on Dell hardware because
the PCI name mapping works. On really old HP gear it doesn't work, so we fall
back on always assuming eth0 is the first onboard port, etc. If the kernel
scanned these devices in a different order we'd get the same breakage you
describe, but that's never happened on it's own, it's only happened if an
engineer has gone and added re-arranged cards.
We still need some sort of "glue record" that says "this interface should be up and have this IP". In our
older designs this was managed entirely in Hiera - so there's a giant multi-level hash that we run create_resources() over to
define every single network interface. You can imagine the amount of Hiera data we have. In the newer designs which are a lot
more of a role/profile approach I've been trying to conceptualise the networking based on our profiles. So if one of our servers
is fulfilling function "database" there will be a Class[profile::database]. This Class might create a bonded interface
for the "STORAGE" network and another interface for the "CLIENT" network. Through various levels of Hiera I
can define the STORAGE network as VLAN 100, because it might be a different vlan tag at a different location. Then at the Hiera
node level (on each individual server) I will have something like:
profile::database::bond_storage_slaves: [ 'p2p1', 'p2p2' ]
That's the glue. At some point I need to tell Puppet that on this specific
server, the storage network is a bond of p2p1 and p2p2. If I took that profile
to a HP server, I'd be specifying a different set of interface names. In some
situations I even just put in one bond interface member, which is useless, but
in most situations I find less entropy is worth more than having a slightly
more efficient networking stack.
I have bounced around the idea of removing this step and trusting the switch -
ie: write a fact to do an LLDP query for the VLAN of the switch port each
interface is connected to, that way you wouldn't need the glue, there'd be a
fact called vlan_100_interfaces. Two problems with this approach: we end up
trusting the switch to be our source of truth (it may not be correct, and, what
if the switch port is down?). Secondly the quality and consistency of LLDP
information you get out of various manufacturers of networking hardware is very
different, so relying on LLDP information to define your OS network config is a
bit risky for me.
It's a different story for our VMs. Since they are Puppet defined we specify a MAC
address and so we "know" which MAC will be attached to which VM bridge. We drop
a MAC based udev rule into the guest to name them similarly, ie: eth100 is on br100. I
could technically use the same Puppet code to write udev rules for my hardware, but the
PCI based naming scheme is fine so far.
That's what we do, but it's made easy by an almost homogeneous hardware
platform and strict physical patch management.
When I read about your problem, it sounds like you are missing a "glue record"
that describes your logical interfaces to your physical devices. If you were to follow
something along the lines of our approach, you might have something like this:
class profile::some_firewall(
$external_interface_name = 'eth0',
$internal_interface_name = 'eth1',
$perimiter_interface_name = 'eth2'
) {
firewall { '001_allow_internal':
chain => 'INPUT',
iniface => $internal_interface_name,
action => 'accept',
proto => 'all',
}
firewall { '002_some_external_rule':
chain => 'INPUT',
iniface => $external_interface_name,
action => 'accept',
proto => 'tcp',
dport => '443',
}
}
That very simple firewall profile probably already works on your HP hardware,
and on your Dell hardware you'd need to override the 3 parameters in Hiera:
profile::some_firewall::internal_interface_name: 'em1'
profile::some_firewall::external_interface_name: 'p3p1'
profile::some_firewall::perimiter_interface_name: 'p1p1'
Hope that helps,
-Luke
On Wednesday, 24 August 2016 14:55:38 UTC+1, Marc Haber wrote:
Hi,
I would like to discuss how to handle systemd's new feature of
predictable network interface names. This is a rather hot topic in the
team I'm currently working in, and I'd like to solicit your opinions
about that.
On systems with more than one interface, the canonical way to handle
this issue in the past was "assume that eth0 is connected to network
foo, eth1 is connected to network bar, and eth2 is connected to
network baz" and to accept that things fail horribly if the order in
which network interfaces are detected changes.
While upstream's focus is as usual on desktop machines where Ethernet,
USB and WWAN interfaces come and go multiple times a day (see
upstream's reasoning in
https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/),
this seldomly happens in our happy server environment, which reduces
the breakage potential to disruptive kernel updates or vendor firmware
changes peddling with the order in which network interfaces are
enumerated.
This happens rather seldomly in my experience.
I would, however, like to stay with the new scheme since I see its
charms.
But, how would I handle this in a puppetized environment?
Currently, the code that is already, for example for a firewall
assumes that eth0 is the external interface, eth1 the internal one and
eth2 the perimeter networks.
In the new setup, all those interfaces can have different names
depending on different hardware being used. That means that the same
puppet code cannot be used on one firewall instance running on Dell
hardware and a second one running on HP hardware because BIOS indices
and/or PCI paths will vary. If I used the MAC scheme, things are even
worse since interface names will be different even on different pieces
of otherwise identical hardware.
Many of my team members thinkt hat one should simply turn of
predictable network interface names altgether and so that our old code
continues to work. I think that this would be a bad idea, but don't
have any logical arguments other than my gut feeling.
Generating udev rules to fix the network names (and assign names like
ext1, int1, per1) already in postinst of the OS does not work since we
don't know how the machine is going to be wired and even used.
Any ideas? How do _you_ do this?
Greetings
Marc
--
-----------------------------------------------------------------------------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
--
You received this message because you are subscribed to the Google Groups "Puppet
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/0521ee3e-519d-4c02-9da1-2fe37aad3fc5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Puppet
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/41582501-2632-48e0-bb7d-033217c3e1c6%40me.com.
For more options, visit https://groups.google.com/d/optout.
