Here we have very strict control over our hardware and what interface goes
where. We keep CentOS 6's naming scheme on Dell hardware, so p2p1 is PCI
slot 2, Port 1, and don't try rename it. We have a 3rd party patch manager
tool (patchmanager.com), LLDP on our switches, and a Nagios check that
tells me if an interface is not plugged into the switch port it is supposed
to be plugged into (according to patchmanager). This works perfectly on
Dell hardware because the PCI name mapping works. On really old HP gear it
doesn't work, so we fall back on always assuming eth0 is the first onboard
port, etc. If the kernel scanned these devices in a different order we'd
get the same breakage you describe, but that's never happened on it's own,
it's only happened if an engineer has gone and added re-arranged cards.
We still need some sort of "glue record" that says "this interface should
be up and have this IP". In our older designs this was managed entirely in
Hiera - so there's a giant multi-level hash that we run create_resources()
over to define every single network interface. You can imagine the amount
of Hiera data we have. In the newer designs which are a lot more of a
role/profile approach I've been trying to conceptualise the networking
based on our profiles. So if one of our servers is fulfilling function
"database" there will be a Class[profile::database]. This Class might
create a bonded interface for the "STORAGE" network and another interface
for the "CLIENT" network. Through various levels of Hiera I can define the
STORAGE network as VLAN 100, because it might be a different vlan tag at a
different location. Then at the Hiera node level (on each individual
server) I will have something like:
profile::database::bond_storage_slaves: [ 'p2p1', 'p2p2' ]
That's the glue. At some point I need to tell Puppet that on this specific
server, the storage network is a bond of p2p1 and p2p2. If I took that
profile to a HP server, I'd be specifying a different set of interface
names. In some situations I even just put in one bond interface member,
which is useless, but in most situations I find less entropy is worth more
than having a slightly more efficient networking stack.
I have bounced around the idea of removing this step and trusting the
switch - ie: write a fact to do an LLDP query for the VLAN of the switch
port each interface is connected to, that way you wouldn't need the glue,
there'd be a fact called vlan_100_interfaces. Two problems with this
approach: we end up trusting the switch to be our source of truth (it may
not be correct, and, what if the switch port is down?). Secondly the
quality and consistency of LLDP information you get out of various
manufacturers of networking hardware is very different, so relying on LLDP
information to define your OS network config is a bit risky for me.
It's a different story for our VMs. Since they are Puppet defined we
specify a MAC address and so we "know" which MAC will be attached to which
VM bridge. We drop a MAC based udev rule into the guest to name them
similarly, ie: eth100 is on br100. I could technically use the same Puppet
code to write udev rules for my hardware, but the PCI based naming scheme
is fine so far.
That's what we do, but it's made easy by an almost homogeneous hardware
platform and strict physical patch management.
When I read about your problem, it sounds like you are missing a "glue
record" that describes your logical interfaces to your physical devices. If
you were to follow something along the lines of our approach, you might
have something like this:
class profile::some_firewall(
$external_interface_name = 'eth0',
$internal_interface_name = 'eth1',
$perimiter_interface_name = 'eth2'
) {
firewall { '001_allow_internal':
chain => 'INPUT',
iniface => $internal_interface_name,
action => 'accept',
proto => 'all',
}
firewall { '002_some_external_rule':
chain => 'INPUT',
iniface => $external_interface_name,
action => 'accept',
proto => 'tcp',
dport => '443',
}
}
That very simple firewall profile probably already works on your HP
hardware, and on your Dell hardware you'd need to override the 3 parameters
in Hiera:
profile::some_firewall::internal_interface_name: 'em1'
profile::some_firewall::external_interface_name: 'p3p1'
profile::some_firewall::perimiter_interface_name: 'p1p1'
Hope that helps,
-Luke
On Wednesday, 24 August 2016 14:55:38 UTC+1, Marc Haber wrote:
>
> Hi,
>
> I would like to discuss how to handle systemd's new feature of
> predictable network interface names. This is a rather hot topic in the
> team I'm currently working in, and I'd like to solicit your opinions
> about that.
>
> On systems with more than one interface, the canonical way to handle
> this issue in the past was "assume that eth0 is connected to network
> foo, eth1 is connected to network bar, and eth2 is connected to
> network baz" and to accept that things fail horribly if the order in
> which network interfaces are detected changes.
>
> While upstream's focus is as usual on desktop machines where Ethernet,
> USB and WWAN interfaces come and go multiple times a day (see
> upstream's reasoning in
>
> https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/),
>
>
> this seldomly happens in our happy server environment, which reduces
> the breakage potential to disruptive kernel updates or vendor firmware
> changes peddling with the order in which network interfaces are
> enumerated.
>
> This happens rather seldomly in my experience.
>
> I would, however, like to stay with the new scheme since I see its
> charms.
>
> But, how would I handle this in a puppetized environment?
>
> Currently, the code that is already, for example for a firewall
> assumes that eth0 is the external interface, eth1 the internal one and
> eth2 the perimeter networks.
>
> In the new setup, all those interfaces can have different names
> depending on different hardware being used. That means that the same
> puppet code cannot be used on one firewall instance running on Dell
> hardware and a second one running on HP hardware because BIOS indices
> and/or PCI paths will vary. If I used the MAC scheme, things are even
> worse since interface names will be different even on different pieces
> of otherwise identical hardware.
>
> Many of my team members thinkt hat one should simply turn of
> predictable network interface names altgether and so that our old code
> continues to work. I think that this would be a bad idea, but don't
> have any logical arguments other than my gut feeling.
>
> Generating udev rules to fix the network names (and assign names like
> ext1, int1, per1) already in postinst of the OS does not work since we
> don't know how the machine is going to be wired and even used.
>
> Any ideas? How do _you_ do this?
>
> Greetings
> Marc
>
> --
> -----------------------------------------------------------------------------
>
> Marc Haber | "I don't trust Computers. They | Mailadresse im
> Header
> Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224
> 1600402
> Nordisch by Nature | How to make an American Quilt | Fax: *49 6224
> 1600421
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/0521ee3e-519d-4c02-9da1-2fe37aad3fc5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
