Thanks Peter. I could have sworn I checked and rechecked exactly which
interface is which, but clearly not enough times :-). The problem
being that while they have distinct MAC addresses due to being from
different manufacturers, they both use the RL driver. Your suggestion
for confirming which ifconfig listing corresponds to which physical
network card is great. So making progress, but not done yet.
As it stands now, I set rl0 as the LAN interface and rl1 for WAN. And
I have confirmed that plugging and unplugging the network cable from
each results in ifconfig showing "active" or "no carrier" for the
correct interface.
On the LAN side (rl0), it is issuing DHCP addresses to clients, and
once it does so, I can ping from a DHCP client to both the LAN and WAN
IP address of OpenBSD (so the IPv4 forwarding seems to work, yay), and
also from OpenBSD to the DHCP-issued IP address of the other system.
Ssh (both directions, from Ubuntu to OpenBSD and vice versa) seems to
hang, after the SSH host key has been received and verified, according
to ssh -v (it stops on SSH2_MSG_SERVICE_ACCEPT received). It may
succeed, but simply be slow, I have to run it again to see if it times
out (I cancelled it), but then the slower system is 600Mhz and they
are 1 meter from each other with a known good ethernet cable so how
long should it take?. So it should not be blocked by pf I don't think,
but again, not certain. Have to figure that one out as I use ssh a
lot. One minor note, somehow Ubuntu "abandons" the DHCP lease
(according to /var/db/dhcpd.leases on OpenBSD) and gets a new one.
Maybe a bug in the Ubuntu dhclient implementation. Not too worried
about it now as long as it maintains/reestablishes connection ok.
WAN (rl1) is still problematic. Still no outbound ping to ISP gateway
(I've verified the same works from Ubuntu with static IP, so ICMP is
not blocked/dropped by them from what I can tell). I retried after
pfctl -F all and also after pfctl -d, so I believe this part of the
problem is due to routing (mis)configuration alone, more than pf. From
OpenBSD, I can ping both the LAN and WAN interface on OpenBSD itself,
so I think the hostname.rl0 and hostname.rl1 files should be in decent
shape.
This is the output of netstat -rn (long! keep in mind almost
everything is put there automatically at boot time by /etc/mygate or
when I execute route add from the terminal, aside from the default
gateway little or nothing is added by me. It also shows how the DHCP
lease was abandoned, as one single MAC addr. shows two IPs 192.168.6.8
and .9 even though I only ran dhclient once on the client system):
Routing tables
Internet:
Destination Gateway Flags Refs Use Mtu Prio Iface
default 188.126.4.1 UGS 1 7188 - 8 rl1
127/8 127.0.0.1 UGRS 0 0 33192 8 lo0
127.0.0.1 127.0.0.1 UH 1 0 33192 4 lo0
188.126.4/24 link#3 UC 2 0 - 4 rl1
188.126.4.1 00:30:48:b8:c5:83 UHLc 1 72 - 4 rl1
188.126.4.24 4c:00:10:3c:23:5c UHLc 0 6 - 4 lo0
192.168.6/24 link#1 UC 3 0 - 4 rl0
192.168.6.6 00:40:f4:44:07:56 UHLc 0 6 - 4 lo0
192.168.6.8 00:80:ad:00:7c:ca UHLc 1 44 - 4 rl0
192.168.6.9 00:80:ad:00:7c:ca UHLc 0 70 - 4 rl0
224/4 127.0.0.1 URS 0 0 33192 8 lo0
Internet6:
Destination Gateway
Flags Refs Use Mtu Prio Iface
::/104 ::1 UGRS
0 0 - 8 lo0
::/96 ::1 UGRS
0 0 - 8 lo0
::1 ::1 UH
14 0 33192 4 lo0
::127.0.0.0/104 ::1 UGRS
0 0 - 8 lo0
::224.0.0.0/100 ::1 UGRS
0 0 - 8 lo0
::255.0.0.0/104 ::1 UGRS
0 0 - 8 lo0
::ffff:0.0.0.0/96 ::1 UGRS
0 0 - 8 lo0
2002::/24 ::1 UGRS
0 0 - 8 lo0
2002:7f00::/24 ::1 UGRS
0 0 - 8 lo0
2002:e000::/20 ::1 UGRS
0 0 - 8 lo0
2002:ff00::/24 ::1 UGRS
0 0 - 8 lo0
fe80::/10 ::1 UGRS
0 0 - 8 lo0
fe80::%rl0/64 link#1 UC
0 0 - 4 rl0
fe80::240:f4ff:fe44:756%rl0 00:40:f4:44:07:56 HL
0 0 - 4 lo0
fe80::%rl1/64 link#3 UC
0 0 - 4 rl1
fe80::4e00:10ff:fe3c:235c%rl1 4c:00:10:3c:23:5c HL
0 0 - 4 lo0
fe80::%lo0/64 fe80::1%lo0 U
0 0 - 4 lo0
fe80::1%lo0 link#5 UHL
0 0 - 4 lo0
fec0::/10 ::1 UGRS
0 0 - 8 lo0
ff01::/16 ::1 UGRS
0 0 - 8 lo0
ff01::%rl0/32 link#1 UC
0 0 - 4 rl0
ff01::%rl1/32 link#3 UC
0 0 - 4 rl1
ff01::%lo0/32 fe80::1%lo0 UC
0 0 - 4 lo0
ff02::/16 ::1 UGRS
0 0 - 8 lo0
ff02::%rl0/32 link#1 UC
0 0 - 4 rl0
ff02::%rl1/32 link#3 UC
0 0 - 4 rl1
ff02::%lo0/32 fe80::1%lo0 UC
0 0 - 4 lo0
Is /etc/mygate the correct way to route on a system with two or more
NICs (IP addresses) or is it better to put route add commands in each
hostname file for the appropriate NIC? (i.e route traffic for the
internet via rl1, and traffic for 192.168.whatever via rl0)
Something else I had not noticed before, the ISP has BGPd running
accrding to nmap under Ubuntu, which it occurred to me might explain a
lot of this, though again, it isn't logical why it works on Ubuntu but
not OpenBSD, but if I need for example a BGP client installed and
configured that might sort it out. No such client or server installed
under Ubuntu though, so that seems unlikely.
