Hi list,
I'm kind of despair.
The history: We recently brought up a new firewall with Gentoo.
There are (for my finding) some big nets behind this firewall (1x public
/24, 2x public /27, 1x public /26, at least 2 private /24).
Filtering is done via iptables and snort should jump as IPS on
software-bridge br0. If it helps: There is also ip rule involved for
source-based routing.
The new firewall replaces an older Gentoo-system which did not show this
behavior. We therefore copied several configfiles from the old to the new
one.
After getting it live, it runs well for a few hours and then becomes
unreachable (also for hosts behind the bridge).
Dmesg / kern.log stated at this time a neighbor table overflow and indeed,
arp -n | wc -l showed a lot of entry's.
As Google suggested, We then adjusted /proc/sys/net/ipv4/neigh/default/ to:
gc_thershold1 -> 8192
gc_thershold2 -> 16384
gc_thershold3 -> 32768
Fireing an "arp -d $bogus-ip-adress" is failing with "SIOCDARP(dontpub):
Network is unreachable", adding -i br0 doesn't fail, but does not remove the
line in the arp-table (it only says "incomplete" after greping arp -n
again)..
Therefore we are currently killing the arp-cache with "ip link set arp off
dev br0 && ip link set arp on dev br0" by a cronjob.
The combination of these workarounds are keeping the firewall reachable and
"alive".
After stabilizing, we looked at the output of arp -n and noticed, that about
99(.999)% of the roundabout 11.000 (and rising) arp-cache-entry's contained
public addresses for which the bridge of the firewall should not feel
responsible (e.g. the public Google-dns-resolver and a load of more).
The MAC-entry for these public addresses is always the one of our router,
which is for sure the correct next hop.
But from my understanding, it should arp-cache only "our" net's directly at
the cable and not those public ones.
It looks like a configuration-issue, but I don't know, where to start
looking. I've already checked the default-gateway, netmasks,
broadcast-addresses and to me, they are looking fine, so any poke where to
start looking is greatly appreciated.
In case it will help, I attached the /etc/conf.d/net, ifconfig -a and route
-n.
If something else is needed, feel free to ask.
Hope, anyone can help.
Thanks in advance,
Ralf
host ~ # route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 89.XXX.XXX.3 0.0.0.0 UG 0 0 0 br0
10.23.42.2 0.0.0.0 255.255.255.255 UH 0 0 0 ppp1
87.186.224.50 0.0.0.0 255.255.255.255 UH 0 0 0 ppp0
89.XXX.XXX.0 0.0.0.0 255.255.255.0 U 0 0 0 br0
127.0.0.0 127.0.0.1 255.0.0.0 UG 0 0 0 lo
134.XX.X.0 0.0.0.0 255.255.255.0 U 0 0 0 lan
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 lan
192.168.2.0 0.0.0.0 255.255.255.0 U 0 0 0 mgm
192.168.7.0 0.0.0.0 255.255.255.0 U 0 0 0 tun0
192.168.9.0 0.0.0.0 255.255.255.0 U 0 0 0 tun0
192.168.20.0 0.0.0.0 255.255.255.0 U 0 0 0 tun1
192.168.42.0 0.0.0.0 255.255.255.0 U 0 0 0 tun1
192.168.254.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan
213.XXX.140.0 0.0.0.0 255.255.255.224 U 0 0 0 br0
213.XXX.141.96 0.0.0.0 255.255.255.224 U 0 0 0 br0
213.XXX.143.128 0.0.0.0 255.255.255.192 U 0 0 0 br0
modules=( "iproute2" )
config_dsl="null"
config_lan="192.168.1.110 netmask 255.255.255.0 brd 192.168.1.255
134.XX.X.102 netmask 255.255.255.240 brd 134.XX.X.111
134.XX.X.103 netmask 255.255.255.240 brd 134.XX.X.111
134.XX.X.104 netmask 255.255.255.240 brd 134.XX.X.111
134.XX.X.105 netmask 255.255.255.240 brd 134.XX.X.111
134.XX.X.106 netmask 255.255.255.240 brd 134.XX.X.111
134.XX.X.107 netmask 255.255.255.240 brd 134.XX.X.111
134.XX.X.108 netmask 255.255.255.240 brd 134.XX.X.111
134.XX.X.109 netmask 255.255.255.240 brd 134.XX.X.111
134.XX.X.110 netmask 255.255.255.240 brd 134.XX.X.111"
config_mgm="192.168.2.254 netmask 255.255.255.0 brd 192.168.2.255"
config_dmz="null"
config_isp="null"
config_wlan="192.168.254.254 netmask 255.255.255.0 brd 192.168.254.255"
dns_domain_lan="herp.derp.local"
dns_servers_lan="192.168.1.XXX 192.168.1.XXY"
dns_search_lan="herp.derp.local"
#-----------------------------------------------------------------------------
# Bridging (802.1d)
bridge_br0="dmz isp"
config_br0="89.XXX.XXX.4 netmask 255.255.255.0 brd 89.XXX.XXX.255
89.XXX.XXX.10 netmask 255.255.255.0 brd 89.XXX.XXX.255
89.XXX.XXX.12 netmask 255.255.255.0 brd 89.XXX.XXX.255
89.XXX.XXX.13 netmask 255.255.255.0 brd 89.XXX.XXX.255
89.XXX.XXX.38 netmask 255.255.255.0 brd 89.XXX.XXX.255
89.XXX.XXX.86 netmask 255.255.255.0 brd 89.XXX.XXX.255
213.XXX.140.2 netmask 255.255.255.224 brd 213.XXX.140.31
213.XXX.140.30 netmask 255.255.255.224 brd 213.XXX.140.31
213.XXX.141.126 netmask 255.255.255.224 brd 213.XXX.141.127
213.XXX.143.132 netmask 255.255.255.192 brd 213.XXX.143.191
213.XXX.143.133 netmask 255.255.255.192 brd 213.XXX.143.191
213.XXX.141.119 netmask 255.255.255.224 brd 213.XXX.141.127
213.XXX.143.150 netmask 255.255.255.192 brd 213.XXX.143.191
213.XXX.143.151 netmask 255.255.255.192 brd 213.XXX.143.191
213.XXX.143.152 netmask 255.255.255.192 brd 213.XXX.143.191
213.XXX.143.153 netmask 255.255.255.192 brd 213.XXX.143.191
213.XXX.143.154 netmask 255.255.255.192 brd 213.XXX.143.191"
routes_br0="default via 89.XXX.XXX.3"
depend_br0() {
before firewall
need net.dmz net.ISP
after net.dmz net.ISP
}
postup() {
if [ "${IFACE}" = "isp" ]; then
ebegin "Setting Interface ISP to 100 mbit full duplex..."
/usr/sbin/ethtool -s isp speed 100 duplex full autoneg off
eend $? "Can't set ISP-Settings!"
fi
if [ "${IFACE}" = "ppp0" ]; then
ebegin "Adding rule"
ip rule add from 192.168.1.0/24 table lan &>/dev/null
eend $? "Can't set rule!"
fi
}
br0 Link encap:Ethernet HWaddr 00:11:85:d6:71:8f
inet addr:89.XXX.XXX.4 Bcast:89.XXX.XXX.255 Mask:255.255.255.0
inet6 addr: fe80::211:85ff:fed6:718f/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:21543996 errors:0 dropped:0 overruns:0 frame:0
TX packets:21869156 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:10737947552 (10.0 GiB) TX bytes:10378221148 (9.6 GiB)
dmz Link encap:Ethernet HWaddr 00:11:85:d6:71:8f
inet6 addr: fe80::211:85ff:fed6:718f/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:36445517 errors:0 dropped:0 overruns:0 frame:0
TX packets:27841566 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:31903525471 (29.7 GiB) TX bytes:4987550990 (4.6 GiB)
Interrupt:26
dsl Link encap:Ethernet HWaddr 00:1b:21:0f:41:41
inet6 addr: fe80::21b:21ff:fe0f:4141/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2461659 errors:0 dropped:0 overruns:0 frame:0
TX packets:1701781 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2384306133 (2.2 GiB) TX bytes:240457064 (229.3 MiB)
lan Link encap:Ethernet HWaddr 00:11:85:d6:71:90
inet addr:192.168.1.110 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::211:85ff:fed6:7190/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:16518448 errors:0 dropped:5243 overruns:0 frame:0
TX packets:19568947 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4005459306 (3.7 GiB) TX bytes:8126688260 (7.5 GiB)
Interrupt:25
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:21543 errors:0 dropped:0 overruns:0 frame:0
TX packets:21543 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1992380 (1.9 MiB) TX bytes:1992380 (1.9 MiB)
mgm Link encap:Ethernet HWaddr 00:1b:21:0f:41:43
inet addr:192.168.2.254 Bcast:192.168.2.255 Mask:255.255.255.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
ppp0 Link encap:Point-to-Point Protocol
inet addr:79.XXX.XXX.XXX P-t-P:87.186.224.50 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1492 Metric:1
RX packets:2442227 errors:0 dropped:0 overruns:0 frame:0
TX packets:1682611 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:3
RX bytes:2319551313 (2.1 GiB) TX bytes:201220349 (191.8 MiB)
ppp1 Link encap:Point-to-Point Protocol
inet addr:10.XX.XX.X P-t-P:10.XX.XX.X Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1396 Metric:1
RX packets:110038 errors:0 dropped:0 overruns:0 frame:0
TX packets:81827 errors:180 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:3
RX bytes:9280361 (8.8 MiB) TX bytes:28518226 (27.1 MiB)
sit0 Link encap:IPv6-in-IPv4
NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
tun0 Link encap:UNSPEC HWaddr
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:192.168.9.1 P-t-P:192.168.9.1 Mask:255.255.255.0
UP POINTOPOINT RUNNING NOARP PROMISC MULTICAST MTU:1500 Metric:1
RX packets:265876 errors:0 dropped:0 overruns:0 frame:0
TX packets:289363 errors:0 dropped:64 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:47003700 (44.8 MiB) TX bytes:151698442 (144.6 MiB)
tun1 Link encap:UNSPEC HWaddr
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:192.168.42.1 P-t-P:192.168.42.1 Mask:255.255.255.0
UP POINTOPOINT RUNNING NOARP PROMISC MULTICAST MTU:1500 Metric:1
RX packets:8516 errors:0 dropped:0 overruns:0 frame:0
TX packets:8559 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:2155733 (2.0 MiB) TX bytes:3801803 (3.6 MiB)
isp Link encap:Ethernet HWaddr 00:1b:21:0f:41:40
inet6 addr: fe80::21b:21ff:fe0f:4140/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:25014276 errors:0 dropped:0 overruns:0 frame:0
TX packets:30773696 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6270023327 (5.8 GiB) TX bytes:31096527998 (28.9 GiB)
wlan Link encap:Ethernet HWaddr 00:1b:21:0f:41:42
inet addr:192.168.254.254 Bcast:192.168.254.255 Mask:255.255.255.0
inet6 addr: fe80::21b:21ff:fe0f:4142/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:350673 errors:0 dropped:0 overruns:0 frame:0
TX packets:467172 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:49669601 (47.3 MiB) TX bytes:207668041 (198.0 MiB)
