Re: PF + BRIDGE + PFSYNC causes system freezing
On 17.03.2010 18:00, Max Laier wrote: Can you enable WITNESS and compile in DDB. Make sure to report any LORs and once the system freezes try to enter the debugger and get ps and locks information. show allchains show alllocks ps After that you can try to "call doadump" so you get the information in the coredump and don't have to transcribe it manually. Thanks, Max Sorry, I'm not really an expert of this, but how can I enter the debugger if the system has frozen? Ctrl+Alt+ESC (in default configuration). ___ I've added this to the kernel option KDB option WITNESS option WITNESS_KDB option DDB Now it can't even boot properly. It stops when it tries to configure networking: uma_zalloc_arg: zone "256" with the following non-sleepable locks held: exclusive rw ifnet_rw (ifnet_rw) r = 0 (0x80e31b20) locked @ /usr/src/sys/net/if.c:414 show allchains show alllocks exclusive rw ifnet_rw (ifnet_rw) r = 0 (0x80e31b20) locked @ /usr/src/sys/net/if.c:414 exclusive sx ifnet_sx (ifnet_sx) r = 0 (0x80e31b40) locked @ /usr/src/sys/net/if.c:414 ps call doadump Cannot dump. Device not defined or unavailable Hope it helps... ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: PF + BRIDGE + PFSYNC causes system freezing
On Thursday 18 March 2010 15:04:06 Giulio Ferro wrote: > On 17.03.2010 18:00, Max Laier wrote: > > Can you enable WITNESS and compile in DDB. Make sure to report any LORs > > > >>> and once the system freezes try to enter the debugger and get ps and > >>> locks information. > >>> > >>> show allchains > >>> show alllocks > >>> ps > >>> > >>> After that you can try to "call doadump" so you get the information in > >>> the coredump and don't have to transcribe it manually. > >>> > >>> Thanks, > >>> Max > >> > >> Sorry, I'm not really an expert of this, but how can I enter the > >> debugger if the system has frozen? > > > > Ctrl+Alt+ESC (in default configuration). > > ___ > > I've added this to the kernel > > option KDB > option WITNESS > option WITNESS_KDB remove WITNESS_KDB, it's not what you want. > option DDB > > > Now it can't even boot properly. It stops when it tries to configure > networking: > uma_zalloc_arg: zone "256" with the following non-sleepable locks held: > exclusive rw ifnet_rw (ifnet_rw) r = 0 (0x80e31b20) locked @ > /usr/src/sys/net/if.c:414 a "bt" would help in this case to see where the bad alloc is. > show allchains > > > show alllocks > exclusive rw ifnet_rw (ifnet_rw) r = 0 (0x80e31b20) locked @ > /usr/src/sys/net/if.c:414 > exclusive sx ifnet_sx (ifnet_sx) r = 0 (0x80e31b40) locked @ > /usr/src/sys/net/if.c:414 > > ps > > > call doadump > Cannot dump. Device not defined or unavailable define "dumpdev" in rc.conf to a swap partition with enough space or call dumpon(8). Thanks, Max ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Synproxy state - advertising 0 window size
I have small network like this: [Internet] --- rl0(FreeBSD -
router)rl1 --- [Lan]
I wanted to make whole outgoing packets from the Lan look the same. It
means that every SYN packet has the same TCP/IP stack. So I thought
about PF's synproxy state. I know synproxy was made for other puprose
but I tried to do something like this (this is piece of my PF
firewall):
ext_if="rl0"
int_if="rl1"
set skip on lo
scrub on $int_if min-ttl 129
nat on $ext_if from !($ext_if) to any -> ($ext_if)
pass in on $int_if proto tcp from any to any port {443, 8074} flags
S/SA synproxy state (ports are only examples)
Everything on this configuration works well but let's see listing from tcpdump:
### NATed synproxy packet###
# tcpdump -i rl0 -n -vvv 'tcp[13] & 2 != 0'
tcpdump: listening on rl0, link-type EN10MB (Ethernet), capture size 96 bytes
15:09:14.680832 IP (tos 0x10, ttl 128, id 35567, offset 0, flags [DF],
proto TCP (6), length 44)
10.0.0.101.51220 > 91.111.111.12.443: Flags [S], cksum 0xf73f
(correct), seq 2917250499, win 0, options [mss 1460], length 0
15:09:14.714002 IP (tos 0x0, ttl 56, id 0, offset 0, flags [DF], proto
TCP (6), length 44)
91.111.111.12.443 > 10.0.0.101.51220: Flags [S.], cksum 0x819e
(correct), seq 1940581141, ack 2917250500, win 5840, options [mss
1460], length 0
###System SYN
15:11:05.876433 IP (tos 0x0, ttl 128, id 35741, offset 0, flags [DF],
proto TCP (6), length 48)
10.0.0.101.55040 > 94.23.95.22.80: Flags [S], cksum 0x7741
(correct), seq 414405961, win 65535, options [mss 1460,sackOK,eol],
length 0
15:11:05.920871 IP (tos 0x0, ttl 56, id 0, offset 0, flags [DF], proto
TCP (6), length 44)
94.23.95.22.80 > 10.0.0.101.55040: Flags [S.], cksum 0xcccf
(correct), seq 106340672, ack 414405962, win 5840, options [mss 1460],
length 0
1. In first SYN packet (from PF's synproxy) we can see that it doesnt
have any options (why?) and it advertises 0 windows size why?
2. In second SYN which comes from FreeBSD (time stamps are disabled
and ttl is changed) there are options and window size. Why do these
both packets are different? Is it normal that synproxy sends SYNs with
0 window size?
___
freebsd-pf@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: PF + BRIDGE + PFSYNC causes system freezing
On 18.03.2010 15:26, Max Laier wrote: Ok, it's happened again... and once the system freezes try to enter the debugger and get ps and locks information. show allchains No result show alllocks Process 4483 (sshd) thread 0xff0002ded3a0 (100159) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xff0002c79b98) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 12 (intr) thread 0xff000242b3a0 (100028) exclusive sleep mutex if_bridge (if_bridge) r = 0 (0xff000282d018) locked @ /usr/src/sys/modules/if_bridge/../../net/if_bridge.c:2162 Process 12 (intr) thread 0xff00023d3ae0 (100021) exclusive slepp mutex Giant (Giant) r = 1 (0x80c6f660) locked @ /usr/src/sys/dev/usb/usb_transfer.c:3009 Process 12 (intr) thread 0xff00022603a0 (107) exclusive sleep mutex carp_if (carp_if) r = 0 (0xff0002730360) locked @ /usr/src/sys/netinet/ip_carp.c:881 ps This yields a lot of lines, tell me if you want me to report something special a "bt" would help in this case to see where the bad alloc is. Tracing pid 12 tid 100021 td 0xff00023d3ae0 kdb_enter() at kdb_enter+0x3d ... Thank for your interest. ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: PF + BRIDGE + PFSYNC causes system freezing
On Thursday 18 March 2010 17:39:29 Giulio Ferro wrote: > On 18.03.2010 15:26, Max Laier wrote: > > Ok, it's happened again... > and once the system freezes try to enter the debugger and get ps and > > > locks information. > > > > show allchains > > No result Okay ... so it looks like this is a live lock (not a deadlock) and it's probably caused by relooping packets. Now we "only" have to find the culprit for the loop ... can you share your setup details, again? The simpler the better. > > show alllocks > > Process 4483 (sshd) thread 0xff0002ded3a0 (100159) > exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xff0002c79b98) locked @ > /usr/src/sys/kern/uipc_sockbuf.c:148 > Process 12 (intr) thread 0xff000242b3a0 (100028) > exclusive sleep mutex if_bridge (if_bridge) r = 0 (0xff000282d018) > locked @ /usr/src/sys/modules/if_bridge/../../net/if_bridge.c:2162 > Process 12 (intr) thread 0xff00023d3ae0 (100021) > exclusive slepp mutex Giant (Giant) r = 1 (0x80c6f660) locked @ > /usr/src/sys/dev/usb/usb_transfer.c:3009 > Process 12 (intr) thread 0xff00022603a0 (107) > exclusive sleep mutex carp_if (carp_if) r = 0 (0xff0002730360) > locked @ /usr/src/sys/netinet/ip_carp.c:881 > > > ps > > This yields a lot of lines, tell me if you want me to report something > special > > > a "bt" would help in this case to see where the bad alloc is. > > Tracing pid 12 tid 100021 td 0xff00023d3ae0 > kdb_enter() at kdb_enter+0x3d > ... > > Thank for your interest. > > > !DSPAM:4ba25756174452108219161! > ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: PF + BRIDGE + PFSYNC causes system freezing
On 18.03.2010 20:35, Max Laier wrote:
Okay ... so it looks like this is a live lock (not a deadlock) and it's
probably caused by relooping packets. Now we "only" have to find the culprit
for the loop ...
can you share your setup details, again? The simpler the better.
Ok
> uname -a
FreeBSD firewall-1.acme.com 8.0-STABLE FreeBSD 8.0-STABLE #2: Thu Mar 18
15:59:27 CET 2010 r...@acme.com:/usr/obj/usr/src/sys/FIREWALL amd64
> cat /etc/sysctl.conf
net.inet.ip.forwarding=1
net.inet.ip.fastforwarding=1
net.inet.carp.preempt=1
Services running : sshd, named, inetd, ntpd, openvpn (tap), racoon,
pptp, asterisk
2 physical interfaces : bce0, bce1
11 vlan interfaces : vlan1, ..., vlan11 (vlandev bce1)
11 carp interfaces ; carp1, ..., carp11 (carp1 has 23 alias addresses)
1 bridge interfaces : bridge0 addm vlan35 (used by openvpn)
2 gif interfaces : gif0, gif1 (racoon / IPSEC)
8 static routes
pf packet filter : 12 rdr rules, 3 nat rules, set skip{lo0, bridge0}, 4
pass quick, block log all, about 30 pass keep state
This should be all.
I'm available for any test / patch...
Thanks.
___
freebsd-pf@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Fwd: Crash in pf(4) with a fairly recent RELENG_8
Duh, never thought of writing to this list first :/
Forwarded conversation
Subject: Crash in pf(4) with a fairly recent RELENG_8
From: *Vlad Galu*
Date: Thu, Mar 18, 2010 at 12:38 AM
To: freebsd-sta...@freebsd.org
Luckily I could find this coredump:
-- cut here --
#0 doadump () at pcpu.h:223
#1 0x802f4ace in boot (howto=260) at
../../../kern/kern_shutdown.c:416
#2 0x802f4eab in panic (fmt=Variable "fmt" is not available.
) at ../../../kern/kern_shutdown.c:579
#3 0x805064d2 in trap_fatal (frame=0xff8345c0, eva=0)
at ../../../amd64/amd64/trap.c:857
#4 0x80506e8c in trap (frame=0xff8345c0)
at ../../../amd64/amd64/trap.c:644
#5 0x804eec93 in calltrap () at
../../../amd64/amd64/exception.S:224
#6 0x801a1140 in pf_state_tree_id_RB_MINMAX ()
at ../../../contrib/pf/net/pf.c:401
#7 0x801a1210 in pf_src_tree_RB_FIND (head=Variable "head" is
not available.
)
at ../../../contrib/pf/net/pf.c:396
#8 0x801a3594 in pf_insert_src_node (sn=0xff834868,
rule=0xff0001694000, src=0xff000d75701c, af=2 '\002')
at ../../../contrib/pf/net/pf.c:850
#9 0x801acd6e in pf_test_tcp (rm=0xff834978,
sm=0xff834970, direction=1, kif=0xff000132ab00,
m=0xff001e052b00, off=20, h=0xff000d757010,
pd=0xff834990,
am=0xff834980, rsm=0xff834968, ifq=0x0, inp=0x0)
at ../../../contrib/pf/net/pf.c:3500
#10 0x801ae7a6 in pf_test (dir=1, ifp=0xff0001201000,
m0=0xff834ac8, eh=Variable "eh" is not available.
) at ../../../contrib/pf/net/pf.c:7066
#11 0x801b33a9 in pf_check_in (arg=Variable "arg" is not available.
)
at ../../../contrib/pf/net/pf_ioctl.c:3646
-- and here --
--
Good, fast & cheap. Pick any two.
--
From: *Vlad Galu*
Date: Thu, Mar 18, 2010 at 12:44 AM
To: freebsd-sta...@freebsd.org
The pf_src_node struct in frame #8 is this:
-- cut here--
(kgdb) p k
$1 = {entry = {rbe_left = 0x0, rbe_right = 0x0,
rbe_parent = 0x, rbe_color = 0}, addr = {pfa = {v4 = {
s_addr = 1684237067}, v6 = {__u6_addr = {
__u6_addr8 = "\vkcd\200???\001\000\000\000\000\000\000",
__u6_addr16 = {27403, 25699, 65408, 65535, 1, 0, 0, 0},
__u6_addr32 = {1684237067, 4294967168, 1, 0}}},
addr8 = "\vkcd\200???\001\000\000\000\000\000\000", addr16 = {27403,
25699, 65408, 65535, 1, 0, 0, 0}, addr32 = {1684237067, 4294967168,
1,
0}}}, raddr = {pfa = {v4 = {s_addr = 12}, v6 = {__u6_addr = {
__u6_addr8 = "\f\000\000\000\000\000\000\000\000?2\001\000???",
__u6_addr16 = {12, 0, 0, 0, 43776, 306, 65280, 65535},
__u6_addr32 = {12, 0, 20097792, 4294967040}}},
addr8 = "\f\000\000\000\000\000\000\000\000?2\001\000???", addr16 =
{12,
0, 0, 0, 43776, 306, 65280, 65535}, addr32 = {12, 0, 20097792,
4294967040}}}, rule = {ptr = 0xff0001694000, nr = 23674880},
kif = 0x801a9858, bytes = {18446743523953737740,
18446742974423724064}, packets = {3354, 17179869187}, states = 23510160,
conn = 4294967040, conn_rate = {limit = 23403040, seconds = 4294967040,
count = 20097792, last = 4294967040}, creation = 2, expire = 0,
af = 2 '\002', ruletype = 0 '\0'}
-- and here--
The byte count looks weird...
--
Good, fast & cheap. Pick any two.
___
freebsd-pf@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
