Hi Adrian,
On 01/06/14 17:21, Adrian Chadd wrote:
I've been seeing this panic under high very short-term connection TCP
throughput tests (30,000 connections a second) with SO_REUSEPORT:
Current language: auto; currently minimal
(kgdb) frame 11
#11 0xffffffff80a6bdf1 in tcp_twclose (tw=0xfffff801348b5780, reuse=0)
at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/tcp_timewait.c:644
644 crfree(tw->tw_cred);
(kgdb) frame 11
#11 0xffffffff80a6bdf1 in tcp_twclose (tw=0xfffff801348b5780, reuse=0)
at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/tcp_timewait.c:644
644 crfree(tw->tw_cred);
(kgdb) print tw
$1 = (struct tcptw *) 0xfffff801348b5780
(kgdb) print tw->tw_cred
$2 = (struct ucred *) 0x0
[...]
... there's only one path to deleting the credentials and that's via
tcp_twclose() -> tcp_tw_2msl_stop().
Has anyone seen this kind of problem before?
Interesting question: I already got this issue also under short-lived
high connection rate (~120k connections per second) and using
SO_REUSEPORT _however_ it was with our TCP scaling patches applied (thus
it might be a red herring for you) and I fixed it with a:
tcp_tw_2msl_stop():
...
if (tw->tw_cred != NULL)
crfree(tw->tw_cred);
https://github.com/verisign/freebsd/commit/918fe94ad3f3ccd1f6bf577e7e79a138378262a0#diff-4f0c0d9222bfb69f2f358113d424cf89R610
At that time I thought you could create a tw with its tw->tw_cred
field set to NULL and then tcp_tw_2msl_stop() simply missed a sanity
check before calling crfree().
Now I re-checked how tw->tw_cred is set, its seems impossible to
create a tw with tw->tw_cred = NULL (aha!). As this tw_cred field seems
well protected by both INP_INFO_WLOCK and INP_WLOCK from race
conditions, I am currently checking if something could inadvertently
overwrite tw_cred in struct tcptw.
My 2 cents.
--
Julien
_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
