Re: RELENG_8 pf stack issue (state count spiraling out of control)

2011-05-03 Thread Jeremy Chadwick
On Tue, May 03, 2011 at 10:31:57AM +0100, Vincent Hoffman wrote: > On 03/05/2011 10:16, Jeremy Chadwick wrote: > > > > Sadly I don't see a way with bsnmpd(8) to monitor things like interrupt > > usage, etc. otherwise I'd be graphing that. The more monitoring the > > better; at least then I could

Re: RELENG_8 pf stack issue (state count spiraling out of control)

2011-05-03 Thread Daniel Hartmeier
On Mon, May 02, 2011 at 06:58:54PM -0700, Jeremy Chadwick wrote: > The next thing I tried was "/etc/rc.d/pf stop", which worked. Then I > did "/etc/rc.d/pf start", which also worked. However, what I saw next > surely indicated a bug in the pf layer somewhere -- "pfctl -s states" > and "pfctl -s

Re: RELENG_8 pf stack issue (state count spiraling out of control)

2011-05-03 Thread Vlad Galu
On Tue, May 3, 2011 at 12:12 PM, Vlad Galu wrote: > > > On Tue, May 3, 2011 at 11:31 AM, Vincent Hoffman wrote: > >> On 03/05/2011 10:16, Jeremy Chadwick wrote: >> >> >> > Sadly I don't see a way with bsnmpd(8) to monitor things like interrupt >> > usage, etc. otherwise I'd be graphing that. Th

Re: RELENG_8 pf stack issue (state count spiraling out of control)

2011-05-03 Thread Vlad Galu
On Tue, May 3, 2011 at 11:31 AM, Vincent Hoffman wrote: > On 03/05/2011 10:16, Jeremy Chadwick wrote: > > > > Sadly I don't see a way with bsnmpd(8) to monitor things like interrupt > > usage, etc. otherwise I'd be graphing that. The more monitoring the > > better; at least then I could say "wo

Re: RELENG_8 pf stack issue (state count spiraling out of control)

2011-05-03 Thread Vincent Hoffman
On 03/05/2011 10:16, Jeremy Chadwick wrote: > Sadly I don't see a way with bsnmpd(8) to monitor things like interrupt > usage, etc. otherwise I'd be graphing that. The more monitoring the > better; at least then I could say "wow, interrupts really did shoot > through the roof -- the box went cra

Re: RELENG_8 pf stack issue (state count spiraling out of control)

2011-05-03 Thread Daniel Hartmeier
On Mon, May 02, 2011 at 06:58:54PM -0700, Jeremy Chadwick wrote: > Here's one piece of core.0.txt which makes no sense to me -- the "rate" > column. I have a very hard time believing that was the interrupt rate > of all the relevant devices at the time (way too high). Maybe this data > becomes w

Re: RELENG_8 pf stack issue (state count spiraling out of control)

2011-05-03 Thread Jeremy Chadwick
On Tue, May 03, 2011 at 10:48:00AM +0200, Daniel Hartmeier wrote: > On Mon, May 02, 2011 at 06:58:54PM -0700, Jeremy Chadwick wrote: > > > Status: Enabled for 76 days 06:49:10 Debug: Urgent > > > The "pf uptime" shown above, by the way, matches system uptime. > > > ps -axl > > > > UI

Re: RELENG_8 pf stack issue (state count spiraling out of control)

2011-05-03 Thread Daniel Hartmeier
On Mon, May 02, 2011 at 06:58:54PM -0700, Jeremy Chadwick wrote: > Status: Enabled for 76 days 06:49:10 Debug: Urgent > The "pf uptime" shown above, by the way, matches system uptime. > ps -axl > > UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND > 0 42

Re: RELENG_8 pf stack issue (state count spiraling out of control)

2011-05-03 Thread Jeremy Chadwick
On Tue, May 03, 2011 at 09:00:42AM +0200, Daniel Hartmeier wrote: > I read those graphs differently: the problem doesn't arise slowly, > but rather seems to start suddenly at 13:00. > > Right after 13:00, traffic on em0 drops, i.e. the firewall seems > to stop forwarding packets completely. > > Y

Re: RELENG_8 pf stack issue (state count spiraling out of control)

2011-05-03 Thread Daniel Hartmeier
I read those graphs differently: the problem doesn't arise slowly, but rather seems to start suddenly at 13:00. Right after 13:00, traffic on em0 drops, i.e. the firewall seems to stop forwarding packets completely. Yet, at the same time, the states start to increase, almost linearly at about one

Re: RELENG_8 pf stack issue (state count spiraling out of control)

2011-05-02 Thread Janne Snabb
On Tue, 3 May 2011, Vlad Galu wrote: > Disabling scrubbing altogether seems like a good next step. I used to get all kinds of strange problems when I tried scrubbing on FreeBSD 8.1. Especially with IPv6 traffic. After I disabled scrubbing altogether I have had zero problems. The IP & TCP stacks b

Re: RELENG_8 pf stack issue (state count spiraling out of control)

2011-05-02 Thread Vlad Galu
On Tue, May 3, 2011 at 8:01 AM, Jeremy Chadwick wrote: > On Tue, May 03, 2011 at 07:22:10AM +0200, Vlad Galu wrote: > > On Tue, May 3, 2011 at 3:58 AM, Jeremy Chadwick < > free...@jdc.parodius.com>wrote: > > > > > (Please keep me CC'd as I'm not subscribed to freebsd-pf. And > apologies > > > for

Re: RELENG_8 pf stack issue (state count spiraling out of control)

2011-05-02 Thread Jeremy Chadwick
On Tue, May 03, 2011 at 07:22:10AM +0200, Vlad Galu wrote: > On Tue, May 3, 2011 at 3:58 AM, Jeremy Chadwick > wrote: > > > (Please keep me CC'd as I'm not subscribed to freebsd-pf. And apologies > > for cross-posting, but the issue is severe enough that I wanted to make > > it known on -stable)

Re: RELENG_8 pf stack issue (state count spiraling out of control)

2011-05-02 Thread Vlad Galu
On Tue, May 3, 2011 at 3:58 AM, Jeremy Chadwick wrote: > (Please keep me CC'd as I'm not subscribed to freebsd-pf. And apologies > for cross-posting, but the issue is severe enough that I wanted to make > it known on -stable) > > The below issue I'm describing is from a machine running 8.2-PREREL

Re: RELENG_8 pf stack issue (state count spiraling out of control)

2011-05-02 Thread Jeremy Chadwick
On Tue, May 03, 2011 at 01:06:34AM -0400, Jason Hellenthal wrote: > On Mon, May 02, 2011 at 06:58:54PM -0700, Jeremy Chadwick wrote: > >(Please keep me CC'd as I'm not subscribed to freebsd-pf. And apologies > >for cross-posting, but the issue is severe enough that I wanted to make > >it known on

Re: RELENG_8 pf stack issue (state count spiraling out of control)

2011-05-02 Thread Jason Hellenthal
Jeremy, On Mon, May 02, 2011 at 06:58:54PM -0700, Jeremy Chadwick wrote: >(Please keep me CC'd as I'm not subscribed to freebsd-pf. And apologies >for cross-posting, but the issue is severe enough that I wanted to make >it known on -stable) > >The below issue I'm describing is from a machine ru

RELENG_8 pf stack issue (state count spiraling out of control)

2011-05-02 Thread Jeremy Chadwick
(Please keep me CC'd as I'm not subscribed to freebsd-pf. And apologies for cross-posting, but the issue is severe enough that I wanted to make it known on -stable) The below issue I'm describing is from a machine running 8.2-PRERELEASE (RELENG_8) using src dated Tue Feb 15 05:46:02 PST 2011. Pl