Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-20 Thread Kris Kennaway
On Mon, Jul 18, 2005 at 05:39:40PM -0400, Gary Mulder wrote: > On Mon, 18 Jul 2005, Pawel Malachowski wrote: > > > On Mon, Jul 18, 2005 at 04:09:58PM -0400, Matt Juszczak wrote: > > > > > Correct. IPF is unstable with our SMP (most of the time) - based 5.x > > > boxes. VERY unstable. VERY VER

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-20 Thread Kris Kennaway
On Wed, Jul 20, 2005 at 03:57:57PM +0200, Maciej Wierzbicki wrote: > Gary Mulder wrote on 2005-07-18 23:39: > > >From personal experience I can repeat what Matt has stated. It seems to be > >related to what NIC you have. I have had crashes with fxp (Intel Pro > >100MBit) and bge (Broadcom Gigabi

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-20 Thread Maciej Wierzbicki
Gary Mulder wrote on 2005-07-18 23:39: From personal experience I can repeat what Matt has stated. It seems to be related to what NIC you have. I have had crashes with fxp (Intel Pro 100MBit) and bge (Broadcom Gigabit) NICs under moderate network load. It seems not. I had crashes with fxp, xl

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-19 Thread Vivek Khera
On Jul 18, 2005, at 5:39 PM, Gary Mulder wrote: Another person on the freebsd-amd64 list reported similar network- related crashes until he switched to em (Intel Gigabit Ethernet) NICs. that was probably me... but I don't have any firewall on these boxes as they are not hooked up to the i

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-18 Thread dick hoogendijk
On Mon, 18 Jul 2005 14:32:09 -0400 (EDT) Matt Juszczak <[EMAIL PROTECTED]> wrote: > > For me, 5 days up time after switching from IPF to PF. Before the switch a > > couple of hours of uptime was the maximum. Seems like the crashes are > > caused > > by ipfilter. > > > Still same for me :) Up

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-18 Thread Matt Juszczak
I find this messages kind of weird. Are you saying your servers only run long periods of uptime with pf and *not* with ipf? I run a server and almost never put it down. IPF performs very well, including a lot of natting for my home network. Correct. IPF is unstable with our SMP (most of the

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-18 Thread Pawel Malachowski
On Mon, Jul 18, 2005 at 04:09:58PM -0400, Matt Juszczak wrote: > Correct. IPF is unstable with our SMP (most of the time) - based 5.x > boxes. VERY unstable. VERY VERY unstable. Hm, this sounds bad. What is debug.mpsafenet set to? How big is traffic? I have one SMP box with ipnat, routing so

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-18 Thread Matt Juszczak
For me, 5 days up time after switching from IPF to PF. Before the switch a couple of hours of uptime was the maximum. Seems like the crashes are caused by ipfilter. Still same for me :) Uptime almost 20 days now after switching to PF. ___ freebsd-st

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-18 Thread Gary Mulder
On Mon, 18 Jul 2005, Pawel Malachowski wrote: > On Mon, Jul 18, 2005 at 04:09:58PM -0400, Matt Juszczak wrote: > > > Correct. IPF is unstable with our SMP (most of the time) - based 5.x > > boxes. VERY unstable. VERY VERY unstable. > > Hm, this sounds bad. What is debug.mpsafenet set to? How

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-17 Thread Blaz Zupan
On Tue, 12 Jul 2005, Matt Juszczak wrote: So far a 13 day up time after switching from IPF to PF. If thats not the problem, I hope I find it soon considering this is a production server ... but it seems to be more stable. For me, 5 days up time after switching from IPF to PF. Before the switc

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-12 Thread Matt Juszczak
Yes, there is absolutely no difference. Disabled HTT in the BIOS and in FreeBSD, the box still crashes. Matt again :) So far a 13 day up time after switching from IPF to PF. If thats not the problem, I hope I find it soon considering this is a production server ... but it seems to be more st

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-12 Thread Maciej Wierzbicki
Blaz Zupan wrote on 2005-07-12 13:17: Interesting that the box has survived almost two days now, while it was always crashing after at least 8 hours. Anyway, I have compiled a new kernel without ipfilter, I have used pf instead (the configuration changes from ipfilter to pf were mostly minor).

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-12 Thread Blaz Zupan
Could you try SMP kernel without IPF support and without using IPF module? Could you confirm, that your SMP kernel is not crashing when you do not use IPF? Interesting that the box has survived almost two days now, while it was always crashing after at least 8 hours. Anyway, I have compiled a n

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-10 Thread Maciej Wierzbicki
On Sun, Jul 10, 2005 at 04:58:08PM +0200, Blaz Zupan wrote: > In order for this problem to not get lost on the freebsd-stable mailing > list, I have opened a PR: > > http://www.freebsd.org/cgi/query-pr.cgi?pr=83220 Could you try SMP kernel without IPF support and without using IPF module? Could

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-10 Thread Blaz Zupan
In order for this problem to not get lost on the freebsd-stable mailing list, I have opened a PR: http://www.freebsd.org/cgi/query-pr.cgi?pr=83220 ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To u

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-06 Thread Eirik Øverby
On Jul 6, 2005, at 6:29 PM, Blaz Zupan wrote: On Wed, 6 Jul 2005, Kris Kennaway wrote: That should be OK as long as you're not cross-compiling for different architectures. No, we only have i386 boxes. Hi, thanks for doing this work. I was working on preparing a similiar set of informa

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-06 Thread Blaz Zupan
On Wed, 6 Jul 2005, Kris Kennaway wrote: That should be OK as long as you're not cross-compiling for different architectures. No, we only have i386 boxes. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-s

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-06 Thread Kris Kennaway
On Wed, Jul 06, 2005 at 06:20:38PM +0200, Blaz Zupan wrote: > On Wed, 6 Jul 2005, Kris Kennaway wrote: > >Interesting, this seems to finger the TCP code. Are you compiling > >your kernel with -O2 though (this causes bogus stack frames like you > >have here)? If so, recompile with -O and try to ob

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-06 Thread Blaz Zupan
On Wed, 6 Jul 2005, Kris Kennaway wrote: Interesting, this seems to finger the TCP code. Are you compiling your kernel with -O2 though (this causes bogus stack frames like you have here)? If so, recompile with -O and try to obtain another trace. Nope, no funky compile options, all at the defa

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-06 Thread Kris Kennaway
On Wed, Jul 06, 2005 at 06:10:20PM +0200, Blaz Zupan wrote: > On Wed, 6 Jul 2005, Kris Kennaway wrote: > >Please obtain the backtrace with kgdb. > > Here you go: > #9 0xc1fa0018 in ?? () > #10 0xc2a40010 in ?? () > #11 0x0010 in ?? () > #12 0xc2216000 in ?? () > #13 0xc0686a2c in tcbinfo ()

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-06 Thread Blaz Zupan
On Wed, 6 Jul 2005, Kris Kennaway wrote: Please obtain the backtrace with kgdb. Here you go: [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free soft

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-06 Thread Kris Kennaway
On Wed, Jul 06, 2005 at 09:40:20AM +0200, Blaz Zupan wrote: > If a developer is willing to investigate, I have: > - the vmcore file from the crash (its size is 1GB) > - the corresponding kernel, compiled with debug symbols Please obtain the backtrace with kgdb. Kris pgpoFrkAp3yjc.pgp Descripti

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-06 Thread Blaz Zupan
Have you tried to disable HTT? It's doesn't give you alot, and in some cases it decreases performance. Yes, there is absolutely no difference. Disabled HTT in the BIOS and in FreeBSD, the box still crashes. ___ freebsd-stable@freebsd.org mailing lis

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-06 Thread Claus Guttesen
> I'm experiencing the same crashes as Matt, but on 5.4-RELEASE-p3. The machine > is a HP DL380 G3 and it is heavily loaded (postfix mail server running > amavisd-new with antivirus and antispam, so it has heavy IO and CPU load). It > does not survive more than a couple of hours, while it is rock s

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-06 Thread Blaz Zupan
On Fri, 1 Jul 2005, Kris Kennaway wrote: On Tue, Jun 28, 2005 at 11:26:06AM -0400, Matt Juszczak wrote: After CPUID: 1, the machine locks cold and nothing else is printed to the screen. Try two things: 1) adding 'options KDB_STOP_NMI' to your kernel config. I just learned that you also need

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-07-01 Thread Kris Kennaway
On Wed, Jun 29, 2005 at 06:05:35AM -0400, Kris Kennaway wrote: > On Tue, Jun 28, 2005 at 11:26:06AM -0400, Matt Juszczak wrote: > > > >OK, when it crashes next and is sat at the "db>" prompt, type "tr" and > > >press enter to get a trace. Copy this down (or have a serial console to > > >capture t

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-06-29 Thread Matt Juszczak
On Wed, 29 Jun 2005, Kris Kennaway wrote: On Tue, Jun 28, 2005 at 11:26:06AM -0400, Matt Juszczak wrote: OK, when it crashes next and is sat at the "db>" prompt, type "tr" and press enter to get a trace. Copy this down (or have a serial console to capture the output). Also, try typing "cal

Re: FreeBSD -STABLE servers repeatedly crashing

2005-06-29 Thread Gleb Smirnoff
On Tue, Jun 28, 2005 at 01:50:48PM -0400, Matt Juszczak wrote: M> >Please try out this patch to aid the above problem with hang instead of M> >dump: M> > M> >http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/i386/i386/trap.c.diff?r1=1.275&r2=1.276 M> This patch wouldn't go through M> I tried patch

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-06-29 Thread Kris Kennaway
On Tue, Jun 28, 2005 at 11:26:06AM -0400, Matt Juszczak wrote: > >OK, when it crashes next and is sat at the "db>" prompt, type "tr" and > >press enter to get a trace. Copy this down (or have a serial console to > >capture the output). Also, try typing "call doadump()" and see if that > >succeed

RE: FreeBSD -STABLE servers repeatedly crashing

2005-06-29 Thread Piotr Gnyp
On Tue, 28 Jun 2005, Matt Juszczak <[EMAIL PROTECTED]> wrote: Please try out this patch to aid the above problem with hang instead of dump: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/i386/i386/trap.c.diff?r1=1.275&r2=1.276 This box is now crashing once every 12 hours. I can't apply this patc

Re: FreeBSD -STABLE servers repeatedly crashing

2005-06-28 Thread Matt Juszczak
Yes, SMP is enabled, as is implied by the kernel config tag. (Very busy compilation, web and database server) Are you using PF? ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send

Re: FreeBSD -STABLE servers repeatedly crashing

2005-06-28 Thread Karl Denninger
FreeBSD 5.4-STABLE #11: Fri Apr 8 09:48:24 CDT 2005 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/KSD-SMP 5:02PM up 80 days, 21:08, 1 user, load averages: 4.04, 3.33, 3.01 Yes, SMP is enabled, as is implied by the kernel config tag. (Very busy compilation, web and database server) -- -- Karl

Re: FreeBSD -STABLE servers repeatedly crashing

2005-06-28 Thread Matt Juszczak
Some people suggested so - pf is supposed to be faster then IPFILTER. However if you are experiencing machine freezing like I did on 5.4-STABLE I'm not sure this will help - if nothing else helps try 6.0-CURRENT. I've also noticed that it is running much faster with all debuging enabled then reg

Re: FreeBSD -STABLE servers repeatedly crashing

2005-06-28 Thread Goran Gajic
Some people suggested so - pf is supposed to be faster then IPFILTER. However if you are experiencing machine freezing like I did on 5.4-STABLE I'm not sure this will help - if nothing else helps try 6.0-CURRENT. I've also noticed that it is running much faster with all debuging enabled then r

Re: FreeBSD -STABLE servers repeatedly crashing

2005-06-28 Thread Matt Juszczak
Hi, I have something like 20 boxes (Dell Power Edge 370, Fujitsu-Siemens PRIMERGY 200 and couple of dual AMD64 Fujitsu-Siemens) servers running 5.4-STABLE. So far, only machine that I have experienced freezing and was unable to get droped into KDB or to get any sort of vmcore was Dell Power E

Re: FreeBSD -STABLE servers repeatedly crashing

2005-06-28 Thread Goran Gajic
Hi, I have something like 20 boxes (Dell Power Edge 370, Fujitsu-Siemens PRIMERGY 200 and couple of dual AMD64 Fujitsu-Siemens) servers running 5.4-STABLE. So far, only machine that I have experienced freezing and was unable to get droped into KDB or to get any sort of vmcore was Dell Power

Re: FreeBSD -STABLE servers repeatedly crashing

2005-06-28 Thread Matt Juszczak
Only way to find out is to try. You could build and install the non-SMP kernel and reboot when you can, or let it boot the new kernel next time the system(s) crash. A lot of the issues seem to be SMP-related. I really loaded up a GENERIC 5.4 kernel and wasn't able to get it to panic. What do you

Re: FreeBSD -STABLE servers repeatedly crashing

2005-06-28 Thread Gary Mu1der
Gary, Do you know what the chances are that this problem I'm experiencing is SMP related? I don't mind turning off SMP, and I guess I could for now to see if that runs stable. Otherwise, I think we're going to switch to OpenBSD, because these crashes are occuring so frequently (twice a day)...

Re: FreeBSD -STABLE servers repeatedly crashing

2005-06-28 Thread Matt Juszczak
Matt, Sadly the FreeBSD guys will need more info before a fix is possible. I would suggest you revert back to FreeBSD 5.3, if you can. Even if you get a patch you'd want to do a whole lot of regression testing before putting it in production as it might break something else. Gary, Do you kn

RE: FreeBSD -STABLE servers repeatedly crashing

2005-06-28 Thread Matt Juszczak
Please try out this patch to aid the above problem with hang instead of dump: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/i386/i386/trap.c.diff?r1=1.275&r2=1.276 This box is now crashing once every 12 hours. I can't apply this patch :-(. Does anyone have any suggestions on how I can work a

RE: FreeBSD -STABLE servers repeatedly crashing

2005-06-28 Thread Matt Juszczak
Please try out this patch to aid the above problem with hang instead of dump: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/i386/i386/trap.c.diff?r1=1.275&r2=1.276 This patch wouldn't go through I tried patching against: __FBSDID("$FreeBSD: src/sys/i386/i386/trap.c,v 1.267.2.3 2005/05/

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-06-28 Thread Matt Juszczak
fsck -y# or fsck and read every question, if you're paranoid mount -f /# remounts root read/write mount /var savecore /var/crash exit Gary Gary: After it crashes, it locks up and hangs, no keyboard response, etc. When I reboot, I go into single user mode and do: fsck -p moun

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-06-28 Thread Gary Mu1der
Matt Juszczak wrote: Ever since I setup the debug kernel the machine is now crashing every 12 hours. I think I have to switch to OpenBSD or 4.11 FreeBSD because this box can't keep crashing. It refuses to do a crash dump. -Matt Matt, Does it refuse to crash dump or is it that you can't g

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-06-28 Thread Matt Juszczak
Gavin Atkinson wrote: On Tue, 2005-06-28 at 10:49 -0400, Matt Juszczak wrote: Gleb Smirnoff wrote: On Mon, Jun 27, 2005 at 01:01:09AM -0400, Matt Juszczak wrote: M> About three weeks ago, I upgraded my 5.3-RELEASE boxes to 5.4-RELEASE. M> I also turned on procmail globally on our ma

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-06-28 Thread Gavin Atkinson
On Tue, 2005-06-28 at 10:49 -0400, Matt Juszczak wrote: > Gleb Smirnoff wrote: > > >On Mon, Jun 27, 2005 at 01:01:09AM -0400, Matt Juszczak wrote: > >M> About three weeks ago, I upgraded my 5.3-RELEASE boxes to 5.4-RELEASE. > >M> I also turned on procmail globally on our mail server. Here is ou

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-06-28 Thread Matt Juszczak
Gleb Smirnoff wrote: On Mon, Jun 27, 2005 at 01:01:09AM -0400, Matt Juszczak wrote: M> About three weeks ago, I upgraded my 5.3-RELEASE boxes to 5.4-RELEASE. M> I also turned on procmail globally on our mail server. Here is our M> current FreeBSD server setup: M> M> URANUS - primary ldap

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-06-28 Thread Gleb Smirnoff
On Mon, Jun 27, 2005 at 07:58:18PM -0400, Matt Juszczak wrote: M> >Can you please build kernel with debugging and obtain a crashdump? M> M> High activity on the box today caused us to be able to crash it again M> within 9 hours. I configured all steps per the developers handbook, but M> when I

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-06-27 Thread Matt Juszczak
Can you please build kernel with debugging and obtain a crashdump? High activity on the box today caused us to be able to crash it again within 9 hours. I configured all steps per the developers handbook, but when I went to do savecore, it said "no dumps". It appears the machine is complet

Re: FreeBSD -STABLE servers repeatedly crashing.

2005-06-27 Thread Gleb Smirnoff
On Mon, Jun 27, 2005 at 01:01:09AM -0400, Matt Juszczak wrote: M> About three weeks ago, I upgraded my 5.3-RELEASE boxes to 5.4-RELEASE. M> I also turned on procmail globally on our mail server. Here is our M> current FreeBSD server setup: M> M> URANUS - primary ldap M> CALIBAN - secondary