Re: FreeBSD 9.1-RELEASE crashes almost daily; backtraces always list zfs routines
on 25/12/2012 02:11 Derek Kulinski said the following:
> Hello Andriy,
>
> Monday, December 24, 2012, 3:28:00 PM, you wrote:
>
>> I've looked through the cores and it does look like in all cases some sort of
>> memory corruption is a precursor to a subsequent crash.
>
>> I can't decidedly say if the corruptions are caused by the hardware, by some
>> code overwriting random memory locations ("rogue" driver) or by a "simpler"
>> bug
>> like use after free.
>
>> I am always inclined to suspect the hardware first.
>
>> You can try to reproduce the problem with some additional checks enabled in
>> the
>> kernel. Those should catch the problem earlier and thus make its source
>> clearer.
>
>> I recommend the following:
>> options INVARIANTS
>> options INVARIANT_SUPPORT
>> options WITNESS
>> options DEBUG_MEMGUARD
>> makeoptions DEBUG+="-DDEBUG"
>
>> The last is really needed only for the ZFS and OpenSolaris compat code. It
>> make
>> result in some extra noise from unrelated subsystems.
>> Perhaps you could just add "#define DEBUG" to
>> sys/cddl/contrib/opensolaris/uts/common/sys/debug.h. I haven't tested this
>> approach though.
>
>> Also, please put vm.memguard.desc="arc_buf_hdr_t" into loader.conf.
>
>> Please note that these options will make your system significantly slower.
>
> I recompiled the kernel and is running with options you specified (I
> enabled DEBUG in the file).
>
> Anyway even at boot time I started getting following warnings, is this
> anything:
These witness warning are OK-ish.
Watch for panics.
BTW, I should have said this earlier. Whatever the kind of the corruptions it
would be much worse if a corruption would get propagated to the stable storage.
Especially if it would be in any kind of pool metadata.
So, your data is at great risk now.
Please also take measures to back it up. Preferably by using a different
system.
> Dec 24 16:06:03 chinatsu kernel: Creating and/or trimming log files
> Dec 24 16:06:03 chinatsu kernel: lock order reversal:
> Dec 24 16:06:03 chinatsu kernel: 1st 0x80bf5780 pf task mtx (pf task
> mtx) @ /usr/src/sys/contrib/pf/net/pf.c:3330
> Dec 24 16:06:03 chinatsu kernel: .
> Dec 24 16:06:03 chinatsu kernel: 2nd 0xfe0009211af8 radix node head
> (radix node head) @ /usr/src/sys/net/route.c:384
> Dec 24 16:06:03 chinatsu kernel: KDB: stack backtrace:
> Dec 24 16:06:03 chinatsu kernel: db_trace_self_wrapper() at
> db_trace_self_wrapper+0x2a
> Dec 24 16:06:03 chinatsu kernel: kdb_backtrace() at kdb_backtrace+0x37
> Dec 24 16:06:03 chinatsu kernel: _witness_debugger() at _witness_debugger+0x2c
> Dec 24 16:06:03 chinatsu kernel: witness_checkorder() at
> witness_checkorder+0x844
> Dec 24 16:06:03 chinatsu kernel: _rw_rlock() at
> Dec 24 16:06:03 chinatsu kernel: Starting syslogd.
> Dec 24 16:06:03 chinatsu kernel: _rw_rlock+0x81
> Dec 24 16:06:03 chinatsu kernel: rtalloc1_fib() at rtalloc1_fib+0x11c
> Dec 24 16:06:03 chinatsu kernel: rtalloc_ign_fib() at rtalloc_ign_fib+0xc5
> Dec 24 16:06:03 chinatsu kernel: pf_routable() at pf_routable+0x1fd
> Dec 24 16:06:03 chinatsu kernel: pf_test_rule() at pf_test_rule+0x6cf
> Dec 24 16:06:03 chinatsu kernel: pf_test() at pf_test+0xf58
> Dec 24 16:06:03 chinatsu kernel: pf_check_in() at pf_check_in+0x2b
> Dec 24 16:06:03 chinatsu kernel: pfil_run_hooks() at pfil_run_hooks+0xd2
> Dec 24 16:06:03 chinatsu kernel: ip_input() at ip_input+0x2dc
> Dec 24 16:06:03 chinatsu kernel: netisr_dispatch_src() at
> netisr_dispatch_src+0x170
> Dec 24 16:06:03 chinatsu kernel: ether_demux() at ether_demux+0x17d
> Dec 24 16:06:03 chinatsu kernel: ether_nh_input() at ether_nh_input+0x209
> Dec 24 16:06:03 chinatsu kernel: netisr_dispatch_src() at
> netisr_dispatch_src+0x170
> Dec 24 16:06:03 chinatsu kernel: alc_int_task() at alc_int_task+0x2ff
> Dec 24 16:06:03 chinatsu kernel: taskqueue_run_locked() at
> taskqueue_run_locked+0x93
> Dec 24 16:06:03 chinatsu kernel: taskqueue_thread_loop() at
> taskqueue_thread_loop+0x3e
> Dec 24 16:06:03 chinatsu kernel: fork_exit() at fork_exit+0x133
> Dec 24 16:06:03 chinatsu kernel: fork_trampoline() at fork_trampoline+0xe
> Dec 24 16:06:03 chinatsu kernel: --- trap 0, rip = 0, rsp =
> 0xff85fb2ebbb0, rbp = 0 ---
> Dec 24 16:06:03 chinatsu kernel: No core dumps found.
> Dec 24 16:06:04 chinatsu kernel: lock order reversal:
> Dec 24 16:06:04 chinatsu kernel: 1st 0xff85b9cb8dd8 bufwait (bufwait) @
> /usr/src/sys/kern/vfs_bio.c:2677
> Dec 24 16:06:04 chinatsu kernel: 2nd 0xfe00092c5c00 dirhash (dirhash) @
> /usr/src/sys/ufs/ufs/ufs_dirhash.c:284
> Dec 24 16:06:04 chinatsu kernel: KDB: stack backtrace:
> Dec 24 16:06:04 chinatsu kernel: db_trace_self_wrapper() at
> db_trace_self_wrapper+0x2a
> Dec 24 16:06:04 chinatsu kernel: kdb_backtrace() at kdb_backtrace+0x37
> Dec 24 16:06:04 chinatsu kernel: _witness_debugger() at _witness_debugger+0x2c
> Dec 24 16:06:04 chinatsu kernel: witness_checkorder() at
> witness_checkorder+0x
Re: 9.1 minimal ram requirements
So, it's fine and recommended to remove ctl device from kernel? I installed from image on the site two weeks ago and consider it release. Zoran ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: 9.1 minimal ram requirements
I always considered FreeBSD to be the most unabiguous, straightforward and sometimes even raw, but still extremely powerful and innovative, operating system out there. Seeing 9.1-RELEASE instead 9.1-PRERELASE or 9.1-RC4 is also a bad suprise for me... I have fallback into RC3 and I think the RELEASE files should be removed as well because there was no RELEASE yet. There is no rush to get RELEASE, I am sure it is better to get RC4, RC5, RC6 and then a RELEASE "when its ready". When people get buggy and unstable RELEASE and install it on a production systems they will neither come back to the system nor even support it in future :-( -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: 9.1 minimal ram requirements
On Tue, Dec 25, 2012 at 6:41 PM, Kimmo Paasiala wrote: > If it works for 99.99% percent of the users and fails for the > remaining miniscule percentage because they have a very peculiar > hardware, very small amount of ram etc, should the release called > buggy and unstable? I really don't think so. Hey hey :-) Its rather a matter of organization, not to rush towards a release (see "do we get 9.1 before christmas"), if there are known issues (see security, etc). I also started to use RC myself as I found some stuff suprising on 9.0. But when I consider someone to use FreeBSD while there are release made before release, or rarely used stuff added by default that takes 1000% of standard kernel RAM usage, or similar - this does not look serious, this makes people think "i will use linux, things like this happens there all the time but i have more drivers", etc, etc. Even for FreeBSD enhousiast it is hard to discuss with people on better organization of FreeBSD over Linux in that case. From what I read there are people working hard to make a release, but we should not rush them at cost of quality. I am still with FreeBSD and I really like it more than Linux, this is why I think quality is more important than bleeding-edge here :-) Best regards :-) Tomek -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: FreeBSD 9.1-RELEASE crashes almost daily; backtraces always list zfs routines
Hello Andriy, Tuesday, December 25, 2012, 12:18:23 AM, you wrote: >> I recompiled the kernel and is running with options you specified (I >> enabled DEBUG in the file). >> >> Anyway even at boot time I started getting following warnings, is this >> anything: > These witness warning are OK-ish. > Watch for panics. Sadly (I guess :) I did not have crash today (I don't see even a warning at that time), I'll update when it happens. Will also try to run that task today and will see if it will do anything. > BTW, I should have said this earlier. Whatever the kind of the corruptions it > would be much worse if a corruption would get propagated to the stable > storage. > Especially if it would be in any kind of pool metadata. > So, your data is at great risk now. > Please also take measures to back it up. Preferably by using a different > system. I'm backing it up using zfs send since dump doesn't appear to work on ZFS. Anyway zpool scrub does not show any problems... I performed my last scan when I started this thread. -- Best regards, Derekmailto:tak...@takeda.tk Hand over the calculator, friends don't let friends derive drunk ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
