Re: panics collections on FreeBSD 11.0-RC1 RC2 PRERELEASE RELEASE STABLE
Next panic uname -a FreeBSD ns 11.0-STABLE FreeBSD 11.0-STABLE #1 r310470: Fri Dec 23 12:00:49 EET 2016 root@ns:/usr/obj/usr/src/sys/bzk11 amd64 Dec 28 09:43:21 ns kernel: Fatal trap 12: page fault while in kernel mode Dec 28 09:43:21 ns kernel: cpuid = 3; apic id = 03 Dec 28 09:43:21 ns kernel: fault virtual address= 0x8 Dec 28 09:43:21 ns kernel: fault code = supervisor read data, page not present Dec 28 09:43:21 ns kernel: instruction pointer = 0x20:0x80bb0090 Dec 28 09:43:21 ns kernel: stack pointer= 0x28:0xfe083b7fc560 Dec 28 09:43:21 ns kernel: frame pointer= 0x28:0xfe083b7fc590 Dec 28 09:43:21 ns kernel: code segment = base 0x0, limit 0xf, type 0x1b Dec 28 09:43:21 ns kernel: = DPL 0, pres 1, long 1, def32 0, gran 1 Dec 28 09:43:21 ns kernel: processor eflags = interrupt enabled, resume, IOPL = 0 Dec 28 09:43:21 ns kernel: current process = 12 (swi5: fast taskq) Dec 28 09:43:21 ns kernel: trap number = 12 Dec 28 09:43:21 ns kernel: panic: page fault Dec 28 09:43:21 ns kernel: cpuid = 3 Dec 28 09:43:21 ns kernel: KDB: stack backtrace: Dec 28 09:43:21 ns kernel: #0 0x80b5f7e7 at kdb_backtrace+0x67 Dec 28 09:43:21 ns kernel: #1 0x80b131e2 at vpanic+0x182 Dec 28 09:43:21 ns kernel: #2 0x80b13053 at panic+0x43 Dec 28 09:43:21 ns kernel: #3 0x8105c230 at trap_fatal+0x330 Dec 28 09:43:21 ns kernel: #4 0x8105c423 at trap_pfault+0x1e3 Dec 28 09:43:21 ns kernel: #5 0x8105ba24 at trap+0x274 Dec 28 09:43:21 ns kernel: #6 0x8103dbb1 at calltrap+0x8 Dec 28 09:43:21 ns kernel: #7 0x80d5492a at tcp_do_segment+0x1a1a Dec 28 09:43:21 ns kernel: #8 0x80d52669 at tcp_input+0x1559 Dec 28 09:43:21 ns kernel: #9 0x80cb6dee at ip_input+0x18e Dec 28 09:43:21 ns kernel: #10 0x80c4aead at netisr_dispatch_src+0xad Dec 28 09:43:21 ns kernel: #11 0x80c325ba at ether_demux+0x13a Dec 28 09:43:21 ns kernel: #12 0x82b6871c at vboxNetFltFreeBSDinput+0x27c Dec 28 09:43:21 ns kernel: #13 0x80b7308a at taskqueue_run_locked+0x14a Dec 28 09:43:21 ns kernel: #14 0x80b72e7f at taskqueue_run+0xbf Dec 28 09:43:21 ns kernel: #15 0x80acbcff at intr_event_execute_handlers+0x20f Dec 28 09:43:21 ns kernel: #16 0x80acbf66 at ithread_loop+0xc6 Dec 28 09:43:21 ns kernel: #17 0x80ac8745 at fork_exit+0x85 Dec 28 09:43:21 ns kernel: Uptime: 4d14h37m22s Dec 28 09:43:21 ns kernel: Dumping 8226 out of 32688 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%Copyright (c) 1992-2016 The FreeBSD Project. ___ freebsd-bugs@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-bugs To unsubscribe, send any mail to "freebsd-bugs-unsubscr...@freebsd.org"
[Bug 215613] [panic] if if_ixl due to NULL pointer dereference
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=215613 Andrey V. Elsukov changed: What|Removed |Added Assignee|freebsd-bugs@FreeBSD.org|freebsd-...@freebsd.org Keywords||IntelNetworking -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-bugs@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-bugs To unsubscribe, send any mail to "freebsd-bugs-unsubscr...@freebsd.org"
[Bug 215634] zfs receive trips up and live-locks for non-incremental fs
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=215634
Bug ID: 215634
Summary: zfs receive trips up and live-locks for
non-incremental fs
Product: Base System
Version: 10.3-STABLE
Hardware: amd64
OS: Any
Status: New
Severity: Affects Only Me
Priority: ---
Component: kern
Assignee: freebsd-bugs@FreeBSD.org
Reporter: johan...@jo-t.de
CC: freebsd-am...@freebsd.org
CC: freebsd-am...@freebsd.org
Hi,
when I'm trying to zfs-send a filesystem from one machine to another,
the receiving end gets stuck with zfskern spinning one CPU core.
No observable problems sending incremental streams.
Here's what top shows (truncated) on the receiver:
> last pid: 2848; load averages: 1.08, 1.08, 1.04
> 243 processes: 5 running, 220 sleeping, 18 waiting
> CPU: 0.0% user, 0.0% nice, 50.6% system, 0.0% interrupt, 49.4% idle
> Mem: 8904K Active, 77M Inact, 551M Wired, 5310M Free
> ARC: 284M Total, 104M MFU, 46M MRU, 2448K Anon, 1979K Header, 131M Other
> Swap: 8192M Total, 8192M Free
> PID USERNAME PRI NICE SIZERES STATE C TIMEWCPU COMMAND
> 5 root-8- 0K 128K CPU00 252:00 100.00%
> zfskern{solthread 0x}
>11 root 155 ki31 0K32K RUN 1 225:09 91.89% idle{idle:
> cpu1}
>11 root 155 ki31 0K32K RUN 0 51:02 7.86% idle{idle:
> cpu0}
> 0 root -16- 0K 2496K swapin 1 0:18 0.00%
> kernel{swapper}
>16 root16- 0K16K syncer 1 0:16 0.00% syncer
>12 root -92- 0K 288K WAIT1 0:09 0.00% intr{irq257:
> virtio_p}
>12 root -60- 0K 288K WAIT1 0:07 0.00% intr{swi4:
> clock}
>15 root -16- 0K16K vlruwt 1 0:02 0.00% vnlru
> 6 root -16- 0K32K psleep 1 0:02 0.00%
> pagedaemon{pagedaemon}
>14 root -16- 0K16K RUN 1 0:02 0.00% rand_harvestq
> 5 root-8- 0K 128K tx->tx 1 0:02 0.00%
> zfskern{txg_thread_enter}
> 1806 root400 44420K 3692K rwa.cv 1 0:01 0.00% zfs
And here's procstat for zfskern on the receiver:
> #procstat -kk 5
> PIDTID COMM TDNAME KSTACK
> 5 100044 zfskern arc_reclaim_thre mi_switch+0xe1
> sleepq_timedwait+0x3a _cv_timedwait_sbt+0x19e arc_reclaim_thread+0x2be
> fork_exit+0x9a fork_trampoline+0xe
> 5 100045 zfskern arc_user_evicts_ mi_switch+0xe1
> sleepq_timedwait+0x3a _cv_timedwait_sbt+0x19e arc_user_evicts_thread+0x17d
> fork_exit+0x9a fork_trampoline+0xe
> 5 100046 zfskern l2arc_feed_threa mi_switch+0xe1
> sleepq_timedwait+0x3a _cv_timedwait_sbt+0x19e l2arc_feed_thread+0xc73
> fork_exit+0x9a fork_trampoline+0xe
> 5 100322 zfskern trim seppel mi_switch+0xe1
> sleepq_timedwait+0x3a _cv_timedwait_sbt+0x19e trim_thread+0x126
> fork_exit+0x9a fork_trampoline+0xe
> 5 100334 zfskern txg_thread_enter mi_switch+0xe1
> sleepq_wait+0x3a _cv_wait+0x17d txg_quiesce_thread+0x16b fork_exit+0x9a
> fork_trampoline+0xe
> 5 100335 zfskern txg_thread_enter mi_switch+0xe1
> sleepq_timedwait+0x3a _cv_timedwait_sbt+0x19e txg_sync_thread+0x160
> fork_exit+0x9a fork_trampoline+0xe
> 5 100425 zfskern solthread 0x
The sender is running (custom trimmed-down GENERIC kernel):
> FreeBSD XXX 10.3-STABLE FreeBSD 10.3-STABLE #1 r308740: Sat Nov 19 21:15:27
> GMT 2016 root@XXX:/usr/obj/usr/src/sys/XXX amd64
And the receiver is running (a differently trimmed GENERIC kernel):
> FreeBSD YYY 10.3-RELEASE-p15 FreeBSD 10.3-RELEASE-p15 #9 r310507: Sat Dec 24
> 21:22:15 UTC 2016 root@XXX:/path/usr/src/sys/YYY amd64
The file systems that zfs-send just fine are clones of snapshots. These
are send as incremental streams. The problematic one is a fresh
zfs-create'd file system, with only a few small files in it.
The command used to send/receive, initiated from the sender-side, is:
> /sbin/zfs send -v -R senderpool/myrootfs | /usr/bin/gzip | /usr/bin/ssh
> root@${HOST} "/usr/bin/gunzip | /sbin/zfs recv -v -ud receiverpool"
And what I thus get in the console (from zfs recv -v) is (trimmed):
> found clone origin receiverpool/base@20.46-r310507
> receiving incremental stream of senderpool/myrootfs@clean-install into
> receiverpool/myrootfs@clean-install
> received 886KB stream in 3 seconds (295KB/sec)
> receiving full stream of senderpool/myrootfs/freshfs@clean-install into
> receiverpool/myrootfs/freshfs@clean-install
> [...stuck here...]
The receiving end seems to be running fine with zfskern spinning. But it
will never finish the the filesystem in question.
Any ideas what might be going on, or what to do about it?
Thanks,
Johannes
--
You are receiving this mail becaus
[Bug 215635] LOR in zfs
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=215635 Bug ID: 215635 Summary: LOR in zfs Product: Base System Version: 10.3-RELEASE Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: kern Assignee: freebsd-bugs@FreeBSD.org Reporter: d...@freebsd.org I saw this in my logs this morning. This server is the same one which has been locking up, most nights, at about 0301 UTC. Dec 28 03:12:20 knew kernel: lock order reversal: Dec 28 03:12:20 knew kernel: 1st 0xf80418bea068 zfs (zfs) @ /usr/src/sys/kern/vfs_extattr.c:171 Dec 28 03:12:20 knew kernel: 2nd 0xf803f973c040 filedesc structure (filedesc structure) @ /usr/src/sys/kern/vfs_lookup.c:213 Dec 28 03:12:20 knew kernel: KDB: stack backtrace: Dec 28 03:12:20 knew kernel: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfe03b95f2fa0 Dec 28 03:12:20 knew kernel: kdb_backtrace() at kdb_backtrace+0x39/frame 0xfe03b95f3050 Dec 28 03:12:20 knew kernel: witness_checkorder() at witness_checkorder+0xc7e/frame 0xfe03b95f30e0 Dec 28 03:12:20 knew kernel: _sx_slock() at _sx_slock+0x46/frame 0xfe03b95f3120 Dec 28 03:12:20 knew kernel: namei() at namei+0x19a/frame 0xfe03b95f31f0 Dec 28 03:12:20 knew kernel: vn_open_cred() at vn_open_cred+0x10d/frame 0xfe03b95f3350 Dec 28 03:12:20 knew kernel: zfs_setextattr() at zfs_setextattr+0x204/frame 0xfe03b95f3670 Dec 28 03:12:20 knew kernel: VOP_SETEXTATTR_APV() at VOP_SETEXTATTR_APV+0xa7/frame 0xfe03b95f36a0 Dec 28 03:12:20 knew kernel: extattr_set_vp() at extattr_set_vp+0x153/frame 0xfe03b95f3770 Dec 28 03:12:20 knew kernel: sys_extattr_set_file() at sys_extattr_set_file+0xf4/frame 0xfe03b95f39a0 Dec 28 03:12:20 knew kernel: amd64_syscall() at amd64_syscall+0x2d4/frame 0xfe03b95f3ab0 Dec 28 03:12:20 knew kernel: Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfe03b95f3ab0 Dec 28 03:12:20 knew kernel: --- syscall (356, FreeBSD ELF64, sys_extattr_set_file), rip = 0x80086637a, rsp = 0x7fffe668, rbp = 0x7fffe700 --- -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-bugs@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-bugs To unsubscribe, send any mail to "freebsd-bugs-unsubscr...@freebsd.org"
[Bug 215635] LOR in zfs
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=215635 --- Comment #1 from Dan Langille --- In case it's relevant: The server has 3x LSI SAS 2008 cards. It boots off zfs, a 10-drive raidz2. There is a second 10-drive raidz3. 8 drives from one zpool are all on one HBA. 8 drives from the other zpool are on another HBA. Two drives from each zpool are on a third HBA (i.e. that HBA has two drives from one pool and two drives from the other). The zroot zpool has been around for several years, but was recently moved from one box to another, getting a new M/B & new HBAs. Shortly afterwards, the raidz3 was added to the box. Since that addition, the system has frozen most nights at about 0301, more or less. The host has 17 jails, all of which have stock /etc/crontab entries (i.e. daily periodic runs). By freeze, I mean: - ssh to the server fails to connect - login at console accepts password but does not present prompt, but responds to CTL-T (https://twitter.com/DLangille/status/804738019857199105) - Backups jobs which used this server as the destination, continued to work. - nagios flips out over the services provided by the jails/host - postfix stops responding (both host and jails) - tail -F /var/log/messages : in an existing ssh session continued to stream entries refs: * https://twitter.com/DLangille/media (lots of screen shots0 * https://twitter.com/search?q=%23frozenserver&src=typd * http://dan.langille.org/2016/12/09/system-freezes-up-with-lots-of-sonewconn-listen-queue-overflow/ * http://dan.langille.org/2016/12/14/server-freeze-2014-12-14/ -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-bugs@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-bugs To unsubscribe, send any mail to "freebsd-bugs-unsubscr...@freebsd.org"
[Bug 215635] LOR in zfs
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=215635 --- Comment #2 from Dan Langille --- Last night, I invoked the server freeze by doing a 'tar -czf'. I took a dump from that, and will provide upon request. See the output of 'show lockedvnods' here: https://twitter.com/DLangille/status/813870668513153024 -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-bugs@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-bugs To unsubscribe, send any mail to "freebsd-bugs-unsubscr...@freebsd.org"
Re: panics collections on FreeBSD 11.0-RC1 RC2 PRERELEASE RELEASE STABLE
- stable@ On 12/28/16 at 09:58P, Ivan Klymenko wrote: > Next panic > uname -a > FreeBSD ns 11.0-STABLE FreeBSD 11.0-STABLE #1 r310470: Fri Dec 23 12:00:49 > EET 2016 root@ns:/usr/obj/usr/src/sys/bzk11 amd64 > > Dec 28 09:43:21 ns kernel: Fatal trap 12: page fault while in kernel mode > Dec 28 09:43:21 ns kernel: cpuid = 3; apic id = 03 > Dec 28 09:43:21 ns kernel: fault virtual address= 0x8 > Dec 28 09:43:21 ns kernel: fault code = supervisor read data, page > not present > Dec 28 09:43:21 ns kernel: instruction pointer = 0x20:0x80bb0090 > Dec 28 09:43:21 ns kernel: stack pointer= > 0x28:0xfe083b7fc560 > Dec 28 09:43:21 ns kernel: frame pointer= > 0x28:0xfe083b7fc590 > Dec 28 09:43:21 ns kernel: code segment = base 0x0, limit 0xf, > type 0x1b > Dec 28 09:43:21 ns kernel: = DPL 0, pres 1, long 1, def32 0, gran 1 > Dec 28 09:43:21 ns kernel: processor eflags = interrupt enabled, resume, > IOPL = 0 > Dec 28 09:43:21 ns kernel: current process = 12 (swi5: fast > taskq) > Dec 28 09:43:21 ns kernel: trap number = 12 > Dec 28 09:43:21 ns kernel: panic: page fault > Dec 28 09:43:21 ns kernel: cpuid = 3 > Dec 28 09:43:21 ns kernel: KDB: stack backtrace: > Dec 28 09:43:21 ns kernel: #0 0x80b5f7e7 at kdb_backtrace+0x67 > Dec 28 09:43:21 ns kernel: #1 0x80b131e2 at vpanic+0x182 > Dec 28 09:43:21 ns kernel: #2 0x80b13053 at panic+0x43 > Dec 28 09:43:21 ns kernel: #3 0x8105c230 at trap_fatal+0x330 > Dec 28 09:43:21 ns kernel: #4 0x8105c423 at trap_pfault+0x1e3 > Dec 28 09:43:21 ns kernel: #5 0x8105ba24 at trap+0x274 > Dec 28 09:43:21 ns kernel: #6 0x8103dbb1 at calltrap+0x8 > Dec 28 09:43:21 ns kernel: #7 0x80d5492a at tcp_do_segment+0x1a1a > Dec 28 09:43:21 ns kernel: #8 0x80d52669 at tcp_input+0x1559 > Dec 28 09:43:21 ns kernel: #9 0x80cb6dee at ip_input+0x18e > Dec 28 09:43:21 ns kernel: #10 0x80c4aead at netisr_dispatch_src+0xad > Dec 28 09:43:21 ns kernel: #11 0x80c325ba at ether_demux+0x13a > Dec 28 09:43:21 ns kernel: #12 0x82b6871c at > vboxNetFltFreeBSDinput+0x27c > Dec 28 09:43:21 ns kernel: #13 0x80b7308a at > taskqueue_run_locked+0x14a > Dec 28 09:43:21 ns kernel: #14 0x80b72e7f at taskqueue_run+0xbf > Dec 28 09:43:21 ns kernel: #15 0x80acbcff at > intr_event_execute_handlers+0x20f > Dec 28 09:43:21 ns kernel: #16 0x80acbf66 at ithread_loop+0xc6 > Dec 28 09:43:21 ns kernel: #17 0x80ac8745 at fork_exit+0x85 > Dec 28 09:43:21 ns kernel: Uptime: 4d14h37m22s > Dec 28 09:43:21 ns kernel: Dumping 8226 out of 32688 > MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%Copyright (c) 1992-2016 > The FreeBSD Project. Can you open a bug report at https://bugs.freebsd.org/bugzilla/ with necessary details? Looks like virtualbox is involved here? Cheers, Hiren pgphODXjsD89z.pgp Description: PGP signature
Re: panics collections on FreeBSD 11.0-RC1 RC2 PRERELEASE RELEASE STABLE
On Wed, 28 Dec 2016 09:41:42 -0800 hiren panchasara wrote: > Can you open a bug report at https://bugs.freebsd.org/bugzilla/ with > necessary details? Looks like virtualbox is involved here? I can not, because that does not make sense. PR are not considered for months and years. It was the last hope for the mail lists. Probably i must to change the operating system. ___ freebsd-bugs@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-bugs To unsubscribe, send any mail to "freebsd-bugs-unsubscr...@freebsd.org"
Re: panics collections on FreeBSD 11.0-RC1 RC2 PRERELEASE RELEASE STABLE
On 12/28/16 at 07:53P, Ivan Klymenko wrote: > On Wed, 28 Dec 2016 09:41:42 -0800 > hiren panchasara wrote: > > > Can you open a bug report at https://bugs.freebsd.org/bugzilla/ with > > necessary details? Looks like virtualbox is involved here? > > I can not, because that does not make sense. > PR are not considered for months and years. > It was the last hope for the mail lists. > Probably i must to change the operating system. Apologies for that. As you know people get busy in such opensource projects. The panic appears in tcp land so if you provide enough information, I'll try to look at it. Again, I cannot promise anything more than an honest attempt. Cheers, Hiren pgp5wAY0jSwgO.pgp Description: PGP signature
