https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=236480
Bug ID: 236480
Summary: [zfs] [panic] solaris assert: dn->dn_type !=
DMU_OT_NONE
Product: Base System
Version: 12.0-RELEASE
Hardware: amd64
OS: Any
Status: New
Severity: Affects Only Me
Priority: ---
Component: kern
Assignee: b...@freebsd.org
Reporter: ncrog...@gmail.com
This is related to bug #226130.
The following panic happens with INVARIANTS enabled. It has happened more than
once but is difficult to reproduce reliably. The affected dataset is home to a
PostgreSQL database.
courtland# kgdb /boot/kernel/kernel /var/crash/vmcore.0
GNU gdb (GDB) 8.2.1 [GDB v8.2.1 for FreeBSD]
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-portbld-freebsd12.0".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /boot/kernel/kernel...Reading symbols from
/usr/lib/debug//boot/kernel/kernel.debug...done.
done.
Unread portion of the kernel message buffer:
panic: solaris assert: dn->dn_type != DMU_OT_NONE, file:
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c, line: 2540
cpuid = 1
time = 1552104873
KDB: stack backtrace:
#0 0xffffffff80d3bf77 at kdb_backtrace+0x67
#1 0xffffffff80cf4a53 at vpanic+0x1a3
#2 0xffffffff80cf4833 at panic+0x43
#3 0xffffffff803bbcba at assfail+0x1a
#4 0xffffffff803f094b at dbuf_create+0x9b
#5 0xffffffff803f08a1 at dbuf_create_bonus+0x71
#6 0xffffffff803f8d10 at dmu_bonus_hold+0xb0
#7 0xffffffff804cde29 at zfs_zget+0x69
#8 0xffffffff804bff37 at zfs_get_data+0x67
#9 0xffffffff804d3faf at zil_commit_impl+0x15af
#10 0xffffffff804c73bb at zfs_freebsd_fsync+0xcb
#11 0xffffffff813861be at VOP_FSYNC_APV+0x9e
#12 0xffffffff80dab2d4 at vop_stdfdatasync+0x34
#13 0xffffffff8138a84e at VOP_FDATASYNC_APV+0x9e
#14 0xffffffff80dc7bd3 at kern_fsync+0x1b3
#15 0xffffffff811fc532 at amd64_syscall+0x272
#16 0xffffffff811d65ad at fast_syscall_common+0x101
Uptime: 3d10h28m54s
Dumping 4867 out of 16241 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
__curthread () at ./machine/pcpu.h:230
230 ./machine/pcpu.h: No such file or directory.
(kgdb) backtrace
#0 __curthread () at ./machine/pcpu.h:230
#1 doadump (textdump=<optimized out>) at /usr/src/sys/kern/kern_shutdown.c:366
#2 0xffffffff80cf468b in kern_reboot (howto=260) at
/usr/src/sys/kern/kern_shutdown.c:446
#3 0xffffffff80cf4ab3 in vpanic (fmt=<optimized out>, ap=0xfffffe00b77c54a0)
at /usr/src/sys/kern/kern_shutdown.c:872
#4 0xffffffff80cf4833 in panic (fmt=<unavailable>) at
/usr/src/sys/kern/kern_shutdown.c:799
#5 0xffffffff803bbcba in assfail (a=<unavailable>, f=<unavailable>,
l=<unavailable>)
at /usr/src/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:81
#6 0xffffffff803f094b in dbuf_create (dn=0xfffff8015f77a770, level=0 '\000',
blkid=18446744073709551615, parent=0xfffff80296803c60, blkptr=0x0)
at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:2540
#7 0xffffffff803f08a1 in dbuf_create_bonus (dn=0xfffff8015f77a770) at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:3085
#8 0xffffffff803f8d10 in dmu_bonus_hold (os=<optimized out>, object=<optimized
out>, tag=0x0, dbp=0xfffffe00b77c5648)
at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:345
#9 0xffffffff804cde29 in zfs_zget (zfsvfs=0xfffffe008ea28000, obj_num=33262,
zpp=0xfffffe00b77c56c0)
at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c:1137
#10 0xffffffff804bff37 in zfs_get_data (arg=0xfffffe008ea28000,
lr=0xfffffe00a6620300, buf=0x0, lwb=0xfffff8042cf828c0, zio=0xfffff80047eb5000)
at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1297
#11 0xffffffff804d3faf in zil_lwb_commit (zilog=<optimized out>, lwb=<optimized
out>, itx=<optimized out>)
at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c:1493
#12 zil_process_commit_list (zilog=<optimized out>) at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c:2069
#13 zil_commit_writer (zilog=<optimized out>, zcw=<optimized out>) at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c:2201
#14 zil_commit_impl (zilog=<optimized out>, foid=<optimized out>) at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c:2712
#15 0xffffffff804c73bb in zfs_fsync (syncflag=0, vp=<optimized out>,
cr=<optimized out>, ct=<optimized out>)
at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:2627
#16 zfs_freebsd_fsync (ap=<optimized out>) at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:5079
#17 0xffffffff813861be in VOP_FSYNC_APV (vop=0xffffffff81c64f70 <zfs_vnodeops>,
a=0xfffffe00b77c58c0) at vnode_if.c:1331
#18 0xffffffff80dab2d4 in VOP_FSYNC (vp=<unavailable>, waitfor=1, td=<optimized
out>) at ./vnode_if.h:549
#19 vop_stdfdatasync (ap=<optimized out>) at
/usr/src/sys/kern/vfs_default.c:732
#20 0xffffffff8138a84e in VOP_FDATASYNC_APV (vop=0xffffffff81d3b338
<default_vnodeops>, a=0xfffffe00b77c5938) at vnode_if.c:4437
#21 0xffffffff80dc7bd3 in VOP_FDATASYNC (vp=<optimized out>, td=<optimized
out>) at ./vnode_if.h:1849
#22 kern_fsync (td=0xfffff8000bb96580, fd=<optimized out>, fullsync=false) at
/usr/src/sys/kern/vfs_syscalls.c:3374
#23 0xffffffff811fc532 in syscallenter (td=0xfffff8000bb96580) at
/usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:135
#24 amd64_syscall (td=0xfffff8000bb96580, traced=0) at
/usr/src/sys/amd64/amd64/trap.c:1076
#25 <signal handler called>
#26 0x000000080131e8aa in ?? ()
Backtrace stopped: Cannot access memory at address 0x7fffffffd078
(kgdb)
#0 __curthread () at ./machine/pcpu.h:230
#1 doadump (textdump=<optimized out>) at /usr/src/sys/kern/kern_shutdown.c:366
#2 0xffffffff80cf468b in kern_reboot (howto=260) at
/usr/src/sys/kern/kern_shutdown.c:446
#3 0xffffffff80cf4ab3 in vpanic (fmt=<optimized out>, ap=0xfffffe00b77c54a0)
at /usr/src/sys/kern/kern_shutdown.c:872
#4 0xffffffff80cf4833 in panic (fmt=<unavailable>) at
/usr/src/sys/kern/kern_shutdown.c:799
#5 0xffffffff803bbcba in assfail (a=<unavailable>, f=<unavailable>,
l=<unavailable>)
at /usr/src/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:81
#6 0xffffffff803f094b in dbuf_create (dn=0xfffff8015f77a770, level=0 '\000',
blkid=18446744073709551615, parent=0xfffff80296803c60, blkptr=0x0)
at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:2540
#7 0xffffffff803f08a1 in dbuf_create_bonus (dn=0xfffff8015f77a770) at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:3085
#8 0xffffffff803f8d10 in dmu_bonus_hold (os=<optimized out>, object=<optimized
out>, tag=0x0, dbp=0xfffffe00b77c5648)
at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:345
#9 0xffffffff804cde29 in zfs_zget (zfsvfs=0xfffffe008ea28000, obj_num=33262,
zpp=0xfffffe00b77c56c0)
at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c:1137
#10 0xffffffff804bff37 in zfs_get_data (arg=0xfffffe008ea28000,
lr=0xfffffe00a6620300, buf=0x0, lwb=0xfffff8042cf828c0, zio=0xfffff80047eb5000)
at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1297
#11 0xffffffff804d3faf in zil_lwb_commit (zilog=<optimized out>, lwb=<optimized
out>, itx=<optimized out>)
at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c:1493
#12 zil_process_commit_list (zilog=<optimized out>) at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c:2069
#13 zil_commit_writer (zilog=<optimized out>, zcw=<optimized out>) at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c:2201
#14 zil_commit_impl (zilog=<optimized out>, foid=<optimized out>) at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c:2712
#15 0xffffffff804c73bb in zfs_fsync (syncflag=0, vp=<optimized out>,
cr=<optimized out>, ct=<optimized out>)
at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:2627
#16 zfs_freebsd_fsync (ap=<optimized out>) at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:5079
#17 0xffffffff813861be in VOP_FSYNC_APV (vop=0xffffffff81c64f70 <zfs_vnodeops>,
a=0xfffffe00b77c58c0) at vnode_if.c:1331
#18 0xffffffff80dab2d4 in VOP_FSYNC (vp=<unavailable>, waitfor=1, td=<optimized
out>) at ./vnode_if.h:549
#19 vop_stdfdatasync (ap=<optimized out>) at
/usr/src/sys/kern/vfs_default.c:732
#20 0xffffffff8138a84e in VOP_FDATASYNC_APV (vop=0xffffffff81d3b338
<default_vnodeops>, a=0xfffffe00b77c5938) at vnode_if.c:4437
#21 0xffffffff80dc7bd3 in VOP_FDATASYNC (vp=<optimized out>, td=<optimized
out>) at ./vnode_if.h:1849
#22 kern_fsync (td=0xfffff8000bb96580, fd=<optimized out>, fullsync=false) at
/usr/src/sys/kern/vfs_syscalls.c:3374
#23 0xffffffff811fc532 in syscallenter (td=0xfffff8000bb96580) at
/usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:135
#24 amd64_syscall (td=0xfffff8000bb96580, traced=0) at
/usr/src/sys/amd64/amd64/trap.c:1076
#25 <signal handler called>
#26 0x000000080131e8aa in ?? ()
Backtrace stopped: Cannot access memory at address 0x7fffffffd078
(kgdb)
(kgdb) fr 6
#6 0xffffffff803f094b in dbuf_create (dn=0xfffff8015f77a770,
level=0 '\000', blkid=18446744073709551615,
parent=0xfffff80296803c60, blkptr=0x0)
at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:2540
2540 ASSERT(dn->dn_type != DMU_OT_NONE);
(kgdb) print *dn
$2 = {dn_struct_rwlock = {lock_object = {
lo_name = 0xffffffff814ada70 "dn->dn_struct_rwlock",
lo_flags = 40960000, lo_data = 0, lo_witness = 0x0},
sx_lock = 18446735277813228928}, dn_link = {
list_next = 0xfffff8026c5973d8, list_prev = 0xfffff8018f9cbb48},
dn_objset = 0xfffff8000b72c400, dn_object = 33262,
dn_dbuf = 0xfffff80296803c60, dn_handle = 0xfffff802edef9520,
dn_phys = 0xfffffe01c6fd1c00, dn_type = DMU_OT_NONE, dn_bonuslen = 168,
dn_bonustype = 44 ',', dn_nblkptr = 1 '\001', dn_checksum = 0 '\000',
dn_compress = 0 '\000', dn_nlevels = 1 '\001',
dn_indblkshift = 17 '\021', dn_datablkshift = 0 '\000',
dn_moved = 0 '\000', dn_datablkszsec = 7, dn_datablksz = 3584,
dn_maxblkid = 0, dn_next_type = "\000\000\000",
dn_num_slots = 1 '\001', dn_next_nblkptr = "\000\000\000",
dn_next_nlevels = "\000\000\000", dn_next_indblkshift = "\000\000\000",
dn_next_bonustype = "\000\000\000", dn_rm_spillblk = "\000\000\000",
dn_next_bonuslen = {0, 0, 0, 0}, dn_next_blksz = {0, 0, 0, 0},
dn_dbufs_count = 0, dn_dirty_link = {{list_next = 0x0,
list_prev = 0x0}, {list_next = 0xfffff803bbf774e0,
list_prev = 0xfffff8034a965830}, {list_next = 0x0,
list_prev = 0x0}, {list_next = 0x0, list_prev = 0x0}}, dn_mtx = {
lock_object = {lo_name = 0xffffffff81488d05 "dn->dn_mtx",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
dn_dirty_records = {{list_size = 344, list_offset = 0, list_head = {
list_next = 0xfffff8015f77a890, list_prev = 0xfffff8015f77a890}},
{list_size = 344, list_offset = 0, list_head = {
list_next = 0xfffff8015f77a8b0, list_prev = 0xfffff8015f77a8b0}},
{list_size = 344, list_offset = 0, list_head = {
list_next = 0xfffff8015f77a8d0, list_prev = 0xfffff8015f77a8d0}},
{list_size = 344, list_offset = 0, list_head = {
list_next = 0xfffff8015f77a8f0,
list_prev = 0xfffff8015f77a8f0}}}, dn_free_ranges = {0x0, 0x0,
0x0, 0x0}, dn_allocated_txg = 0, dn_free_txg = 0,
dn_assigned_txg = 0, dn_notxholds = {
cv_description = 0xffffffff814ba0a4 "dn->dn_notxholds",
cv_waiters = 0}, dn_dirtyctx = DN_DIRTY_OPEN,
dn_dirtyctx_firstset = 0xfffff8038681d070
"\336\300\255\336\336\300\255\336\360G\305\201\377\377\377\377\336\300\255\336\336\300\255\336\360G\305\201\377\377\377\377\336\300\255\336\336\300\255\336\360G\305\201\377\377\377\377\336\300\255\336\336\300\255\336\360G\305\201\377\377\377\377\336\300\255\336\336\300\255\336\360G\305\201\377\377\377\377\336\300\255\336\336\300\255\336\360G\305\201\377\377\377\377\336\300\255\336\336\300\255\336\360G\305\201\377\377\377\377\336\300\255\336\336\300\255\336\360G\305\201\377\377\377\377\336\300\255\336\336\300\255\336\360G\305\201\377\377\377\377\336\300\255\336\336\300\255\336\360G\305\201\377\377\377\377\336\300\255\336\336\300\255\336\360G\305\201\377\377\377\377\336\300\255\336\336\300\255\336\3--Type
<RET> for more, q to quit, c to continue without paging--
60G\305\201\377\377\377\377\336\300\255\336\336\300\255\336"...,
dn_tx_holds = {rc_mtx = {lock_object = {
lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags = 577830912,
lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, rc_tracked = 0,
rc_list = {list_size = 40, list_offset = 0, list_head = {
list_next = 0xfffff8015f77a990, list_prev = 0xfffff8015f77a990}},
rc_removed = {list_size = 40, list_offset = 0, list_head = {
list_next = 0xfffff8015f77a9b0, list_prev = 0xfffff8015f77a9b0}},
rc_count = 0, rc_removed_count = 0}, dn_holds = {rc_mtx = {
lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0},
sx_lock = 1}, rc_tracked = 0, rc_list = {list_size = 40,
list_offset = 0, list_head = {list_next = 0xfffff8015f77aa08,
list_prev = 0xfffff8015f77aa08}}, rc_removed = {list_size = 40,
list_offset = 0, list_head = {list_next = 0xfffff8015f77aa28,
list_prev = 0xfffff8015f77aa28}}, rc_count = 2,
rc_removed_count = 0}, dn_dbufs_mtx = {lock_object = {
lo_name = 0xffffffff814404f1 "dn->dn_dbufs_mtx",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
dn_dbufs = {avl_root = 0x0,
avl_compar = 0xffffffff80414810 <dbuf_compare>, avl_offset = 288,
avl_numnodes = 0, avl_size = 352}, dn_bonus = 0x0, dn_have_spill = 0,
dn_zio = 0xfffff8024a314000, dn_oldused = 0, dn_oldflags = 0,
dn_olduid = 0, dn_oldgid = 0, dn_newuid = 70, dn_newgid = 70,
dn_id_flags = 13, dn_zfetch = {zf_rwlock = {lock_object = {
lo_name = 0xffffffff813ec741 "zf->zf_rwlock",
lo_flags = 40960000, lo_data = 0, lo_witness = 0x0},
sx_lock = 1}, zf_stream = {list_size = 80, list_offset = 64,
list_head = {list_next = 0xfffff8015f77ab10,
list_prev = 0xfffff8015f77ab10}}, zf_dnode = 0xfffff8015f77a770}}
(kgdb) print *parent
$3 = {db = {db_object = 0, db_offset = 17022976, db_size = 16384,
db_data = 0xfffffe01c6fd0000}, db_objset = 0xfffff8000b72c400,
db_dnode_handle = 0xfffff8000b72c420, db_parent = 0xfffff800743a9000,
db_hash_next = 0x0, db_blkid = 1039, db_blkptr = 0xfffffe017b400780,
db_level = 0 '\000', db_mtx = {lock_object = {
lo_name = 0xffffffff81427def "db->db_mtx", lo_flags = 577830912,
lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, db_state = DB_CACHED,
db_holds = {rc_mtx = {lock_object = {
lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags = 577830912,
lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, rc_tracked = 0,
rc_list = {list_size = 40, list_offset = 0, list_head = {
list_next = 0xfffff80296803d18, list_prev = 0xfffff80296803d18}},
rc_removed = {list_size = 40, list_offset = 0, list_head = {
list_next = 0xfffff80296803d38, list_prev = 0xfffff80296803d38}},
rc_count = 19, rc_removed_count = 0}, db_buf = 0xfffff80225c1de00,
db_changed = {cv_description = 0xffffffff8138c10b "db->db_changed",
cv_waiters = 0}, db_data_pending = 0xfffff80449928000,
db_last_dirty = 0xfffff80449928000, db_link = {avl_child = {
0xfffff802abcc6ee0, 0xfffff8030efbfc20},
avl_pcb = 18446735290778607909}, db_cache_link = {list_next = 0x0,
list_prev = 0x0}, db_caching_status = DB_NO_CACHE,
db_user = 0xfffff802edef9000, db_user_immediate_evict = 0 '\000',
db_freed_in_flight = 0 '\000', db_pending_evict = 0 '\000',
db_dirtycnt = 1 '\001'}
(kgdb) fr 9
#9 0xffffffff804cde29 in zfs_zget (zfsvfs=0xfffffe008ea28000, obj_num=33262,
zpp=0xfffffe00b77c56c0)
at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c:1137
1137 err = sa_buf_hold(zfsvfs->z_os, obj_num, NULL, &db);
(kgdb) print *zfsvfs
$4 = {z_vfs = 0xfffff800133fd000, z_parent = 0xfffffe008ea28000, z_os =
0xfffff8000b72c400, z_root = 4, z_rootvnode = 0xfffff8001340eb40,
z_rootvnodelock = {lock_object = {lo_name = 0xffffffff814594a9 "zfs root
vnode lock", lo_flags = 50528256, lo_data = 0, lo_witness = 0x0},
rm_writecpus = {__bits = {0, 0, 0, 0}}, rm_activeReaders = {lh_first =
0x0}, _rm_lock = {_rm_wlock_object = {
lo_name = 0xffffffff814594a9 "zfs root vnode lock", lo_flags =
16842752, lo_data = 0, lo_witness = 0x0}, _rm_lock_mtx = {lock_object = {
lo_name = 0xffffffff814594a9 "zfs root vnode lock", lo_flags =
16842752, lo_data = 0, lo_witness = 0x0}, mtx_lock = 0}, _rm_lock_sx = {
lock_object = {lo_name = 0xffffffff814594a9 "zfs root vnode lock",
lo_flags = 16842752, lo_data = 0, lo_witness = 0x0}, sx_lock = 0}}},
z_unlinkedobj = 3, z_max_blksz = 16384, z_fuid_obj = 0, z_fuid_size = 0,
z_fuid_idx = {avl_root = 0x0, avl_compar = 0x0, avl_offset = 0,
avl_numnodes = 0, avl_size = 0}, z_fuid_domain = {avl_root = 0x0,
avl_compar = 0x0, avl_offset = 0, avl_numnodes = 0, avl_size = 0},
z_fuid_lock = {lock_object = {lo_name = 0xffffffff81496098
"zfsvfs->z_fuid_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0},
sx_lock = 1},
z_fuid_loaded = 0, z_fuid_dirty = 0, z_fuid_replay = 0x0, z_log =
0xfffff80003b09800, z_acl_mode = 0, z_acl_inherit = 4,
z_case = ZFS_CASE_SENSITIVE, z_utf8 = 0, z_norm = 0, z_atime = 0, z_unmounted
= 0, z_teardown_lock = {locks = {{rr_lock = {lock_object = {
lo_name = 0xffffffff813bcb70 "rrl->rr_lock", lo_flags = 577830912,
lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, rr_cv = {
cv_description = 0xffffffff8141c850 "rrl->rr_cv", cv_waiters = 0},
rr_writer = 0x0, rr_anon_rcount = {rc_mtx = {lock_object = {
lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags = 577830912,
lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, rc_tracked = 0,
rc_list = {list_size = 40, list_offset = 0, list_head = {list_next =
0xfffffe008ea281c0, list_prev = 0xfffffe008ea281c0}}, rc_removed = {
list_size = 40, list_offset = 0, list_head = {list_next =
0xfffffe008ea281e0, list_prev = 0xfffffe008ea281e0}}, rc_count = 1,
rc_removed_count = 0}, rr_linked_rcount = {rc_mtx = {lock_object =
{lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags = 577830912,
lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, rc_tracked = 0,
rc_list = {list_size = 40, list_offset = 0, list_head = {
list_next = 0xfffffe008ea28238, list_prev = 0xfffffe008ea28238}},
rc_removed = {list_size = 40, list_offset = 0, list_head = {
list_next = 0xfffffe008ea28258, list_prev = 0xfffffe008ea28258}},
rc_count = 0, rc_removed_count = 0}, rr_writer_wanted = 0,
rr_track_all = 0}, {rr_lock = {lock_object = {lo_name =
0xffffffff813bcb70 "rrl->rr_lock", lo_flags = 577830912, lo_data = 0,
lo_witness = 0x0}, sx_lock = 1}, rr_cv = {cv_description =
0xffffffff8141c850 "rrl->rr_cv", cv_waiters = 0}, rr_writer = 0x0,
rr_anon_rcount = {rc_mtx = {lock_object = {lo_name = 0xffffffff814893e6
"rc->rc_mtx", lo_flags = 577830912, lo_data = 0, lo_witness = 0x0},
sx_lock = 1}, rc_tracked = 0, rc_list = {list_size = 40,
list_offset = 0, list_head = {list_next = 0xfffffe008ea282f0,
list_prev = 0xfffffe008ea282f0}}, rc_removed = {list_size = 40,
list_offset = 0, list_head = {list_next = 0xfffffe008ea28310,
list_prev = 0xfffffe008ea28310}}, rc_count = 0, rc_removed_count
= 0}, rr_linked_rcount = {rc_mtx = {lock_object = {
lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags = 577830912,
lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, rc_tracked = 0,
rc_list = {list_size = 40, list_offset = 0, list_head = {list_next =
0xfffffe008ea28368, list_prev = 0xfffffe008ea28368}}, rc_removed = {
list_size = 40, list_offset = 0, list_head = {list_next =
0xfffffe008ea28388, list_prev = 0xfffffe008ea28388}}, rc_count = 0,
rc_removed_count = 0}, rr_writer_wanted = 0, rr_track_all = 0},
{rr_lock = {lock_object = {lo_name = 0xffffffff813bcb70 "rrl->rr_lock",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rr_cv = {cv_description = 0xffffffff8141c850 "rrl->rr_cv",
cv_waiters = 0}, rr_writer = 0x0, rr_anon_rcount = {rc_mtx =
{lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock =
1}, rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0,
list_head = {list_next = 0xfffffe008ea28420, list_prev =
0xfffffe008ea28420}}, rc_removed = {list_size = 40, list_offset = 0, list_head
= {
list_next = 0xfffffe008ea28440, list_prev = 0xfffffe008ea28440}},
rc_count = 0, rc_removed_count = 0}, rr_linked_rcount = {rc_mtx = {
lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags
= 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0, list_head
= {list_next = 0xfffffe008ea28498, list_prev = 0xfffffe008ea28498}},
rc_removed = {list_size = 40, list_offset = 0, list_head = {list_next
= 0xfffffe008ea284b8, list_prev = 0xfffffe008ea284b8}}, rc_count = 0,
rc_removed_count = 0}, rr_writer_wanted = 0, rr_track_all = 0},
{rr_lock = {lock_object = {lo_name = 0xffffffff813bcb70 "rrl->rr_lock",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rr_cv = {cv_description = 0xffffffff8141c850 "rrl->rr_cv",
cv_waiters = 0}, rr_writer = 0x0, rr_anon_rcount = {rc_mtx =
{lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock =
1}, rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0,
list_head = {list_next = 0xfffffe008ea28550, list_prev =
0xfffffe008ea28550}}, rc_removed = {list_size = 40, list_offset = 0, list_head
= {
--Type <RET> for more, q to quit, c to continue without paging--
list_next = 0xfffffe008ea28570, list_prev = 0xfffffe008ea28570}},
rc_count = 0, rc_removed_count = 0}, rr_linked_rcount = {rc_mtx = {
lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags
= 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0, list_head
= {list_next = 0xfffffe008ea285c8, list_prev = 0xfffffe008ea285c8}},
rc_removed = {list_size = 40, list_offset = 0, list_head = {list_next
= 0xfffffe008ea285e8, list_prev = 0xfffffe008ea285e8}}, rc_count = 0,
rc_removed_count = 0}, rr_writer_wanted = 0, rr_track_all = 0},
{rr_lock = {lock_object = {lo_name = 0xffffffff813bcb70 "rrl->rr_lock",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rr_cv = {cv_description = 0xffffffff8141c850 "rrl->rr_cv",
cv_waiters = 0}, rr_writer = 0x0, rr_anon_rcount = {rc_mtx =
{lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock =
1}, rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0,
list_head = {list_next = 0xfffffe008ea28680, list_prev =
0xfffffe008ea28680}}, rc_removed = {list_size = 40, list_offset = 0, list_head
= {
list_next = 0xfffffe008ea286a0, list_prev = 0xfffffe008ea286a0}},
rc_count = 0, rc_removed_count = 0}, rr_linked_rcount = {rc_mtx = {
lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags
= 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0, list_head
= {list_next = 0xfffffe008ea286f8, list_prev = 0xfffffe008ea286f8}},
rc_removed = {list_size = 40, list_offset = 0, list_head = {list_next
= 0xfffffe008ea28718, list_prev = 0xfffffe008ea28718}}, rc_count = 0,
rc_removed_count = 0}, rr_writer_wanted = 0, rr_track_all = 0},
{rr_lock = {lock_object = {lo_name = 0xffffffff813bcb70 "rrl->rr_lock",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rr_cv = {cv_description = 0xffffffff8141c850 "rrl->rr_cv",
cv_waiters = 0}, rr_writer = 0x0, rr_anon_rcount = {rc_mtx =
{lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock =
1}, rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0,
list_head = {list_next = 0xfffffe008ea287b0, list_prev =
0xfffffe008ea287b0}}, rc_removed = {list_size = 40, list_offset = 0, list_head
= {
list_next = 0xfffffe008ea287d0, list_prev = 0xfffffe008ea287d0}},
rc_count = 0, rc_removed_count = 0}, rr_linked_rcount = {rc_mtx = {
lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags
= 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0, list_head
= {list_next = 0xfffffe008ea28828, list_prev = 0xfffffe008ea28828}},
rc_removed = {list_size = 40, list_offset = 0, list_head = {list_next
= 0xfffffe008ea28848, list_prev = 0xfffffe008ea28848}}, rc_count = 0,
rc_removed_count = 0}, rr_writer_wanted = 0, rr_track_all = 0},
{rr_lock = {lock_object = {lo_name = 0xffffffff813bcb70 "rrl->rr_lock",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rr_cv = {cv_description = 0xffffffff8141c850 "rrl->rr_cv",
cv_waiters = 0}, rr_writer = 0x0, rr_anon_rcount = {rc_mtx =
{lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock =
1}, rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0,
list_head = {list_next = 0xfffffe008ea288e0, list_prev =
0xfffffe008ea288e0}}, rc_removed = {list_size = 40, list_offset = 0, list_head
= {
list_next = 0xfffffe008ea28900, list_prev = 0xfffffe008ea28900}},
rc_count = 0, rc_removed_count = 0}, rr_linked_rcount = {rc_mtx = {
lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags
= 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0, list_head
= {list_next = 0xfffffe008ea28958, list_prev = 0xfffffe008ea28958}},
rc_removed = {list_size = 40, list_offset = 0, list_head = {list_next
= 0xfffffe008ea28978, list_prev = 0xfffffe008ea28978}}, rc_count = 0,
rc_removed_count = 0}, rr_writer_wanted = 0, rr_track_all = 0},
{rr_lock = {lock_object = {lo_name = 0xffffffff813bcb70 "rrl->rr_lock",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rr_cv = {cv_description = 0xffffffff8141c850 "rrl->rr_cv",
cv_waiters = 0}, rr_writer = 0x0, rr_anon_rcount = {rc_mtx =
{lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock =
1}, rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0,
list_head = {list_next = 0xfffffe008ea28a10, list_prev =
0xfffffe008ea28a10}}, rc_removed = {list_size = 40, list_offset = 0, list_head
= {
list_next = 0xfffffe008ea28a30, list_prev = 0xfffffe008ea28a30}},
rc_count = 0, rc_removed_count = 0}, rr_linked_rcount = {rc_mtx = {
lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags
= 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0, list_head
= {list_next = 0xfffffe008ea28a88, list_prev = 0xfffffe008ea28a88}},
rc_removed = {list_size = 40, list_offset = 0, list_head = {list_next
= 0xfffffe008ea28aa8, list_prev = 0xfffffe008ea28aa8}}, rc_count = 0,
rc_removed_count = 0}, rr_writer_wanted = 0, rr_track_all = 0},
{rr_lock = {lock_object = {lo_name = 0xffffffff813bcb70 "rrl->rr_lock",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rr_cv = {cv_description = 0xffffffff8141c850 "rrl->rr_cv",
cv_waiters = 0}, rr_writer = 0x0, rr_anon_rcount = {rc_mtx =
{lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx",
--Type <RET> for more, q to quit, c to continue without paging--
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock =
1}, rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0,
list_head = {list_next = 0xfffffe008ea28b40, list_prev =
0xfffffe008ea28b40}}, rc_removed = {list_size = 40, list_offset = 0, list_head
= {
list_next = 0xfffffe008ea28b60, list_prev = 0xfffffe008ea28b60}},
rc_count = 0, rc_removed_count = 0}, rr_linked_rcount = {rc_mtx = {
lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags
= 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0, list_head
= {list_next = 0xfffffe008ea28bb8, list_prev = 0xfffffe008ea28bb8}},
rc_removed = {list_size = 40, list_offset = 0, list_head = {list_next
= 0xfffffe008ea28bd8, list_prev = 0xfffffe008ea28bd8}}, rc_count = 0,
rc_removed_count = 0}, rr_writer_wanted = 0, rr_track_all = 0},
{rr_lock = {lock_object = {lo_name = 0xffffffff813bcb70 "rrl->rr_lock",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rr_cv = {cv_description = 0xffffffff8141c850 "rrl->rr_cv",
cv_waiters = 0}, rr_writer = 0x0, rr_anon_rcount = {rc_mtx =
{lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock =
1}, rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0,
list_head = {list_next = 0xfffffe008ea28c70, list_prev =
0xfffffe008ea28c70}}, rc_removed = {list_size = 40, list_offset = 0, list_head
= {
list_next = 0xfffffe008ea28c90, list_prev = 0xfffffe008ea28c90}},
rc_count = 0, rc_removed_count = 0}, rr_linked_rcount = {rc_mtx = {
lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags
= 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0, list_head
= {list_next = 0xfffffe008ea28ce8, list_prev = 0xfffffe008ea28ce8}},
rc_removed = {list_size = 40, list_offset = 0, list_head = {list_next
= 0xfffffe008ea28d08, list_prev = 0xfffffe008ea28d08}}, rc_count = 0,
rc_removed_count = 0}, rr_writer_wanted = 0, rr_track_all = 0},
{rr_lock = {lock_object = {lo_name = 0xffffffff813bcb70 "rrl->rr_lock",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rr_cv = {cv_description = 0xffffffff8141c850 "rrl->rr_cv",
cv_waiters = 0}, rr_writer = 0x0, rr_anon_rcount = {rc_mtx =
{lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock =
1}, rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0,
list_head = {list_next = 0xfffffe008ea28da0, list_prev =
0xfffffe008ea28da0}}, rc_removed = {list_size = 40, list_offset = 0, list_head
= {
list_next = 0xfffffe008ea28dc0, list_prev = 0xfffffe008ea28dc0}},
rc_count = 0, rc_removed_count = 0}, rr_linked_rcount = {rc_mtx = {
lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags
= 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0, list_head
= {list_next = 0xfffffe008ea28e18, list_prev = 0xfffffe008ea28e18}},
rc_removed = {list_size = 40, list_offset = 0, list_head = {list_next
= 0xfffffe008ea28e38, list_prev = 0xfffffe008ea28e38}}, rc_count = 0,
rc_removed_count = 0}, rr_writer_wanted = 0, rr_track_all = 0},
{rr_lock = {lock_object = {lo_name = 0xffffffff813bcb70 "rrl->rr_lock",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rr_cv = {cv_description = 0xffffffff8141c850 "rrl->rr_cv",
cv_waiters = 0}, rr_writer = 0x0, rr_anon_rcount = {rc_mtx =
{lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock =
1}, rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0,
list_head = {list_next = 0xfffffe008ea28ed0, list_prev =
0xfffffe008ea28ed0}}, rc_removed = {list_size = 40, list_offset = 0, list_head
= {
list_next = 0xfffffe008ea28ef0, list_prev = 0xfffffe008ea28ef0}},
rc_count = 0, rc_removed_count = 0}, rr_linked_rcount = {rc_mtx = {
lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags
= 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0, list_head
= {list_next = 0xfffffe008ea28f48, list_prev = 0xfffffe008ea28f48}},
rc_removed = {list_size = 40, list_offset = 0, list_head = {list_next
= 0xfffffe008ea28f68, list_prev = 0xfffffe008ea28f68}}, rc_count = 0,
rc_removed_count = 0}, rr_writer_wanted = 0, rr_track_all = 0},
{rr_lock = {lock_object = {lo_name = 0xffffffff813bcb70 "rrl->rr_lock",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rr_cv = {cv_description = 0xffffffff8141c850 "rrl->rr_cv",
cv_waiters = 0}, rr_writer = 0x0, rr_anon_rcount = {rc_mtx =
{lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock =
1}, rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0,
list_head = {list_next = 0xfffffe008ea29000, list_prev =
0xfffffe008ea29000}}, rc_removed = {list_size = 40, list_offset = 0, list_head
= {
list_next = 0xfffffe008ea29020, list_prev = 0xfffffe008ea29020}},
rc_count = 0, rc_removed_count = 0}, rr_linked_rcount = {rc_mtx = {
lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags
= 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0, list_head
= {list_next = 0xfffffe008ea29078, list_prev = 0xfffffe008ea29078}},
rc_removed = {list_size = 40, list_offset = 0, list_head = {list_next
= 0xfffffe008ea29098, list_prev = 0xfffffe008ea29098}}, rc_count = 0,
rc_removed_count = 0}, rr_writer_wanted = 0, rr_track_all = 0},
{rr_lock = {lock_object = {lo_name = 0xffffffff813bcb70 "rrl->rr_lock",
--Type <RET> for more, q to quit, c to continue without paging--
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rr_cv = {cv_description = 0xffffffff8141c850 "rrl->rr_cv",
cv_waiters = 0}, rr_writer = 0x0, rr_anon_rcount = {rc_mtx =
{lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock =
1}, rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0,
list_head = {list_next = 0xfffffe008ea29130, list_prev =
0xfffffe008ea29130}}, rc_removed = {list_size = 40, list_offset = 0, list_head
= {
list_next = 0xfffffe008ea29150, list_prev = 0xfffffe008ea29150}},
rc_count = 0, rc_removed_count = 0}, rr_linked_rcount = {rc_mtx = {
lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags
= 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0, list_head
= {list_next = 0xfffffe008ea291a8, list_prev = 0xfffffe008ea291a8}},
rc_removed = {list_size = 40, list_offset = 0, list_head = {list_next
= 0xfffffe008ea291c8, list_prev = 0xfffffe008ea291c8}}, rc_count = 0,
rc_removed_count = 0}, rr_writer_wanted = 0, rr_track_all = 0},
{rr_lock = {lock_object = {lo_name = 0xffffffff813bcb70 "rrl->rr_lock",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rr_cv = {cv_description = 0xffffffff8141c850 "rrl->rr_cv",
cv_waiters = 0}, rr_writer = 0x0, rr_anon_rcount = {rc_mtx =
{lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock =
1}, rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0,
list_head = {list_next = 0xfffffe008ea29260, list_prev =
0xfffffe008ea29260}}, rc_removed = {list_size = 40, list_offset = 0, list_head
= {
list_next = 0xfffffe008ea29280, list_prev = 0xfffffe008ea29280}},
rc_count = 0, rc_removed_count = 0}, rr_linked_rcount = {rc_mtx = {
lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags
= 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0, list_head
= {list_next = 0xfffffe008ea292d8, list_prev = 0xfffffe008ea292d8}},
rc_removed = {list_size = 40, list_offset = 0, list_head = {list_next
= 0xfffffe008ea292f8, list_prev = 0xfffffe008ea292f8}}, rc_count = 0,
rc_removed_count = 0}, rr_writer_wanted = 0, rr_track_all = 0},
{rr_lock = {lock_object = {lo_name = 0xffffffff813bcb70 "rrl->rr_lock",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rr_cv = {cv_description = 0xffffffff8141c850 "rrl->rr_cv",
cv_waiters = 0}, rr_writer = 0x0, rr_anon_rcount = {rc_mtx =
{lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock =
1}, rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0,
list_head = {list_next = 0xfffffe008ea29390, list_prev =
0xfffffe008ea29390}}, rc_removed = {list_size = 40, list_offset = 0, list_head
= {
list_next = 0xfffffe008ea293b0, list_prev = 0xfffffe008ea293b0}},
rc_count = 0, rc_removed_count = 0}, rr_linked_rcount = {rc_mtx = {
lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags
= 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0, list_head
= {list_next = 0xfffffe008ea29408, list_prev = 0xfffffe008ea29408}},
rc_removed = {list_size = 40, list_offset = 0, list_head = {list_next
= 0xfffffe008ea29428, list_prev = 0xfffffe008ea29428}}, rc_count = 0,
rc_removed_count = 0}, rr_writer_wanted = 0, rr_track_all = 0},
{rr_lock = {lock_object = {lo_name = 0xffffffff813bcb70 "rrl->rr_lock",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rr_cv = {cv_description = 0xffffffff8141c850 "rrl->rr_cv",
cv_waiters = 0}, rr_writer = 0x0, rr_anon_rcount = {rc_mtx =
{lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock =
1}, rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0,
list_head = {list_next = 0xfffffe008ea294c0, list_prev =
0xfffffe008ea294c0}}, rc_removed = {list_size = 40, list_offset = 0, list_head
= {
list_next = 0xfffffe008ea294e0, list_prev = 0xfffffe008ea294e0}},
rc_count = 0, rc_removed_count = 0}, rr_linked_rcount = {rc_mtx = {
lock_object = {lo_name = 0xffffffff814893e6 "rc->rc_mtx", lo_flags
= 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
rc_tracked = 0, rc_list = {list_size = 40, list_offset = 0, list_head
= {list_next = 0xfffffe008ea29538, list_prev = 0xfffffe008ea29538}},
rc_removed = {list_size = 40, list_offset = 0, list_head = {list_next
= 0xfffffe008ea29558, list_prev = 0xfffffe008ea29558}}, rc_count = 0,
rc_removed_count = 0}, rr_writer_wanted = 0, rr_track_all = 0}}},
z_teardown_inactive_lock = {lock_object = {
lo_name = 0xffffffff81441592 "zfsvfs->z_teardown_inactive_lock", lo_flags
= 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
z_all_znodes = {list_size = 272, list_offset = 240, list_head = {list_next =
0xfffff8000ba58420, list_prev = 0xfffff8006add8ca0}}, z_znodes_lock = {
lock_object = {lo_name = 0xffffffff81489faa "zfsvfs->z_znodes_lock",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1},
z_ctldir = 0xfffff8000ba05400, z_show_ctldir = 0, z_issnap = 0, z_vscan = 0,
z_use_fuids = 1, z_replay = 0, z_use_sa = 1, z_use_namecache = 1,
z_version = 5, z_shares_dir = 7, z_lock = {lock_object = {lo_name =
0xffffffff8138ce8d "zfsvfs->z_lock", lo_flags = 577830912, lo_data = 0,
lo_witness = 0x0}, sx_lock = 1}, z_userquota_obj = 0, z_groupquota_obj =
0, z_replay_eof = 0, z_attr_table = 0xfffff80013351b00, z_hold_mtx = {{
lock_object = {lo_name = 0xffffffff81429501 "zfsvfs->z_hold_mtx[i]",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0},
--Type <RET> for more, q to quit, c to continue without paging--
sx_lock = 1} <repeats 46 times>, {lock_object = {lo_name =
0xffffffff81429501 "zfsvfs->z_hold_mtx[i]", lo_flags = 577830912, lo_data = 0,
lo_witness = 0x0}, sx_lock = 18446735277813228928}, {lock_object =
{lo_name = 0xffffffff81429501 "zfsvfs->z_hold_mtx[i]",
lo_flags = 577830912, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}
<repeats 17 times>}, z_unlinked_drain_task = {ta_link = {stqe_next = 0x0},
ta_pending = 0, ta_priority = 0, ta_func = 0xffffffff804bdcd0
<zfsvfs_task_unlinked_drain>, ta_context = 0xfffffe008ea28000}}
(kgdb) print *zfsvfs->z_vfs
$5 = {mnt_mtx = {lock_object = {lo_name = 0xffffffff81393043 "struct mount
mtx", lo_flags = 16973824, lo_data = 0, lo_witness = 0x0}, mtx_lock = 0},
mnt_gen = 1, mnt_list = {tqe_next = 0xfffff8001343a000, tqe_prev =
0xfffff800133fe028}, mnt_op = 0xffffffff81c64dc8 <zfs_vfsops>,
mnt_vfc = 0xffffffff81c64f18 <zfs_vfsconf>, mnt_vnodecovered =
0xfffff8001340ed20, mnt_syncer = 0xfffff8001340e960, mnt_ref = 896,
mnt_nvnodelist = {tqh_first = 0xfffff8001340eb40, tqh_last =
0xfffff80137ee83e0}, mnt_nvnodelistsize = 896, mnt_writeopcount = 0,
mnt_kern_flag = 1073742272, mnt_flag = 268439568, mnt_opt =
0xfffff8000ba57e10, mnt_optnew = 0x0, mnt_maxsymlinklen = 0, mnt_stat = {
f_version = 538182936, f_type = 222, f_flags = 268439568, f_bsize = 512,
f_iosize = 16384, f_blocks = 404148032, f_bfree = 390589344,
f_bavail = 390589344, f_files = 390593281, f_ffree = 390589344,
f_syncwrites = 0, f_asyncwrites = 0, f_syncreads = 0, f_asyncreads = 0,
f_spare = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, f_namemax = 255, f_owner = 0,
f_fsid = {val = {-901290623, -1775448610}},
f_charspare = '\000' <repeats 79 times>, f_fstypename = "zfs", '\000'
<repeats 12 times>,
f_mntfromname = "zroot/space/pgsql", '\000' <repeats 1006 times>,
f_mntonname = "/space/pgsql", '\000' <repeats 1011 times>},
mnt_cred = 0xfffff800133ef900, mnt_data = 0xfffffe008ea28000, mnt_time = 0,
mnt_iosize_max = 65536, mnt_export = 0x0, mnt_label = 0x0,
mnt_hashseed = 127686091, mnt_lockref = 0, mnt_secondary_writes = 0,
mnt_secondary_accwrites = 0, mnt_susp_owner = 0x0, mnt_gjprovider = 0x0,
mnt_listmtx = {lock_object = {lo_name = 0xffffffff8142f7e7 "struct mount
vlist mtx", lo_flags = 16973824, lo_data = 0, lo_witness = 0x0},
mtx_lock = 0}, mnt_activevnodelist = {tqh_first = 0xfffff802882ebb40,
tqh_last = 0xfffff8001340ec00}, mnt_activevnodelistsize = 438,
mnt_tmpfreevnodelist = {tqh_first = 0xfffff800244071e0, tqh_last =
0xfffff803c87a82a0}, mnt_tmpfreevnodelistsize = 11, mnt_explock = {
lock_object = {lo_name = 0xffffffff81469cce "explock", lo_flags =
108199936, lo_data = 0, lo_witness = 0x0}, lk_lock = 1, lk_exslpfail = 0,
lk_timo = 0, lk_pri = 96}, mnt_upper_link = {tqe_next = 0x0, tqe_prev =
0x0}, mnt_uppers = {tqh_first = 0x0, tqh_last = 0xfffff800133fdaa8}}
(kgdb)
courtland# zdb -dddd zroot/space/pgsql 33262
Dataset zroot/space/pgsql [ZPL], ID 184, cr_txg 37, 6.37G, 3905 objects, rootbp
DVA[0]=<0:95f64f000:1000> DVA[1]=<0:11e4fbc000:1000> [L0 DMU objset] fletcher4
uncompressed LE contiguous unique double size=800L/800P birth=925624L/925624P
fill=3905 cksum=d3563dce4:eed0e01cca3:a72f616c79c87:5770ae867335855
Object lvl iblk dblk dsize dnsize lsize %full type
zdb: dmu_bonus_hold(33262) failed, errno 2
--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
freebsd-bugs@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-bugs
To unsubscribe, send any mail to "freebsd-bugs-unsubscr...@freebsd.org"
