Andriy Gapon <a...@freebsd.org> wrote: > on 01/04/2013 17:18 Fabian Keil said the following: > > #10 0xffffffff8130323f in assfail3 (a=<value optimized out>, lv=<value > > optimized out>, op=<value optimized out>, rv=<value optimized out>, > > f=<value optimized out>, l=<value optimized out>) > > at > > /usr/src/sys/modules/opensolaris/../../cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:89 > > #11 0xffffffff8117924e in zfs_space_delta_cb (bonustype=<value optimized > > out>, data=0xffffff8015eeb8c0, userp=0xfffffe004261c640, > > groupp=0xfffffe004261c648) > > at > > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c:625 > > #12 0xffffffff8110003b in dmu_objset_userquota_get_ids > > (dn=0xfffffe004261c358, before=0, tx=<value optimized out>) at > > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c:1249 > > #13 0xffffffff811071b6 in dnode_sync (dn=0xfffffe004261c358, > > tx=0xfffffe00186e1300) at > > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dnode_sync.c:554 > > #14 0xffffffff810ff98b in dmu_objset_sync_dnodes (list=0xfffffe00691a5250, > > newlist=<value optimized out>, tx=<value optimized out>) > > at > > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c:910 > > #15 0xffffffff810ff825 in dmu_objset_sync (os=0xfffffe00691a5000, > > pio=<value optimized out>, tx=0xfffffe00186e1300) > > at > > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c:1027 > > #16 0xffffffff8110cb0d in dsl_dataset_sync (ds=0xfffffe001f3d0c00, > > zio=0x780, tx=0xfffffe00186e1300) at > > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c:1411 > > #17 0xffffffff8111399a in dsl_pool_sync (dp=0xfffffe0069ec4000, txg=<value > > optimized out>) at > > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c:409 > > #18 0xffffffff8112f0ee in spa_sync (spa=0xfffffe0050f00000, txg=3292) at > > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:6328 > > #19 0xffffffff81137c45 in txg_sync_thread (arg=0xfffffe0069ec4000) at > > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c:493 > > #20 0xffffffff80569c1a in fork_exit (callout=0xffffffff811378d0 > > <txg_sync_thread>, arg=0xfffffe0069ec4000, frame=0xffffff80dc9d6c00) at > > /usr/src/sys/kern/kern_fork.c:991 > > #21 0xffffffff8086a1ee in fork_trampoline () at exception.S:602 > > #22 0x0000000000000000 in ?? () > > Current language: auto; currently minimal > > (kgdb) f 12 > > #12 0xffffffff8110003b in dmu_objset_userquota_get_ids > > (dn=0xfffffe004261c358, before=0, tx=<value optimized out>) at > > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c:1249 > > 1249 error = > > used_cbs[os->os_phys->os_type](dn->dn_bonustype, data, > > (kgdb) p *dn > > Could you please also provide *dn->dn_phys?
vmcore.12:
(kgdb) p *dn->dn_phys
Cannot access memory at address 0xffffff8019492800
(kgdb) p *dn->dn_dbuf
$1 = {db = {db_object = 0, db_offset = 28491776, db_size = 16384, db_data =
0xffffff8019492000}, db_objset = 0xfffffe002bc62400, db_dnode_handle =
0xfffffe002bc62420, db_parent = 0xfffffe005f1071c0,
db_hash_next = 0x0, db_blkid = 1739, db_blkptr = 0xffffff801936c580, db_level
= 0 '\0', db_mtx = {lock_object = {lo_name = 0xffffffff811d8696 "db->db_mtx",
lo_flags = 40960000, lo_data = 0,
lo_witness = 0x0}, sx_lock = 1}, db_state = DB_CACHED, db_holds =
{rc_count = 2}, db_buf = 0xfffffe005f34c798, db_changed = {cv_description =
0xffffffff811d86a2 "db->db_changed", cv_waiters = 0},
db_data_pending = 0xfffffe004bcc0c00, db_last_dirty = 0xfffffe004bcc0c00,
db_link = {list_next = 0xfffffe005f3506d0, list_prev = 0xfffffe0030c392a0},
db_user_ptr = 0xfffffe005f269000,
db_user_data_ptr_ptr = 0x0, db_evict_func = 0xffffffff81105770
<dnode_buf_pageout>, db_immediate_evict = 0 '\0', db_freed_in_flight = 0 '\0',
db_dirtycnt = 1 '\001'}
vmcore.13:
(kgdb) p *dn->dn_phys
Cannot access memory at address 0xffffff8015eeb800
(kgdb) p *dn->dn_dbuf
$1 = {db = {db_object = 0, db_offset = 28491776, db_size = 16384, db_data =
0xffffff8015eeb000}, db_objset = 0xfffffe00691a5000, db_dnode_handle =
0xfffffe00691a5020, db_parent = 0xfffffe004254ab60,
db_hash_next = 0x0, db_blkid = 1739, db_blkptr = 0xffffff8015d65580, db_level
= 0 '\0', db_mtx = {lock_object = {lo_name = 0xffffffff811d8696 "db->db_mtx",
lo_flags = 40960000, lo_data = 0,
lo_witness = 0x0}, sx_lock = 1}, db_state = DB_CACHED, db_holds =
{rc_count = 2}, db_buf = 0xfffffe0042bedcf0, db_changed = {cv_description =
0xffffffff811d86a2 "db->db_changed", cv_waiters = 0},
db_data_pending = 0xfffffe00406d6500, db_last_dirty = 0xfffffe00406d6500,
db_link = {list_next = 0xfffffe0042758c10, list_prev = 0xfffffe0069cab5f8},
db_user_ptr = 0xfffffe0069f70000,
db_user_data_ptr_ptr = 0x0, db_evict_func = 0xffffffff81105770
<dnode_buf_pageout>, db_immediate_evict = 0 '\0', db_freed_in_flight = 0 '\0',
db_dirtycnt = 1 '\001'}
vmcore.14:
(kgdb) p *dn->dn_phys
Cannot access memory at address 0xffffff8014d21800
(kgdb) p *dn->dn_dbuf
$1 = {db = {db_object = 0, db_offset = 28508160, db_size = 16384, db_data =
0xffffff8014d21000}, db_objset = 0xfffffe0073372000, db_dnode_handle =
0xfffffe0073372020, db_parent = 0xfffffe00369eec40,
db_hash_next = 0x0, db_blkid = 1740, db_blkptr = 0xffffff8014ac3600, db_level
= 0 '\0', db_mtx = {lock_object = {lo_name = 0xffffffff811d8696 "db->db_mtx",
lo_flags = 40960000, lo_data = 0,
lo_witness = 0x0}, sx_lock = 1}, db_state = DB_CACHED, db_holds =
{rc_count = 2}, db_buf = 0xfffffe0036b759d8, db_changed = {cv_description =
0xffffffff811d86a2 "db->db_changed", cv_waiters = 0},
db_data_pending = 0xfffffe0034467700, db_last_dirty = 0xfffffe0034467700,
db_link = {list_next = 0xfffffe0036c17970, list_prev = 0xfffffe00737bc950},
db_user_ptr = 0xfffffe00369fe000,
db_user_data_ptr_ptr = 0x0, db_evict_func = 0xffffffff81105770
<dnode_buf_pageout>, db_immediate_evict = 0 '\0', db_freed_in_flight = 0 '\0',
db_dirtycnt = 1 '\001'}
vmcore.15:
(kgdb) p *dn->dn_phys
Cannot access memory at address 0xffffff8030e73800
(kgdb) p *dn->dn_dbuf
$1 = {db = {db_object = 0, db_offset = 28508160, db_size = 16384, db_data =
0xffffff8030e73000}, db_objset = 0xfffffe0028fe2800, db_dnode_handle =
0xfffffe0028fe2820, db_parent = 0xfffffe005ac4dd20,
db_hash_next = 0x0, db_blkid = 1740, db_blkptr = 0xffffff8030d49600, db_level
= 0 '\0', db_mtx = {lock_object = {lo_name = 0xffffffff811d8696 "db->db_mtx",
lo_flags = 40960000, lo_data = 0,
lo_witness = 0x0}, sx_lock = 1}, db_state = DB_CACHED, db_holds =
{rc_count = 2}, db_buf = 0xfffffe0010130a68, db_changed = {cv_description =
0xffffffff811d86a2 "db->db_changed", cv_waiters = 0},
db_data_pending = 0xfffffe0057b85200, db_last_dirty = 0xfffffe0057b85200,
db_link = {list_next = 0xfffffe005ad57890, list_prev = 0xfffffe00568765f8},
db_user_ptr = 0xfffffe005a7a1000,
db_user_data_ptr_ptr = 0x0, db_evict_func = 0xffffffff81105770
<dnode_buf_pageout>, db_immediate_evict = 0 '\0', db_freed_in_flight = 0 '\0',
db_dirtycnt = 1 '\001'}
> > The offending sa_magic in the panic message is always the same.
>
> Which part, left side or right side?
I meant the left side, but it looks like I only compared the first two panics:
fk@r500 /usr/crash $grep ^panic: core.txt.1[2-5]
core.txt.12:panic: solaris assert: sa.sa_magic == 0x2F505A (0x4d5ea364 ==
0x2f505a), file:
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c,
line: 625
core.txt.13:panic: solaris assert: sa.sa_magic == 0x2F505A (0x4d5ea364 ==
0x2f505a), file:
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c,
line: 625
core.txt.14:panic: solaris assert: sa.sa_magic == 0x2F505A (0x4be9810a ==
0x2f505a), file:
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c,
line: 625
core.txt.15:panic: solaris assert: sa.sa_magic == 0x2F505A (0x4be9810a ==
0x2f505a), file:
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c,
line: 625
The matches seem to correlate with the matching dn->dn_dbuf->db->db_offset
values above.
Fabian
signature.asc
Description: PGP signature
