Stefan, > one question related with this: would KAIO be supported on such > configuration ?
Yes, but not as one might expect. As seen from the truss output below, the call to kaio() fails with EBADFD, a direct result of the fact that for ZFS its cb_ops interface for asynchronous read and write are set to nodev, resulting in ENXIO being returned. The kaio() routine detects this ENXIO error, allocates a LWP thread to perform the now synchronous I/O, asynchronously. Although the following following CR does not explicitly call it out, http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6496357 , it resulting in the following changes being made to zfs_ioclt.c and zvol.c, being the removal of the kernel mode AIO interfaces. http://cvs.opensolaris.org/source/diff/onnv/onnv-gate/usr/src/uts/common/fs/zfs/zfs_ioctl.c?r1=%2Fonnv%2Fonnv-gate%2Fusr%2Fsrc%2Futs%2Fcommon%2Ffs%2Fzfs%2Fzfs_ioctl.c%403638&r2=%2Fonnv%2Fonnv-gate%2Fusr%2Fsrc%2Futs%2Fcommon%2Ffs%2Fzfs%2Fzfs_ioctl.c%403444 http://cvs.opensolaris.org/source/diff/onnv/onnv-gate/usr/src/uts/common/fs/zfs/zvol.c?r1=%2Fonnv%2Fonnv-gate%2Fusr%2Fsrc%2Futs%2Fcommon%2Ffs%2Fzfs%2Fzvol.c%403638&r2=%2Fonnv%2Fonnv-gate%2Fusr%2Fsrc%2Futs%2Fcommon%2Ffs%2Fzfs%2Fzvol.c%403461 These routines were removed as part of 6496357, since both invoked zvol_strategy, a routine the is somewhat synchronous in nature. Even when invoked from multiple LPW threads, parallel threads can still synchronize, depending on byte-range-locking at the following location. http://cvs.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/zvol.c#1130 - Jim > At least trying to open /dev/zvol/rdsk/datapool/master where master > is defined > as: > > # zpool create -f datapool mirror c1t1d0 c2t0d0 > # zfs create -V 1gb datapool/master > # ls -lrt /dev/zvol/rdsk/datapool/master > lrwxrwxrwx 1 root root 39 Feb 1 18:29 /dev/zvol/rdsk/datapool/master > -> ../../../../devices/pseudo/z...@0:3c,raw > > > resulted with: > # truss -t kaio,lwp_create ./test-kaio /dev/zvol/rdsk/datapool/master > kaio(AIOINIT) = 0 > lwp_create(0x08046860, LWP_DAEMON|LWP_DETACHED|LWP_SUSPENDED, > 0x0804685C) = 2 > /2: lwp_create() (returning as new lwp ...) = 0 > /1: kaio(AIOREAD, 5, 0x08046C2C, 1024, 0, 0x0804702C) Err#81 EBADFD > /1: lwp_create(0x08046820, LWP_DAEMON|LWP_DETACHED|LWP_SUSPENDED, > 0x0804681C) = 3 > /3: lwp_create() (returning as new lwp ...) = 0 > /1: lwp_create(0x08046820, LWP_DAEMON|LWP_DETACHED|LWP_SUSPENDED, > 0x0804681C) = 4 > /4: lwp_create() (returning as new lwp ...) = 0 > /1: lwp_create(0x08046820, LWP_DAEMON|LWP_DETACHED|LWP_SUSPENDED, > 0x0804681C) = 5 > /5: lwp_create() (returning as new lwp ...) = 0 > /1: lwp_create(0x08046820, LWP_DAEMON|LWP_DETACHED|LWP_SUSPENDED, > 0x0804681C) = 6 > /6: lwp_create() (returning as new lwp ...) = 0 > /1: lwp_create(0x08046820, LWP_DAEMON|LWP_DETACHED|LWP_SUSPENDED, > 0x0804681C) = 7 > /7: lwp_create() (returning as new lwp ...) = 0 > /4: kaio(AIONOTIFY) = 0 > /1: kaio(AIOWAIT, 0x00000000, 0) = 1 > aio succeeded > > > Comments ? The original question came from a real setup using Sybase > which spits a lot of: > > 733/1: 4137604 6 1 kaio(0x2, 0xFFFFFFFFFFFFFFFF, > 0x1) = -1 Err#22 > 733/1: 4137611 7 3 pollsys(0x1000010B3A8, 0x1A, > 0x10009FE6BF0) = 1 0 > 733/1: 4137614 3 0 kaio(0x2, 0xFFFFFFFFFFFFFFFF, > 0x1) = -1 Err#22 > 733/1: 4137622 5 2 recv(0x17, 0x100044F20F0, > 0x800) = 109 0 > 733/1: 4137626 3 0 kaio(0x2, 0xFFFFFFFFFFFFFFFF, > 0x1) = -1 Err#22 > 733/1: 4137631 6 2 pollsys(0x1000010B3A8, 0x1A, > 0x10009FE6BF0) = 0 0 > 733/1: 4137633 3 0 kaio(0x2, 0xFFFFFFFFFFFFFFFF, > 0x1) = -1 Err#22 > 733/1: 4137636 4 1 pollsys(0x1000010B3A8, 0x1A, > 0x10009FE6BF0) = 0 0 > 733/1: 4137637 3 0 kaio(0x2, 0xFFFFFFFFFFFFFFFF, > 0x1) = -1 Err#22 > 733/1: 4137641 4 1 pollsys(0x1000010B3A8, 0x1A, > 0x10009FE6BF0) = 0 0 > 733/1: 4137642 3 0 kaio(0x2, 0xFFFFFFFFFFFFFFFF, > 0x1) = -1 Err#22 > 733/1: 4137646 4 1 pollsys(0x1000010B3A8, 0x1A, > 0x10009FE6BF0) = 0 0 > 733/1: 4137647 3 0 kaio(0x2, 0xFFFFFFFFFFFFFFFF, > 0x1) = -1 Err#22 > 733/1: 4137650 4 1 pollsys(0x1000010B3A8, 0x1A, > 0x10009FE6BF0) = 0 0 > 733/1: 4137652 3 0 kaio(0x2, 0xFFFFFFFFFFFFFFFF, > 0x1) = -1 Err#22 > 733/1: 4137655 4 1 pollsys(0x1000010B3A8, 0x1A, > 0x10009FE6BF0) = 0 0 > 733/1: 4137657 4 0 kaio(0x2, 0xFFFFFFFFFFFFFFFF, > 0x1) = -1 Err#22 > 733/1: 4137660 4 1 pollsys(0x1000010B3A8, 0x1A, > 0x10009FE6BF0) = 0 0 > > > Thanks, > Stefan > > test-kaio.c > http://sunsite.uakom.sk/sunworldonline/swol-07-1998/swol-07-insidesolaris.html > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
