Le 26 déc. 09 à 04:47, Tim Cook a écrit :
On Fri, Dec 25, 2009 at 11:57 AM, Saso Kiselkov <skisel...@gmail.com> wrote:-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I've started porting a video streaming application to opensolaris onZFS, and am hitting some pretty weird performance issues. The thing I'mtrying to do is run 77 concurrent video capture processes (roughly 430Mbit/s in total) all writing into separate files on a 12TB J4200storage array. The disks in the array are arranged into a single RAID-0ZFS volume (though I've tried different RAID levels, none helped). CPUperformance is not an issue (barely hitting 35% utilization on a singleCPU quad-core X2250). I/O bottlenecks can also be ruled out, since the storage array's sequential write performance is around 600MB/s. The problem is the bursty behavior of ZFS writes. All the capture processes do, in essence is poll() on a socket and then read() and write() any available data from it to a file. The poll() call is done with a timeout of 250ms, expecting that if no data arrives within 0.25seconds, the input is dead and recording stops (I tried increasing thisvalue, but the problem still arises, although not as frequently). When ZFS decides that it wants to commit a transaction group to disk (every30 seconds), the system stalls for a short amount of time and dependingon the number capture of processes currently running, the poll() call(which usually blocks for 1-2ms), takes on the order of hundreds of ms, sometimes even longer. I figured that I might be able to resolve this bylowering the txg timeout to something like 1-2 seconds (I need ZFS to write as soon as data arrives, since it will likely never beoverwritten), but I couldn't find any tunable parameter for it anywhereon the net. On FreeBSD, I think this can be done via the vfs.zfs.txg_timeout sysctl. A glimpse into the source at http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/txg.con line 40 made me worry that somebody maybe hard-coded this value intothe kernel, in which case I'd be pretty much screwed in opensolaris. Any help would be greatly appreciated. Regards, - -- SasoHang on... if you've got 77 concurrent threads going, I don't see how that's a "sequential" I/O load. To the backend storage it's going to look like the equivalent of random I/O.
I see this posted once in a while and I'm not sure where that comes from. Sequential workloads are important inasmuch as the FS/VM can detect and issue large request to disk (followed by cache hits) instead of multiple small ones. The detection for ZFS is done at the file level and so the fact that one has N concurrent streams going is not relevant. On writes ZFS and the Copy-On-Write model makes sequential/random distinction not very defining. All writes are targetting free blocks.
-r
I'd also be surprised to see 12 1TB disks supporting 600MB/sec throughput and would be interested in hearing where you got those numbers from.Is your video capture doing 430MB or 430Mbit? -- --Tim _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
