-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The application I'm working on is a kind of large-scale network-PVR system for our IPTV services. It records all running TV channels in a X-hour carrousel (typically 24 or 48-hours), retaining only those bits which users have marked as being interesting to them. The current setup I'm doing development on is a small 12TB array, future deployment is planned on several 96TB X4540 machines.
I agree that I kind of misused the term `sequential' - it really is 77 concurrent sequential writes. However, as I explained, I/O is not the bottleneck here, as the array is capable of writes around 600MBytes/s, and the write load I'm putting on it is around 55MBytes/s (430Mbit/s). The problem is, as Brent explained, that as soon as the OS decides it wants to write the transaction group to disk, it totally ignores all other time-critical activity in the system and focuses on just that, causing an input poll() stall on all network sockets. What I'd need to do is force it to commit transactions to disk more often so as to even the load out over a longer period of time, to bring the CPU usage spikes down to a more manageable and predictable level. Regards, - -- Saso Tim Cook wrote: > On Fri, Dec 25, 2009 at 11:43 PM, Brent Jones <br...@servuhome.net> wrote: > >>>> >>> >>> Hang on... if you've got 77 concurrent threads going, I don't see how >> that's >>> a "sequential" I/O load. To the backend storage it's going to look like >> the >>> equivalent of random I/O. I'd also be surprised to see 12 1TB disks >>> supporting 600MB/sec throughput and would be interested in hearing where >> you >>> got those numbers from. >>> >>> Is your video capture doing 430MB or 430Mbit? >>> >>> -- >>> --Tim >>> >> > >> >> Think he said 430Mbit/sec, which if these are security cameras, would >> be a good sized installation (30+ cameras). >> We have a similar system, albeit running on Windows. Writing about >> 400Mbit/sec using just 6, 1TB SATA drives is entirely possible, and >> working quite well on our system without any frame loss or much >> latency. >> > > Once again, Mb or MB? They're two completely different numbers. As for > getting 400Mbit out of 6 SATA drive, that's not really impressive at all. > If you're saying you got 400MB, that's a different story entirely, and while > possible with sequential I/O and a proper raid setup, it isn't happening > with random. > > >> The writes lag is noticeable however with ZFS, and the behavior of the >> transaction group writes. If you have a big write that needs to land >> on disk, it seems all other I/O, CPU and "niceness" is thrown out the >> window in favor of getting all that data on disk. >> I was on a watch list for a ZFS I/O scheduler bug with my paid Solaris >> support, I'll try to find that bug number, but I believe some >> improvements were done in 129 and 130. >> >> >> > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAks1y8oACgkQRO8UcfzpOHBkDQCgxScaPPS7d+peoiY16Nafo8lu 1nsAoNMwiUdOdQKCZpdyPGoAWz36IWY5 =T6fy -----END PGP SIGNATURE----- _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
