Re: [zfs-discuss] VM's on ZFS - 7210

2010-08-27 Thread Saso Kiselkov
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 If I remember correctly, ESX always uses synchronous writes over NFS. If so, adding a dedicated log device (such as a DDRdrive) might help you out here. You should be able to test it by disabling the ZIL for a short while and see if performance improve

Re: [zfs-discuss] ZFS offline ZIL corruption not detected

2010-08-26 Thread Saso Kiselkov
m not familiar with ZFS source code.) - -- Saso On 08/26/2010 04:31 PM, Darren J Moffat wrote: > On 26/08/2010 15:08, Saso Kiselkov wrote: >> If I might add my $0.02: it appears that the ZIL is implemented as a >> kind of circular log buffer. As I understand it, when a corrupt che

Re: [zfs-discuss] ZFS offline ZIL corruption not detected

2010-08-26 Thread Saso Kiselkov
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 If I might add my $0.02: it appears that the ZIL is implemented as a kind of circular log buffer. As I understand it, when a corrupt checksum is detected, it is taken to be the end of the log, but this kind of defeats the checksum's original purpose, w

Re: [zfs-discuss] Disk space on Raidz1 configuration

2010-08-06 Thread Saso Kiselkov
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 ZFS and "du" use binary byte multipliers (1kB = 1024 B, etc.), while drive manufacturers use decimal conversion (1kB = 1000 B). So your 1.5TB drives are in fact ~1.36 TiB (binary TB): 7 x 1,36 TiB = 9.52 TiB - 1,36 TiB for parity = 8.16 TiB - -- Saso

Re: [zfs-discuss] CPU requirements for zfs performance

2010-07-22 Thread Saso Kiselkov
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I didn't mean to imply that I use it for my media storage, just that I occasionally encounter situations when it could be useful. BR, - -- Saso On 07/22/2010 11:23 AM, Roy Sigurd Karlsbakk wrote: > - Original Message - >> I do encounter situa

Re: [zfs-discuss] CPU requirements for zfs performance

2010-07-21 Thread Saso Kiselkov
iscuss- >> boun...@opensolaris.org] On Behalf Of Saso Kiselkov >> >> If you plan on using it as a storage server for multimedia data >> (movies), don't even bother considering compression, as most media >> files >> already come heavily compressed. Dedup might

Re: [zfs-discuss] CPU requirements for zfs performance

2010-07-21 Thread Saso Kiselkov
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 If you plan on using it as a storage server for multimedia data (movies), don't even bother considering compression, as most media files already come heavily compressed. Dedup might still come in handy, though. - -- Saso On 07/21/2010 05:03 PM, Eugen

Re: [zfs-discuss] CR 6880994 and pkg fix

2010-03-24 Thread Saso Kiselkov
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 How about running memtest86+ (http://www.memtest.org/) on the machine for a while? It doesn't test the arithmetics on the CPU very much, but it stresses data paths quite a lot. Just a quick suggestion... - -- Saso Damon Atkins wrote: > You could try

[zfs-discuss] Booting OpenSolaris on ZFS root on Sun Netra 240

2010-02-04 Thread Saso Kiselkov
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I'm kind stuck at trying to get my aging Netra 240 machine to boot OpenSolaris. The live CD and installation worked perfectly, but when I reboot and try to boot from the installed disk, I get: Rebooting with command: boot disk0 Boot device: /p...

[zfs-discuss] Writing faster than storage can handle crashes system

2010-01-15 Thread Saso Kiselkov
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I've run up into yet another peculiarity while building my video storage solution on top of ZFS: if I fill up the disk to 98%, writes slow down - this is expected. However, what happens when the write load overwhelms the storage systems is puzzling to

Re: [zfs-discuss] ZFS write bursts cause short app stalls

2010-01-07 Thread Saso Kiselkov
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Just tried and didn't help :-(. Regards, - -- Saso Brent Jones wrote: > On Wed, Jan 6, 2010 at 2:40 PM, Saso Kiselkov wrote: >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA1 >> >> Buffering the writes in the OS

Re: [zfs-discuss] ZFS write bursts cause short app stalls

2010-01-06 Thread Saso Kiselkov
f its one thing people don't like about IPTV, it's packet loss. Eliminating even the possibility of packet loss completely would be the best way to go, I think. Regards, - -- Saso Bob Friesenhahn wrote: > On Wed, 6 Jan 2010, Saso Kiselkov wrote: > >> I'm aware of the

Re: [zfs-discuss] ZFS write bursts cause short app stalls

2010-01-06 Thread Saso Kiselkov
ind live TV). If I buffer writes for up to 10 seconds in user-space, other playback processes can fail due to running out of data. Regards, - -- Saso Bob Friesenhahn wrote: > On Wed, 6 Jan 2010, Saso Kiselkov wrote: > >> I've encountered a new problem on the opposite end of my ap

Re: [zfs-discuss] ZFS write bursts cause short app stalls

2010-01-06 Thread Saso Kiselkov
filling up the network input buffer and starting to drop packets. Is there a way that I can increase the write I/O priority, or increase the write buffer in ZFS so that write()s won't block? Regards, - -- Saso Saso Kiselkov wrote: > Ok, I figured out that apparently I was the idiot in

Re: [zfs-discuss] ZFS write bursts cause short app stalls

2010-01-02 Thread Saso Kiselkov
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Be sure to also update to the latest dev b130 release, as that also helps with a more smooth scheduling class for the zfs threads. If the upgrade breaks anything, you can always just boot back into the old environment before the upgrade. Regards, - --

Re: [zfs-discuss] ZFS write bursts cause short app stalls

2009-12-30 Thread Saso Kiselkov
when commiting data to disk), but by increasing network input buffer sizes it seems I was able to cut input packet loss to zero. Thanks for all the valuable advice! Regards, - -- Saso Saso Kiselkov wrote: > I tried removing the flow and subjectively packet loss occurs a bit less > often, but

Re: [zfs-discuss] ZFS write bursts cause short app stalls

2009-12-29 Thread Saso Kiselkov
orking-discuss@ > > > On 28/12/2009 15:50, Saso Kiselkov wrote: > Thank you for the advice. After trying flowadm the situation improved > somewhat, but I'm still getting occasional packet overflow (10-100 > packets about every 10-15 minutes). This is somewhat unner

Re: [zfs-discuss] ZFS write bursts cause short app stalls

2009-12-28 Thread Saso Kiselkov
ses checks to fail therefore triggering unnecessary alarms. > > Yours > Markus Kovero > > -Original Message- > From: zfs-discuss-boun...@opensolaris.org > [mailto:zfs-discuss-boun...@opensolaris.org] On Behalf Of Saso Kiselkov > Sent: 28. joulukuuta 2009 15:25 > To: zfs-d

Re: [zfs-discuss] ZFS write bursts cause short app stalls

2009-12-28 Thread Saso Kiselkov
processes) to even out the CPU load. Any ideas on why ZFS should completely thrash the network layer and make it drop incomming packets? Regards, - -- Saso Robert Milkowski wrote: > On 26/12/2009 12:22, Saso Kiselkov wrote: >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA1 >

Re: [zfs-discuss] ZFS write bursts cause short app stalls

2009-12-27 Thread Saso Kiselkov
atest stable OpenSolaris release, rather than having to go to a development branch or tuning kernel parameters to even get the software working as it should. Regards, - -- Saso Robert Milkowski wrote: > On 26/12/2009 12:22, Saso Kiselkov wrote: >> -BEGIN PGP SIGNED MESSAGE-

Re: [zfs-discuss] ZFS write bursts cause short app stalls

2009-12-26 Thread Saso Kiselkov
tting small delays every 60 seconds (on the order of 20-30ms). I'm not sure these have something to do with ZFS, though... they happen outside of the write bursts. Thank you all for the valuable advice! Regards, - -- Saso Richard Elling wrote: > > On Dec 26, 2009, at 1:10 AM, Sas

Re: [zfs-discuss] ZFS write bursts cause short app stalls

2009-12-26 Thread Saso Kiselkov
of the usual 0-2ms). It's not that big a problem, it's just that I'm curious as to where it's being created. I assume some 60-second timer is firing, but I don't know where. Regards, - -- Saso Fajar A. Nugraha wrote: > On Sat, Dec 26, 2009 at 4:10 PM, Saso Kiselkov w

Re: [zfs-discuss] ZFS write bursts cause short app stalls

2009-12-26 Thread Saso Kiselkov
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Brent Jones wrote: > On Fri, Dec 25, 2009 at 9:56 PM, Tim Cook wrote: >> >> On Fri, Dec 25, 2009 at 11:43 PM, Brent Jones wrote: > Hang on... if you've got 77 concurrent threads going, I don't see how that's a "sequential" I/O

Re: [zfs-discuss] ZFS write bursts cause short app stalls

2009-12-26 Thread Saso Kiselkov
ning would take me another 1-2 days. Regards, - -- Saso Leonid Kogan wrote: > Try b130. > http://genunix.org/ > > Cheers, > LK > > > On 12/26/2009 12:59 AM, Saso Kiselkov wrote: >> Hi, >> >> I tried it and I got the following error message: >> >

Re: [zfs-discuss] ZFS write bursts cause short app stalls

2009-12-26 Thread Saso Kiselkov
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The application I'm working on is a kind of large-scale network-PVR system for our IPTV services. It records all running TV channels in a X-hour carrousel (typically 24 or 48-hours), retaining only those bits which users have marked as being interestin

Re: [zfs-discuss] ZFS write bursts cause short app stalls

2009-12-25 Thread Saso Kiselkov
Hi, I tried it and I got the following error message: # zfs set logbias=throughput content cannot set property for 'content': invalid property 'logbias' Is it because I'm running some older version which does not have this feature? (2009.06) Regards, -- Saso Leonid Kogan wrote: > Hi there, > T

Re: [zfs-discuss] ZFS write bursts cause short app stalls

2009-12-25 Thread Saso Kiselkov
Hi, I'm not sure what "b130" means, I'm fairly new to OpenSolaris. How do I find out? As for the OS version, it is OpenSolaris 2009.06. Regards, -- Saso Richard Elling wrote: > On Dec 25, 2009, at 9:57 AM, Saso Kiselkov wrote: > > I've started portin

[zfs-discuss] ZFS write bursts cause short app stalls

2009-12-25 Thread Saso Kiselkov
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I've started porting a video streaming application to opensolaris on ZFS, and am hitting some pretty weird performance issues. The thing I'm trying to do is run 77 concurrent video capture processes (roughly 430Mbit/s in total) all writing into separat