Re: [zfs-discuss] Slow death-spiral with zfs gzip-9 compression

2008-11-30 Thread Ray Clark
This has been an evolution, see defect.opensolaris.com's 5482 memory leak with top. I have not run anything bug gzip-9 since fixing that by only running top in one-time mode. I will start the same copy with compress=off in about 45 minutes (Got to go do an errand now). Glad to run tests. --R

Re: [zfs-discuss] Slow death-spiral with zfs gzip-9 compression

2008-11-30 Thread Karl Hakimian
> This is about not having kernel threads completely > lock out user (and other kernel) processes for > undesirable lengths of time. It is about improving > Solaris. It is about having more appropriate CPU > sharing between all of the threads in the system, > kernel and user. This is the root cau

Re: [zfs-discuss] Slow death-spiral with zfs gzip-9 compression

2008-11-30 Thread Miles Nordin
> "t" == Tim <[EMAIL PROTECTED]> writes: > "ic" == Ian Collins <[EMAIL PROTECTED]> writes: > "mp" == Mattias Pantzare <[EMAIL PROTECTED]> writes: t> My point is you're not looking at the bigger picture. And you're not hearing this: * 100KByte/s * sudden slowdown 30x after 12

Re: [zfs-discuss] Slow death-spiral with zfs gzip-9 compression

2008-11-30 Thread Ray Clark
Unless something new develops, from my perspective this thread has served its purpose. I don't want to drag it out and waste people's time. If any "late readers" have insights that should be captured for the archive please add them. Thank you all VERY much for the discussion, insights, suggest

Re: [zfs-discuss] Slow death-spiral with zfs gzip-9 compression

2008-11-30 Thread Ray Clark
> > I think Chris has the right idea. This would give more little opportunities > > for user > > processes to get a word in edgewise. Since the blocks are *obviously* > > taking a > > LONG time, this would not be a big hit efficiency in the bogged-down > > condition. > I still think you are exp

Re: [zfs-discuss] HP Smart Array and b99?

2008-11-30 Thread sim
Hi, I've got similar problem. I'm trying to install Opensolaris on HP blade 460c. My problem is that the newest version (snv_103) hangs during boot. After reading your post I started testing old version and. Every installation up to snv_98 is ok, snv_99 and newer hangs. As I'm using origina

Re: [zfs-discuss] Problem importing degraded Pool

2008-11-30 Thread Philipp Haußleiter
i found some tasks to do, to get more information about the problem. zpool import tank tells me: cannot import 'tank': pool may be in use from other system use '-f' to import anyway zpool import -f tank cannot import 'tank': no such pool or dataset using zpool import fmdump -eV gives me N

Re: [zfs-discuss] Slow death-spiral with zfs gzip-9 compression

2008-11-30 Thread Ross
I'd agree there Ian, but this still sounds like a good idea to me. I don't like any system becoming unresponsive, and this sounds like a good general purpose tweak to help Solaris cope with worst case scenarios. -- This message posted from opensolaris.org ___

Re: [zfs-discuss] Solved - a big THANKS to Victor Latushkin @ Sun / Moscow

2008-11-30 Thread Ray Clark
It would be extremely helpful to know what brands/models of disks lie and which don't. This information could be provided diplomatically simply as threads documenting problems you are working on, stating the facts. Use of a specific string of words would make searching for it easy. There shou

Re: [zfs-discuss] Problem importing degraded Pool

2008-11-30 Thread Philipp Haußleiter
zdb on the two devices told me: zdb -l /dev/rdsk/c0t2d0 LABEL 0 failed to unpack label 0 LABEL 1 failed to unpack lab

Re: [zfs-discuss] Slow death-spiral with zfs gzip-9 compression

2008-11-30 Thread Ray Clark
I just want to interject here that I think, if memory serves me correctly, that SPARC has been 64 bit for 10~15 years, and so had LOTS of address space to map stuff. x86 brought a new restriction. Regarding the practice of mapping files etc. into virtual memory that does not exist, now I under

Re: [zfs-discuss] Slow death-spiral with zfs gzip-9 compression

2008-11-30 Thread Ray Clark
Re: I don't believe either are bundled. Search google for arcstat.pl and nicstat.pl -- Thanks. On a related note, there are at leaset 2 or 3 places proclaiming to provide Solaris 10 packages. How do I know which ones are "safe", both in terms of quality, and in terms of adhering to a consist

Re: [zfs-discuss] Slow death-spiral with zfs gzip-9 compression

2008-11-30 Thread Ian Collins
Ray Clark wrote: > Re: gzip compression works a lot better now the compression is threaded. > It's a shame userland gzip isn't! > > --- > What does "now than" mean? I didn't type that. > I assume you mean the zfs / kernel gzip (right?) at some point became > threaded. Is this in the past, or

Re: [zfs-discuss] Slow death-spiral with zfs gzip-9 compression

2008-11-30 Thread Ray Clark
Re: Experimentation will show that the compression ratio does not increase much at -9 so it is not worth it when you are short on time or CPU. --- Yes, and a large part of my experiment was to understand the cost (time) vs. compression ratio curve. lj?? only gave me 7%, which to me is not worth

Re: [zfs-discuss] Slow death-spiral with zfs gzip-9 compression

2008-11-30 Thread Ian Collins
Ray Clark wrote: [some context would be nice for mail list subscribers] > I think Chris has the right idea. This would give more little opportunities > for user processes to get a word in edgewise. Since the blocks are > *obviously* taking a LONG time, this would not be a big hit efficiency i

Re: [zfs-discuss] Slow death-spiral with zfs gzip-9 compression

2008-11-30 Thread Ray Clark
Re: gzip compression works a lot better now the compression is threaded. It's a shame userland gzip isn't! --- What does "now than" mean? I assume you mean the zfs / kernel gzip (right?) at some point became threaded. Is this in the past, or in a kernel post 208.11 B2? --Ray -- This message

Re: [zfs-discuss] Slow death-spiral with zfs gzip-9 compression

2008-11-30 Thread Ray Clark
I think Chris has the right idea. This would give more little opportunities for user processes to get a word in edgewise. Since the blocks are *obviously* taking a LONG time, this would not be a big hit efficiency in the bogged-down condition. It would however increase overhead in the well-be

Re: [zfs-discuss] Slow death-spiral with zfs gzip-9 compression

2008-11-30 Thread Ray Clark
Andewk8 at 11:39 on 11/29 said: Solaris reports "virtual memory" as the sum of physical memory and page file - so this is where your strange vmstat output comes from. Running ZFS stress tests on a system with only 768MB of memory is not a good idea since ZFS uses large amounts of memory for its

Re: [zfs-discuss] Slow death-spiral with zfs gzip-9 compression

2008-11-30 Thread Ian Collins
Chris Ridd wrote: > On 30 Nov 2008, at 02:59, Bob Friesenhahn wrote: > > >> On Sun, 30 Nov 2008, Ian Collins wrote: >> >> >>> What did you expect? A 3GHz Opteron core takes about a minutes to >>> attempt to compress a 1GB .mkv file. So your P3 would probably take >>> between 5 and 10 minu

Re: [zfs-discuss] Slow death-spiral with zfs gzip-9 compression

2008-11-30 Thread Chris Ridd
On 30 Nov 2008, at 02:59, Bob Friesenhahn wrote: > On Sun, 30 Nov 2008, Ian Collins wrote: > >> What did you expect? A 3GHz Opteron core takes about a minutes to >> attempt to compress a 1GB .mkv file. So your P3 would probably take >> between 5 and 10 minutes. Now move that to the kernel and