On 04 January, 2007 - Tomas Ögren sent me these 1,0K bytes:
> On 03 January, 2007 - [EMAIL PROTECTED] sent me these 0,5K bytes:
>
> >
> > >Hmmm, so there is lots of evictable cache here (mostly in the MFU
> > >part of the cache)... could you make your core file available?
> > >I would like to take a look at it.
> >
> > Isn't this just like:
> > 6493923 nfsfind on ZFS filesystem quickly depletes memory in a 1GB system
> >
> > Which was introduced in b51(or 52) and fixed in snv_54.
>
> I've upgraded to snv54 now and even forced 500k dnlc entries in memory
> (ncsize=500000,arc_reduce_dnlc_percent=0).. Let's see how it copes with
> that ;)
Not too well.. Just went into swap hell.. and why is anon/exec&lib only
using 5+2 MB when there's about 50MB marked as Free?
> arc_reduce_dnlc_percent/D
arc_reduce_dnlc_percent:
arc_reduce_dnlc_percent: 0
> dnlc_nentries/D
dnlc_nentries:
dnlc_nentries: 499126
> freemem/J
freemem:
freemem: 9ea
> ::memstat
Page Summary Pages MB %Tot
------------ ---------------- ---------------- ----
Kernel 248982 1945 97%
Anon 730 5 0%
Exec and libs 271 2 0%
Page cache 38 0 0%
Free (cachelist) 738 5 0%
Free (freelist) 6108 47 2%
Total 256867 2006
Physical 253028 1976
> arc_reduce_dnlc_percent/W 0t3
arc_reduce_dnlc_percent: 0 = 0x3
> ::memstat
Page Summary Pages MB %Tot
------------ ---------------- ---------------- ----
Kernel 217511 1699 85%
Anon 1502 11 1%
Exec and libs 468 3 0%
Page cache 48 0 0%
Free (cachelist) 333 2 0%
Free (freelist) 37005 289 14%
Total 256867 2006
Physical 253028 1976
> dnlc_nentries/D
dnlc_nentries:
dnlc_nentries: 15183
> freemem/J
freemem:
freemem: 9351
> arc::print
{
size = 0xc34aa00
p = 0x4000000
c = 0x4000000
c_min = 0x4000000
c_max = 0x5e114800
> ARC_mru::print
{
lsize = 0x4039400
size = 0x8bf9e00
> ARC_mru_ghost::print
{
lsize = 0x5a6c00
size = 0x5a6c00
> ARC_mfu::print
{
lsize = 0x2308800
size = 0x2fd4800
> ARC_mfu_ghost::print
{
lsize = 0xd8000
size = 0xd8000
Anything useful there? I did a savecore -L after this info was gathered
(it was too slow to be useful before that.. had to wait a long time for
mdb to show up), so a dump is available..
I also have a logfile of ~every minute with ::kmastat, ::memstat and
dnlc_nentries from boot up until it started having problems, if someone
knows what to look for..
Seems like Kernel used up around 84% steadily, then all of a sudden it
started growing and within minutes Kernel was doing 97% and the machine
more or less stopped moving..
It could be related to TSM starting backups (which it did about 1h
before the "issue").. and after it had looked through whatever dnlc had
cached, stuff went haywire.. </end theory> :)
/Tomas
--
Tomas Ögren, [EMAIL PROTECTED], http://www.acc.umu.se/~stric/
|- Student at Computing Science, University of Umeå
`- Sysadmin at {cs,acc}.umu.se
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
