Re: mbuf leakage with nfs/zfs? (was: em0 freezes on ZFS server)

2010-02-26 Thread Gerrit Kühn
On Fri, 26 Feb 2010 22:09:32 +0200 Daniel Braniss wrote about Re: mbuf leakage with nfs/zfs? (was: em0 freezes on ZFS server) : DB> at the moment there is not much activity, but if you check the latest DB> plot.ps you will see that the bottom is slowly increasing, so my bet DB> is t

Re: mbuf leakage with nfs/zfs? (was: em0 freezes on ZFS server)

2010-02-26 Thread Gerrit Kühn
On Fri, 26 Feb 2010 22:09:32 +0200 Daniel Braniss wrote about Re: mbuf leakage with nfs/zfs? (was: em0 freezes on ZFS server) : DB> > Furthermore I found this via Google: DB> > <http://lists.freebsd.org/pipermail/freebsd-current/2009-December/014062.html>. This did not hel

Re: mbuf leakage with nfs/zfs? (was: em0 freezes on ZFS server)

2010-02-26 Thread Daniel Braniss
> On Fri, 26 Feb 2010 17:41:02 +0200 Daniel Braniss > wrote about Re: em0 freezes on ZFS server : > > DB> check: > DB> ftp://ftp.cs.huji.ac.il/users/danny/freebsd/plot.ps > DB> x is seconds, y is mbus current. > > Looks not as bad as mine. I had 37k when I re

mbuf leakage with nfs/zfs? (was: em0 freezes on ZFS server)

2010-02-26 Thread Gerrit Kühn
On Fri, 26 Feb 2010 17:41:02 +0200 Daniel Braniss wrote about Re: em0 freezes on ZFS server : DB> check: DB> ftp://ftp.cs.huji.ac.il/users/danny/freebsd/plot.ps DB> x is seconds, y is mbus current. Looks not as bad as mine. I had 37k when I rebooted the machine some minutes ago

Re: em0 freezes on ZFS server

2010-02-26 Thread Gerrit Kühn
On Fri, 26 Feb 2010 17:07:13 +0200 Daniel Braniss wrote about Re: em0 freezes on ZFS server : DB> it's only purpose in life is a nfs server. I thought so, but you did not mention it explicitely. DB> but I wouldn't exclude zfs from the equation yet. DB> I have othere nfs ser

Re: em0 freezes on ZFS server

2010-02-26 Thread Daniel Braniss
> when I have enough data i'll plot it. > check: ftp://ftp.cs.huji.ac.il/users/danny/freebsd/plot.ps x is seconds, y is mbus current. > Cheers, > danny > > > ___ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailm

Re: em0 freezes on ZFS server

2010-02-26 Thread Willem Jan Withagen
On 26-2-2010 16:07, Daniel Braniss wrote: On Fri, 26 Feb 2010 15:04:37 +0200 Daniel Braniss wrote about Re: em0 freezes on ZFS server : DB> > At least in my case I found out what is eating the buffers: nfsd DB> > does! The buffers stop increasing as soon as I stop nfsd.

Re: em0 freezes on ZFS server

2010-02-26 Thread Daniel Braniss
> On Fri, 26 Feb 2010 15:04:37 +0200 Daniel Braniss > wrote about Re: em0 freezes on ZFS server : > > DB> > At least in my case I found out what is eating the buffers: nfsd > DB> > does! The buffers stop increasing as soon as I stop nfsd. However, > DB> > they

Re: em0 freezes on ZFS server

2010-02-26 Thread Daniel Braniss
> On Fri, 26 Feb 2010 13:31:38 +0100 Gerrit Kühn > wrote about Re: em0 freezes on ZFS server: > > GK> JC> Note how close the "current" value is to that of "total". I'm not > GK> JC> too surprised you're seeing what you are as a result

Re: em0 freezes on ZFS server

2010-02-26 Thread Gerrit Kühn
On Fri, 26 Feb 2010 15:04:37 +0200 Daniel Braniss wrote about Re: em0 freezes on ZFS server : DB> > At least in my case I found out what is eating the buffers: nfsd DB> > does! The buffers stop increasing as soon as I stop nfsd. However, DB> > they start increasing as soon as

Re: em0 freezes on ZFS server

2010-02-26 Thread Willem Jan Withagen
On 26-2-2010 13:44, Gerrit Kühn wrote: On Fri, 26 Feb 2010 13:31:38 +0100 Gerrit Kühn wrote about Re: em0 freezes on ZFS server: GK> JC> Note how close the "current" value is to that of "total". I'm not GK> JC> too surprised you're seeing what y

Re: em0 freezes on ZFS server

2010-02-26 Thread Gerrit Kühn
On Fri, 26 Feb 2010 13:31:38 +0100 Gerrit Kühn wrote about Re: em0 freezes on ZFS server: GK> JC> Note how close the "current" value is to that of "total". I'm not GK> JC> too surprised you're seeing what you are as a result of this. GK> JC> What

Re: em0 freezes on ZFS server

2010-02-26 Thread Gerrit Kühn
On Fri, 26 Feb 2010 04:03:39 -0800 Jeremy Chadwick wrote about Re: em0 freezes on ZFS server: JC> Note how close the "current" value is to that of "total". I'm not too JC> surprised you're seeing what you are as a result of this. What on JC> earth is

Re: em0 freezes on ZFS server

2010-02-26 Thread Willem Jan Withagen
On 26-2-2010 13:16, Gerrit Kühn wrote: On Thu, 25 Feb 2010 14:59:28 -0800 Jack Vogel wrote about Re: em0 freezes on ZFS server: JV> The failure to "setup receive structures" means it did not have JV> sufficient mbufs JV> to setup the RX ring and buffer structs. I'm

Re: em0 freezes on ZFS server

2010-02-26 Thread Willem Jan Withagen
On 26-2-2010 13:03, Jeremy Chadwick wrote: On Fri, Feb 26, 2010 at 10:34:41AM +0100, Willem Jan Withagen wrote: This is wat netstat -m told me when it refused to revive em0: Below are the netstat -m counters/lines of concern: 24980/2087/27067 mbufs in use (current/cache/total) 24530/1070/256

Re: em0 freezes on ZFS server

2010-02-26 Thread Jeremy Chadwick
On Fri, Feb 26, 2010 at 10:34:41AM +0100, Willem Jan Withagen wrote: > This is wat netstat -m told me when it refused to revive em0: Below are the netstat -m counters/lines of concern: > 24980/2087/27067 mbufs in use (current/cache/total) > 24530/1070/25600/25600 mbuf clusters in use (current/cac

Re: em0 freezes on ZFS server

2010-02-26 Thread Gerrit Kühn
On Thu, 25 Feb 2010 14:59:28 -0800 Jack Vogel wrote about Re: em0 freezes on ZFS server: JV> The failure to "setup receive structures" means it did not have JV> sufficient mbufs JV> to setup the RX ring and buffer structs. I'm monitoring mbufs since I rebooted my serv

Re: em0 freezes on ZFS server

2010-02-26 Thread Gerrit Kühn
On Thu, 25 Feb 2010 14:59:28 -0800 Jack Vogel wrote about Re: em0 freezes on ZFS server: JV> The failure to "setup receive structures" means it did not have JV> sufficient mbufs JV> to setup the RX ring and buffer structs. I don't know if this is related, but I upda

Re: em0 freezes on ZFS server

2010-02-26 Thread Gerrit Kühn
On Fri, 26 Feb 2010 10:34:41 +0100 Willem Jan Withagen wrote about Re: em0 freezes on ZFS server: WJW> Probably the reason why this happened yesterday is that I started WJW> doing major software builds (over ZFS/NFS/TCP/v3) against data stored WJW> on this box. I saw a similar pro

Re: em0 freezes on ZFS server

2010-02-26 Thread Willem Jan Withagen
On 26-2-2010 10:58, Gerrit Kühn wrote: On Fri, 26 Feb 2010 10:34:41 +0100 Willem Jan Withagen wrote about Re: em0 freezes on ZFS server: WJW> Probably the reason why this happened yesterday is that I started WJW> doing major software builds (over ZFS/NFS/TCP/v3) against data stored WJ

Re: em0 freezes on ZFS server

2010-02-26 Thread Willem Jan Withagen
On 25-2-2010 23:59, Jack Vogel wrote: The failure to "setup receive structures" means it did not have sufficient mbufs to setup the RX ring and buffer structs. Not sure why this results in a lockup, but try and increase kern.ipc.nmbclusters. Let me know what happens, I've doubled the value 256

Re: em0 freezes on ZFS server

2010-02-25 Thread Jack Vogel
The failure to "setup receive structures" means it did not have sufficient mbufs to setup the RX ring and buffer structs. Not sure why this results in a lockup, but try and increase kern.ipc.nmbclusters. Let me know what happens, Jack On Thu, Feb 25, 2010 at 2:02 PM, Willem Jan Withagen wrote:

em0 freezes on ZFS server

2010-02-25 Thread Willem Jan Withagen
Hi, I build an ZFS server based on Supermicro C2SBX with 8Gb and intel Q9550 http://www.supermicro.com/products/motherboard/Core2Duo/X38/C2SBX.cfm with an areca 1120 with 8*1,5T It has an Inter em0, as in: Feb 25 14:46:29 zfs kernel: pci0: on pcib0 Feb 25 14:46:29 zfs kernel: em