Re: Packet loss every 30.999 seconds

2007-12-17 Thread David G Lawrence
continue; } ...like the i_flag flags aren't ever getting properly cleared (or bv_cnt is always non-zero). ...but I don't have the time to chase this down. -DG David G. Lawrence President Download Technologies, Inc. - http://www.downloadtech.com - (866) 399

Re: Packet loss every 30.999 seconds

2007-12-17 Thread David G Lawrence
MNT_ILOCK(mp); + if (flushed_count++ > 500) { + flushed_count = 0; + msleep(&flushed_count, MNT_MTX(mp), PZERO, "syncw", 1); + } } MNT_IUNLOCK(mp); /* -DG David G.

Re: Packet loss every 30.999 seconds

2007-12-18 Thread David G Lawrence
order to get the vnodes allocated. As I mentioned previously, I suspect that either ip->i_flag is not getting completely cleared in ffs_syncvnode or its children or v_bufobj.bo_dirty.bv_cnt accounting is broken. -DG David G. Lawrence President Download Technologies, Inc

Re: Packet loss every 30.999 seconds

2007-12-18 Thread David G Lawrence
instead of /dev/null, if you use GNU tar, to disable its "optimization"). You can stop it after it has gone through a 100K files. Verify by looking at "sysctl vfs.numvnodes". Doing this would help to further prove that lots of allocated vnodes is the prerequisite for the p

Re: Packet loss every 30.999 seconds

2007-12-18 Thread David G Lawrence
> On Tue, 18 Dec 2007, David G Lawrence wrote: > > >>Thanks. Have a kernel building now. It takes about a day of uptime > >>after reboot before I'll see the problem. > > > > You may also wish to try to get the problem to occur sooner after boot >

Re: Packet loss every 30.999 seconds

2007-12-18 Thread David G Lawrence
. I'm going to have to bow out of this discussion now. I just don't have the time for it. -DG David G. Lawrence President Download Technologies, Inc. - http://www.downloadtech.com - (866) 399 8500 The FreeBSD Project - http://www.freebsd.org P

Re: Packet loss every 30.999 seconds

2007-12-18 Thread David G Lawrence
runs at 150MHz when it is in the lowest running CPU power save mode. At that speed, this bug causes a delay of more than 300ms and is enough to cause loss of keyboard input. I have to switch into high speed mode before I try to type anything, else I end up with random typos. Very annoying. -DG Dav

Re: Packet loss every 30.999 seconds

2007-12-19 Thread David G Lawrence
> On Tue, 18 Dec 2007, David G Lawrence wrote: > > >>>I got an almost identical delay (with 64000 vnodes). > >>> > >>>Now, 17ms isn't much. > >> > >> Says you. On modern systems, trying to run a pseudo real-time > >>

Re: Packet loss every 30.999 seconds

2007-12-19 Thread David G Lawrence
ittle bit). I guess that got removed when the size of the vnode pool was dramatically increased. -DG David G. Lawrence President Download Technologies, Inc. - http://www.downloadtech.com - (866) 399 8500 The FreeBSD Project - http://www.freebsd.org Pave the road of life

Re: Packet loss every 30.999 seconds

2007-12-19 Thread David G Lawrence
1 and then setting it back to whatever it was previously (probably 6-10). If the problem then goes away for awhile, that would be another good indicator. -DG David G. Lawrence President Download Technologies, Inc. - http://www.downloadtech.com - (866) 399 8500 The FreeBSD Project - ht

Re: Packet loss every 30.999 seconds

2007-12-19 Thread David G Lawrence
in problem is to figure out why PREEMPTION doesn't work. I'm > not working on this directly since I'm running ~5.2 where nearly-full > kernel preemption doesn't work due to Giant locking. I don't understand how PREEMPTION is supposed to work (I mean to any sig

Re: Packet loss every 30.999 seconds

2007-12-19 Thread David G Lawrence
might be low enough to not trigger the problem, but also be high enough to not significantly affect system I/O performance. -DG David G. Lawrence President Download Technologies, Inc. - http://www.downloadtech.com - (866) 399 8500 The FreeBSD Project - http://www.freebsd.org Pave the road of life

Re: Packet loss every 30.999 seconds

2007-12-21 Thread David G Lawrence
ificantly increase the overhead of the loop. The solution provided by Kostik Belousov that uses uio_yield looks like a find solution. I intend to try it out on some servers RSN. -DG David G. Lawrence President Download Technologies, Inc. - http://www.downloadtech.com - (866) 399 8500 The Fr

Re: Packet loss every 30.999 seconds

2007-12-21 Thread David G Lawrence
cond version is that uio_yield doesn't lower the priority enough for the other threads to run. Forcing it to msleep for a tick will eliminate the priority from the consideration. -DG David G. Lawrence President Download Technologies, Inc. - http://www.downloadtech.com - (866) 399 8500 T

Re: Packet loss every 30.999 seconds

2007-12-21 Thread David G Lawrence
truct, that has on it any vnodes that need to be synced. Unfortuantely, such a change would be extensive, scattered throughout much of the ufs/ffs code. -DG David G. Lawrence President Download Technologies, Inc. - http://www.downloadtech.com - (866) 399 8500 The Fre

Re: Packet loss every 30.999 seconds

2007-12-21 Thread David G Lawrence
y when you have files modified through the mmap interface (kind of rare on most systems). Obviously I have mixed feelings about vfs_msync, but I'm not suggesting here that we should get rid of it as any sort of solution. -DG David G. Lawrence President Download Technologies,

Re: Packet loss every 30.999 seconds

2007-12-21 Thread David G Lawrence
ready uses a marker vnode. It is hidden and obfuscated in the MNT_VNODE_FOREACH macro, further hidden in the __mnt_vnode_first/next functions, so it should be safe from vnode reclaimation/free problems. -DG David G. Lawrence President Download Technologies, Inc. - http://www.downloadtech.com - (

Re: complement to sendfile()?

2006-07-19 Thread David G. Lawrence
network card and it could DMA the TCP stream directly into file buffers. If pigs had wings, they could fly. :-) -DG David G. Lawrence President Download Technologies, Inc. - http://www.downloadtech.com - (866) 399 8500 The FreeBSD Project - http://www.freebsd.

Re: complement to sendfile()?

2006-07-19 Thread David G. Lawrence
> ?? 19 ?? 2006 22:50, David G. Lawrence ???: > > ? ?sendfile() could be extended to allow arbitrary file descriptor types as > > the source and destination, but the zero-copy nature of it can only work > > in the file to socket direction. This is because n