Writing tests for write back

2008-01-07 Thread Michael Rubin
I am writing tests for the write back section of the kernel. This is the stuff found in fs/fs-writeback.c. Currently I have tests to fsync millions of inodes concurrently, tests involving mounting and unmounting file systems, and other tests to tickle some starvation of big file situations. Has an

Re: Possible fix for lockup in drop_caches

2008-01-08 Thread Michael Rubin
On Dec 22, 2007 2:06 AM, Andrew Morton <[EMAIL PROTECTED]> wrote: > Oh boy. Do we really want to add all this stuff to JBD just for > drop_caches which is a silly root-only broken-in-22-other-ways thing? > > Michael, might your convert-inode-lists-to-tree patches eliminate the need > for taking in

Re: Possible fix for lockup in drop_caches

2008-01-08 Thread Michael Rubin
On Dec 22, 2007 2:06 AM, Andrew Morton <[EMAIL PROTECTED]> wrote: > On Mon, 17 Dec 2007 12:13:22 + richard <[EMAIL PROTECTED]> wrote: > Michael, might your convert-inode-lists-to-tree patches eliminate the need > for taking inode_lock in drop_pagecache_sb()? Probably not, as it uses an > rbtre

[patch 1/1] Writeback fix for concurrent large and small file writes.

2007-12-10 Thread Michael Rubin
From: Michael Rubin <[EMAIL PROTECTED]> Fixing a bug where writing to large files while concurrently writing to smaller ones creates a situation where writeback cannot keep up with the traffic and memory baloons until the we hit the threshold watermark. This can result in surprising l

Re: [patch 1/1] Writeback fix for concurrent large and small file writes.

2007-12-12 Thread Michael Rubin
On Dec 12, 2007 12:55 PM, Peter Zijlstra <[EMAIL PROTECTED]> wrote: > > On Mon, 2007-12-10 at 18:02 -0800, Michael Rubin wrote: > > From: Michael Rubin <[EMAIL PROTECTED]> > The part I miss here is the rationale on _how_ you solve the problem. > > The patch itsel

[patch] Converting writeback linked lists to a tree based data structure

2007-12-12 Thread Michael Rubin
From: Michael Rubin <[EMAIL PROTECTED]> This is an attempt to unify the writeback data structures. By adding an rb tree we are able to have one consistent time ordering mechanism for writeback. This should aid debugging and allow for future work with more sophisticated time ordering methods

[patch 1/1] Writeback fix for concurrent large and small file writes

2007-11-28 Thread Michael Rubin
>From [EMAIL PROTECTED] Wed Nov 28 11:10:06 2007 Message-Id: <[EMAIL PROTECTED]> Date: Wed, 28 Nov 2007 11:01:21 -0800 From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: [patch 1/1] Writeback fix for concurrent large and small file writes. From: Michael Rubin <[EMAIL PROTECTED]>

Re: [patch 1/1] Writeback fix for concurrent large and small file writes

2007-11-28 Thread Michael Rubin
Thank you. Integrated the fixes in my patch. On Nov 28, 2007 6:13 PM, Frans Pop <[EMAIL PROTECTED]> wrote: > Two typos in comments. > > Cheers, > FJP > > Michael Rubin wrote: > > + * The flush tree organizes the dirtied_when keys with the rb_tree. Any > > + *

Re: [patch 1/1] Writeback fix for concurrent large and small file writes

2007-11-29 Thread Michael Rubin
at happens. mrubin On Nov 28, 2007 4:34 PM, Fengguang Wu <[EMAIL PROTECTED]> wrote: > On Wed, Nov 28, 2007 at 11:29:57AM -0800, Michael Rubin wrote: > > >From [EMAIL PROTECTED] Wed Nov 28 11:10:06 2007 > > Message-Id: <[EMAIL PROTECTED]> > > Date: Wed, 28 Nov

Re: [patch 1/1] Writeback fix for concurrent large and small file writes

2007-11-29 Thread Michael Rubin
Due to my faux pas of top posting (see http://www.zip.com.au/~akpm/linux/patches/stuff/top-posting.txt) I am resending this email. On Nov 28, 2007 4:34 PM, Fengguang Wu <[EMAIL PROTECTED]> wrote: > Could you demonstrate the situation? Or if I guess it right, could it > be fixed by the following pa

Re: [patch 1/1] Writeback fix for concurrent large and small file writes

2007-12-04 Thread Michael Rubin
On Nov 29, 2007 5:32 PM, Fengguang Wu <[EMAIL PROTECTED]: > > On Nov 28, 2007 4:34 PM, Fengguang Wu <[EMAIL PROTECTED]> wrote: > > > Could you demonstrate the situation? Or if I guess it right, could it > > > be fixed by the following patch? Feng I am sorry to have been mistaken but I reran my tes

[patch] Converting writeback linked lists to a tree based data structure

2008-01-15 Thread Michael Rubin
From: Michael Rubin <[EMAIL PROTECTED]> For those of you who have waited so long. This is the third submission of the first attempt at this patch. It is a trilogy. Two changes are in this patch. They are dependant on each other. In addition we get an unintended performance improvement. S

Re: [patch] Converting writeback linked lists to a tree based data structure

2008-01-15 Thread Michael Rubin
On Jan 15, 2008 12:46 AM, Peter Zijlstra <[EMAIL PROTECTED]> wrote: > Just a quick question, how does this interact/depend-uppon etc.. with > Fengguangs patches I still have in my mailbox? (Those from Dec 28th) They don't. They apply to a 2.6.24rc7 tree. This is a candidte for 2.6.25. This work w

Re: [PATCH 00/13] writeback bug fixes and simplifications take 2

2008-01-15 Thread Michael Rubin
On Jan 15, 2008 4:36 AM, Fengguang Wu <[EMAIL PROTECTED]> wrote: > Andrew, > > This patchset mainly polishes the writeback queuing policies. Anyone know which tree is this patched based out of? > The main goals are: > > (1) small files should not be starved by big dirty files > (2) sync as fast a

Re: [patch] Converting writeback linked lists to a tree based data structure

2008-01-16 Thread Michael Rubin
On Jan 15, 2008 7:01 PM, Fengguang Wu <[EMAIL PROTECTED]> wrote: > Basically I think rbtree is an overkill to do time based ordering. > Sorry, Michael. But s_dirty would be enough for that. Plus, s_more_io > provides fair queuing between small/large files, and s_more_io_wait > provides waiting mech

Crash with 2.6.24-rc6-mm1 in restore_i387_ia32

2008-01-17 Thread Michael Rubin
When I boot my system with 2.6.24-rc6 everything is great. When I apply the mm1 patch I crash with the output below. Anyone seen this before? Starting sendmail: Unable to handle kernel paging request at 00010013 RIP: [] restore_i387_ia32+0x1f/0x150 PGD 47e134067 PUD 0 Oops: [1] SM

Re: [patch] Converting writeback linked lists to a tree based data structure

2008-01-17 Thread Michael Rubin
On Jan 17, 2008 1:41 AM, Fengguang Wu <[EMAIL PROTECTED]> wrote: > On Tue, Jan 15, 2008 at 12:09:21AM -0800, Michael Rubin wrote: > The main benefit of rbtree is possibly better support of future policies. > Can you demonstrate an example? These are ill-formed thoughts as of now on

Re: [patch] Converting writeback linked lists to a tree based data structure

2008-01-17 Thread Michael Rubin
On Jan 17, 2008 9:01 PM, David Chinner <[EMAIL PROTECTED]> wrote: First off thank you for the very detailed reply. This rocks and gives me much to think about. > On Thu, Jan 17, 2008 at 01:07:05PM -0800, Michael Rubin wrote: > This seems suboptimal for large files. If you keep feed

Re: [patch] Converting writeback linked lists to a tree based data structure

2008-01-17 Thread Michael Rubin
On Jan 17, 2008 8:56 PM, Fengguang Wu <[EMAIL PROTECTED]> wrote: Once again thanks for the speedy replies. :-) > On Thu, Jan 17, 2008 at 01:07:05PM -0800, Michael Rubin wrote: > Suppose we want to grant longer expiration window for temp files, > adding a new list named s_dirty_tmp

Re: [PATCH 00/13] writeback bug fixes and simplifications take 2

2008-01-17 Thread Michael Rubin
On Jan 15, 2008 4:36 AM, Fengguang Wu <[EMAIL PROTECTED]> wrote: > Andrew, > > This patchset mainly polishes the writeback queuing policies. > The main goals are: > > (1) small files should not be starved by big dirty files > (2) sync as fast as possible for not-blocked inodes/pages > - don't l

Re: [patch] Converting writeback linked lists to a tree based data structure

2008-01-18 Thread Michael Rubin
On Jan 18, 2008 12:54 AM, David Chinner <[EMAIL PROTECTED]> wrote: > At this point, I'd say it is best to leave it to the filesystem and > the elevator to do their jobs properly. Amen. mrubin -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [E