Re: [PATCH] powerpc: Set a smaller value for RECLAIM_DISTANCE to enable zone reclaim

2010-03-01 Thread Christoph Lameter
On Mon, 1 Mar 2010, Mel Gorman wrote: > Christoph, how feasible would it be to allow parallel reclaimers in > __zone_reclaim() that back off at a rate depending on the number of > reclaimers? Not too hard. Zone locking is there but there may be a lot of bouncing cachelines if you run it concurren

Re: [PATCH] powerpc: Set a smaller value for RECLAIM_DISTANCE to enable zone reclaim

2010-03-01 Thread Mel Gorman
On Tue, Feb 23, 2010 at 12:55:51PM +1100, Anton Blanchard wrote: > > Hi Mel, > I'm back but a bit vague. Am on painkillers for the bashing I gave myself down the hills. > > You're pretty much on the button here. Only one thread at a time enters > > zone_reclaim. The others back off and try the

Re: [PATCH] powerpc: Set a smaller value for RECLAIM_DISTANCE to enable zone reclaim

2010-02-24 Thread Christoph Lameter
On Tue, 23 Feb 2010, Anton Blanchard wrote: > zone_reclaim_mode.txt > Now we set zone_reclaim_mode = 1. On each iteration we continue to improve, > but even after 10 runs of stream we have > 10% remote node memory usage. The intend of zone reclaim was never to allocate all memory from on node. Yo

Re: [PATCH] powerpc: Set a smaller value for RECLAIM_DISTANCE to enable zone reclaim

2010-02-23 Thread Mel Gorman
On Tue, Feb 23, 2010 at 12:55:51PM +1100, Anton Blanchard wrote: > > Hi Mel, > I'm afraid I'm on vacation at the moment. This mail is costing me shots with penaltys every minute it's open. It'll be early next week before I can look at this closely. Sorry. > > You're pretty much on the button

Re: [PATCH] powerpc: Set a smaller value for RECLAIM_DISTANCE to enable zone reclaim

2010-02-22 Thread Anton Blanchard
Hi Mel, > You're pretty much on the button here. Only one thread at a time enters > zone_reclaim. The others back off and try the next zone in the zonelist > instead. I'm not sure what the original intention was but most likely it > was to prevent too many parallel reclaimers in the same zone po

Re: [PATCH] powerpc: Set a smaller value for RECLAIM_DISTANCE to enable zone reclaim

2010-02-22 Thread Anton Blanchard
Hi Balbir, > A reclaim distance of 10 implies a ratio of 1, that means we'll always > do zone_reclaim() to free page cache and slab cache before moving on > to another node? I want to make an effort to reclaim local pagecache before ever going off node. As an example, a completely off node stre

Re: [PATCH] powerpc: Set a smaller value for RECLAIM_DISTANCE to enable zone reclaim

2010-02-19 Thread Balbir Singh
* Christoph Lameter [2010-02-19 09:51:12]: > On Fri, 19 Feb 2010, Balbir Singh wrote: > > > >> zone_reclaim. The others back off and try the next zone in the zonelist > > >> instead. I'm not sure what the original intention was but most likely it > > >> was to prevent too many parallel reclaimer

Re: [PATCH] powerpc: Set a smaller value for RECLAIM_DISTANCE to enable zone reclaim

2010-02-19 Thread Christoph Lameter
On Fri, 19 Feb 2010, Balbir Singh wrote: > >> zone_reclaim. The others back off and try the next zone in the zonelist > >> instead. I'm not sure what the original intention was but most likely it > >> was to prevent too many parallel reclaimers in the same zone potentially > >> dumping out way mor

Re: [PATCH] powerpc: Set a smaller value for RECLAIM_DISTANCE to enable zone reclaim

2010-02-19 Thread Balbir Singh
On Fri, Feb 19, 2010 at 3:59 AM, Anton Blanchard wrote: > > I noticed /proc/sys/vm/zone_reclaim_mode was 0 on a ppc64 NUMA box. It gets > enabled via this: > >        /* >         * If another node is sufficiently far away then it is better >         * to reclaim pages in a zone before going off n

Re: [PATCH] powerpc: Set a smaller value for RECLAIM_DISTANCE to enable zone reclaim

2010-02-19 Thread Balbir Singh
On Fri, Feb 19, 2010 at 8:42 PM, Christoph Lameter wrote: > On Fri, 19 Feb 2010, Mel Gorman wrote: > >> > > The patch below sets a smaller value for RECLAIM_DISTANCE and thus >> > > enables >> > > zone reclaim. >> > >> >> I've no problem with the patch anyway. > > Nor do I. > >> > - We seem to en

Re: [PATCH] powerpc: Set a smaller value for RECLAIM_DISTANCE to enable zone reclaim

2010-02-19 Thread Christoph Lameter
On Fri, 19 Feb 2010, Mel Gorman wrote: > > > The patch below sets a smaller value for RECLAIM_DISTANCE and thus enables > > > zone reclaim. > > > > I've no problem with the patch anyway. Nor do I. > > - We seem to end up racing between zone_watermark_ok, zone_reclaim and > > buffered_rmqueue.

Re: [PATCH] powerpc: Set a smaller value for RECLAIM_DISTANCE to enable zone reclaim

2010-02-19 Thread Mel Gorman
On Fri, Feb 19, 2010 at 11:07:30AM +1100, Anton Blanchard wrote: > > Hi, > > > The patch below sets a smaller value for RECLAIM_DISTANCE and thus enables > > zone reclaim. > I've no problem with the patch anyway. > FYI even with this enabled I could trip it up pretty easily with a multi > thre

Re: [PATCH] powerpc: Set a smaller value for RECLAIM_DISTANCE to enable zone reclaim

2010-02-18 Thread Anton Blanchard
Hi, > The patch below sets a smaller value for RECLAIM_DISTANCE and thus enables > zone reclaim. FYI even with this enabled I could trip it up pretty easily with a multi threaded application. I tried running stream across all threads in node 0. The machine looks like: node 0 cpus: 0 1 2 3 4 5 6

[PATCH] powerpc: Set a smaller value for RECLAIM_DISTANCE to enable zone reclaim

2010-02-18 Thread Anton Blanchard
I noticed /proc/sys/vm/zone_reclaim_mode was 0 on a ppc64 NUMA box. It gets enabled via this: /* * If another node is sufficiently far away then it is better * to reclaim pages in a zone before going off node. */ if (distance > RECLAIM_DISTANCE)