Re: crash in kmem_cache_init

2008-01-23 Thread Mel Gorman
On (23/01/08 13:14), Olaf Hering didst pronounce: > On Wed, Jan 23, Mel Gorman wrote: > > > Sorry this is dragging out. Can you post the full dmesg with loglevel=8 of > > the > > following patch against 2.6.24-rc8 please? It contains the debug information > > that helped me figure out what was go

Re: crash in kmem_cache_init

2008-01-23 Thread Olaf Hering
earlier patches, no idea. --- mm/slab.c | 17 + 1 file changed, 13 insertions(+), 4 deletions(-) --- a/mm/slab.c +++ b/mm/slab.c @@ -1590,7 +1590,7 @@ void __init kmem_cache_init(void) /* Replace the static kmem_list3 structures for the boot cpu */ init_list(

Re: crash in kmem_cache_init

2008-01-23 Thread Olaf Hering
h table entries: 262144 (order: 9, 2097152 bytes) freeing bootmem node 1 Memory: 3496632k/3571712k available (6188k kernel code, 75080k reserved, 1324k data, 1220k bss, 304k init) Online nodes o 0 o 1 Nodes with regular memory o 1 Current running CPU 0 is associated with node 0 Current node is 0

Re: crash in kmem_cache_init

2008-01-23 Thread Mel Gorman
ock_init(&ptr->list_lock); MAKE_ALL_LISTS(cachep, ptr, nodeid); + printk("init_list RESETTING %s node %d\n", cachep->name, nodeid); cachep->nodelists[nodeid] = ptr; local_irq_enable(); } @@ -1427,10 +1433,23 @@ void __init kmem_cache_init(void)

Re: crash in kmem_cache_init

2008-01-23 Thread Olaf Hering
On Wed, Jan 23, Pekka Enberg wrote: > Hi Christoph, > > On Jan 23, 2008 1:18 AM, Christoph Lameter <[EMAIL PROTECTED]> wrote: > > My patch is useless (fascinating history of the changelog there through). > > fallback_alloc calls kmem_getpages without GFP_THISNODE. This means that > > alloc_pages_

Re: crash in kmem_cache_init

2008-01-23 Thread Pekka Enberg
Hi Christoph, On Jan 23, 2008 1:18 AM, Christoph Lameter <[EMAIL PROTECTED]> wrote: > My patch is useless (fascinating history of the changelog there through). > fallback_alloc calls kmem_getpages without GFP_THISNODE. This means that > alloc_pages_node() will try to allocate on the current node b

Re: crash in kmem_cache_init

2008-01-22 Thread Olaf Hering
(6188k kernel code, 75080k reserved, 1324k data, 1220k bss, 304k init) Kernel panic - not syncing: kmem_cache_create(): failed to create slab `size-32(DMA)' Rebooting in 1 seconds.. --- mm/slab.c | 17 ++--- 1 file changed, 14 insertions(+), 3 deletions(-) ---

Re: crash in kmem_cache_init

2008-01-22 Thread Christoph Lameter
On Tue, 22 Jan 2008, Christoph Lameter wrote: > But I doubt that this is it. The fallback logic was added later and it > worked fine. My patch is useless (fascinating history of the changelog there through). fallback_alloc calls kmem_getpages without GFP_THISNODE. This means that alloc_pages_n

Re: crash in kmem_cache_init

2008-01-22 Thread Christoph Lameter
On Tue, 22 Jan 2008, Mel Gorman wrote: > Rather it should be 2. I'll admit the physical setup of this machine is > less than ideal but clearly it's something that can happen even if > it's a bad idea. Ok. Lets hope that Pekka's find does the trick. But this would mean that fallback gets mem

Re: crash in kmem_cache_init

2008-01-22 Thread Christoph Lameter
On Wed, 23 Jan 2008, Pekka Enberg wrote: > When we call fallback_alloc() because the current node has ->nodelists set to > NULL, we end up calling kmem_getpages() with -1 as the node id which is then > translated to numa_node_id() by alloc_pages_node. But the reason we called > fallback_alloc() in

Re: crash in kmem_cache_init

2008-01-22 Thread Mel Gorman
On (22/01/08 14:57), Christoph Lameter didst pronounce: > On Tue, 22 Jan 2008, Mel Gorman wrote: > > > > > Whatever this was a problem fixed in the past or not, it's broken again > > > > now > > > > :( . It's possible that there is a __GFP_THISNODE that can be dropped > > > > early > > > > at bo

Re: crash in kmem_cache_init

2008-01-22 Thread Pekka Enberg
+0xe0/0x14c [c05c3d50] c00d .kmem_cache_create+0x230/0x4cc [c05c3e30] c04c05f4 .kmem_cache_init+0x310/0x640 [c05c3ee0] c049f8d8 .start_kernel+0x304/0x3fc [c05c3f90] c0008594 .start_here_common+0x54/0xc0 0:mon> I mentioned t

Re: crash in kmem_cache_init

2008-01-22 Thread Christoph Lameter
On Tue, 22 Jan 2008, Mel Gorman wrote: > > > Whatever this was a problem fixed in the past or not, it's broken again > > > now > > > :( . It's possible that there is a __GFP_THISNODE that can be dropped > > > early > > > at boot-time that would also fix this problem in a way that doesn't > > > a

Re: crash in kmem_cache_init

2008-01-22 Thread Mel Gorman
3cf1] shift[16] cpu[0] Console: colour dummy device 80x25 console handover: boot [udbg0] -> real [hvc0] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes) Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes) freeing bootmem node 2 Memory: 4105560k/4194304k avail

Re: crash in kmem_cache_init

2008-01-22 Thread Christoph Lameter
On Tue, 22 Jan 2008, Olaf Hering wrote: > It crashes now in a different way if the patch below is applied: Yup no l3 structure for the current node. We are early in boostrap. You could just check if the l3 is there and if not just skip starting the reaper? This will be redone later anyways. Not

Re: crash in kmem_cache_init

2008-01-22 Thread Nish Aravamudan
setup_cpu_cache+0x114/0x1f4 > sp: c075bd40 >msr: 80009032 >dar: 58 > dsisr: 4200 > current = 0xc0665a50 > paca= 0xc0666380 > pid = 0, comm = swapper > enter ? for help > [c075bd40] c00fb368 .kmem

Re: crash in kmem_cache_init

2008-01-22 Thread Olaf Hering
sisr: 4200 current = 0xc0665a50 paca= 0xc0666380 pid = 0, comm = swapper enter ? for help [c075bd40] c00fb368 .kmem_cache_create+0x3c0/0x478 (unreliable) [c075be20] c05e6780 .kmem_cache_init+0x284/0x4f4 [c075bee0] c05b

Re: crash in kmem_cache_init

2008-01-22 Thread Christoph Lameter
On Tue, 22 Jan 2008, Mel Gorman wrote: > > After you reverted the slab memoryless node patch there should be per node > > structures created for node 0 unless the node is marked offline. Is it? If > > so then you are booting a cpu that is associated with an offline node. > > > > I'll roll a p

Re: crash in kmem_cache_init

2008-01-22 Thread Mel Gorman
On (22/01/08 12:11), Christoph Lameter didst pronounce: > On Tue, 22 Jan 2008, Mel Gorman wrote: > > > Christoph/Pekka, this patch is papering over the problem and something > > more fundamental may be going wrong. The crash occurs because l3 is NULL > > and the cache is kmem_cache so this is earl

Re: crash in kmem_cache_init

2008-01-22 Thread Christoph Lameter
On Tue, 22 Jan 2008, Mel Gorman wrote: > Christoph/Pekka, this patch is papering over the problem and something > more fundamental may be going wrong. The crash occurs because l3 is NULL > and the cache is kmem_cache so this is early in the boot process. It is > selecting l3 based on node 2 which

Re: crash in kmem_cache_init

2008-01-22 Thread Mel Gorman
believed. On two test machines here, this kmem_cache_init problem still happens even with the revert which fixed a third machine. I was delayed in testing because these boxen unavailable from Friday until yesterday evening (a stellar display of timing). It was missed on TKO because it was SLAB-specific an

Re: crash in kmem_cache_init

2008-01-18 Thread Christoph Lameter
On Thu, 17 Jan 2008, Olaf Hering wrote: > On Thu, Jan 17, Olaf Hering wrote: > > > Since -mm boots further, what patch should I try? > > rc8-mm1 crashes as well, l3 passed to reap_alien() is NULL. Sigh. It looks like we need alien cache structures in some cases for nodes that have no memory. W

Re: crash in kmem_cache_init

2008-01-18 Thread Christoph Lameter
ng bootstrap. You need to run kmem_cache_init() on a cpu on a processor with memory. Or we need to revert the patch which would allocate control structures again for all online nodes regardless if they have memory or not. Does reverting 04231b3002ac53f8a64a7bd142fde3fa4b6808c6 change the situation?

Re: crash in kmem_cache_init

2008-01-18 Thread Olaf Hering
On Fri, Jan 18, Christoph Lameter wrote: > Could you try this patch? Does not help, same crash. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read t

Re: crash in kmem_cache_init

2008-01-18 Thread Christoph Lameter
On Fri, 18 Jan 2008, Christoph Lameter wrote: > Memoryless nodes: Set N_NORMAL_MEMORY for a node if we do not support HIGHMEM If !CONFIG_HIGHMEM then enum node_states { #ifdef CONFIG_HIGHMEM N_HIGH_MEMORY, /* The node has regular or high memory */ #else N_HIGH_MEMORY = N

Re: crash in kmem_cache_init

2008-01-18 Thread Nish Aravamudan
On 1/18/08, Christoph Lameter <[EMAIL PROTECTED]> wrote: > Could you try this patch? > > Memoryless nodes: Set N_NORMAL_MEMORY for a node if we do not support > HIGHMEM > > It seems that we only scan through zones to set N_NORMAL_MEMORY only if > CONFIG_HIGHMEM and CONFIG_NUMA are set. We need to s

Re: crash in kmem_cache_init

2008-01-18 Thread Christoph Lameter
Could you try this patch? Memoryless nodes: Set N_NORMAL_MEMORY for a node if we do not support HIGHMEM It seems that we only scan through zones to set N_NORMAL_MEMORY only if CONFIG_HIGHMEM and CONFIG_NUMA are set. We need to set N_NORMAL_MEMORY in the !CONFIG_HIGHMEM case. Signed-off-by: Chris

Re: crash in kmem_cache_init

2008-01-18 Thread Christoph Lameter
EMORY. But... it only does this on CONFIG_HIGHMEM which on > PPC64 is not going to be set so N_NORMAL_MEMORY never gets set on > POWER That sounds bad. Argh. We may need to do a node_set_state(zone_to_nid(zone), N_NORMAL_MEMORY) in the !HIGHMEM case. > and one of them is in kmem_cache

Re: crash in kmem_cache_init

2008-01-18 Thread Mel Gorman
-n N_NORMAL_MEMORY slab.c 1593: for_each_node_state(nid, N_NORMAL_MEMORY) { 1971: for_each_node_state(node, N_NORMAL_MEMORY) { 2102: for_each_node_state(node, N_NORMAL_MEMORY) { 3818: for_each_node_state(node, N_NORMAL_MEMORY) { and one of them is in kmem_cac

Re: crash in kmem_cache_init

2008-01-18 Thread Christoph Lameter
On Thu, 17 Jan 2008, Olaf Hering wrote: > Normal 892928 -> 892928 > Movable zone start PFN for each node > early_node_map[1] active PFN ranges > 1:0 -> 892928 > Could not find start_pfn for node 0 We only have a single node that is node 1? And then we initialize nodes 0 to

Re: crash in kmem_cache_init

2008-01-18 Thread Christoph Lameter
On Thu, 17 Jan 2008, Olaf Hering wrote: > early_node_map[1] active PFN ranges > 1:0 -> 892928 > Could not find start_pfn for node 0 Corrupted min_pfn? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More major

Re: crash in kmem_cache_init

2008-01-18 Thread Christoph Lameter
On Fri, 18 Jan 2008, Olaf Hering wrote: > calls cache_grow with nodeid 0 > > [c075bbd0] [c00f82d0] .cache_alloc_refill+0x234/0x2c0 > calls cache_grow with nodeid 0 > > [c075bbe0] [c00f7f38] .cache_alloc_node+0x17c/0x1e8 > > calls cache_grow with nodeid 1 > > [c

Re: crash in kmem_cache_init

2008-01-17 Thread Olaf Hering
On Thu, Jan 17, Olaf Hering wrote: > On Thu, Jan 17, Christoph Lameter wrote: > > > On Thu, 17 Jan 2008, Olaf Hering wrote: > > > > > The patch does not help. > > > > Duh. We need to know more about the problem. > > cache_grow is called from 3 places. The third call has cleared l3 for > some r

Re: crash in kmem_cache_init

2008-01-17 Thread Olaf Hering
grow+0xb4/0x39c (unreliable) [c075bbd0] [c00f82d0] .cache_alloc_refill+0x234/0x2c0 [c075bc90] [c00f842c] .kmem_cache_alloc+0xd0/0x294 [c075bd40] [c00fb4e8] .kmem_cache_create+0x208/0x478 [c0000075be20] [c05e670c] .kmem_cache_init+0x

Re: crash in kmem_cache_init

2008-01-17 Thread Olaf Hering
On Thu, Jan 17, Olaf Hering wrote: > Since -mm boots further, what patch should I try? rc8-mm1 crashes as well, l3 passed to reap_alien() is NULL. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http:

Re: crash in kmem_cache_init

2008-01-17 Thread Olaf Hering
On Thu, Jan 17, Christoph Lameter wrote: > > freeing bootmem node 1 > > Memory: 3496632k/3571712k available (6188k kernel code, 75080k reserved, > > 1324k data, 1220k bss, 304k init) > > cache_grow(2781) swapper(0):c0,j4294937299 cp c06a4fb8 !l3 > > Is there more backtrace information? W

Re: crash in kmem_cache_init

2008-01-17 Thread Christoph Lameter
Could you try Pekka's suggestion of reverting 04231b3002ac53f8a64a7bd142fde3fa4b6808c6 ? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ

Re: crash in kmem_cache_init

2008-01-17 Thread Christoph Lameter
On Thu, 17 Jan 2008, Olaf Hering wrote: > The patch does not help. Duh. We need to know more about the problem. > > --- linux-2.6.orig/mm/slab.c2008-01-03 12:26:42.0 -0800 > > +++ linux-2.6/mm/slab.c 2008-01-09 15:59:49.0 -0800 > > @@ -2977,7 +2977,10 @@ retry: > >

Re: crash in kmem_cache_init

2008-01-17 Thread Olaf Hering
On Thu, Jan 17, Christoph Lameter wrote: > On Thu, 17 Jan 2008, Pekka Enberg wrote: > > > Looks similar to the one discussed on linux-mm ("[BUG] at > > mm/slab.c:3320" thread). Christoph? > > Right. Try the latest version of the patch to fix it: The patch does not help. > Index: linux-2.6/mm/

Re: crash in kmem_cache_init

2008-01-17 Thread Christoph Lameter
On Thu, 17 Jan 2008, Pekka Enberg wrote: > Looks similar to the one discussed on linux-mm ("[BUG] at > mm/slab.c:3320" thread). Christoph? Right. Try the latest version of the patch to fix it: Index: linux-2.6/mm/slab.c === --- linu

Re: crash in kmem_cache_init

2008-01-17 Thread Pekka Enberg
Hi Olaf, [Adding Christoph as cc.] On Jan 15, 2008 5:09 PM, Olaf Hering <[EMAIL PROTECTED]> wrote: > Current linus tree crashes in kmem_cache_init, as shown below. The > system is a 8cpu 2.2GHz POWER5 system, model 9117-570, with 4GB ram. > Firmware is 240_332, 2.6.23 boots o

Re: crash in kmem_cache_init

2008-01-15 Thread Olaf Hering
On Tue, Jan 15, Olaf Hering wrote: > > Current linus tree crashes in kmem_cache_init, as shown below. The > system is a 8cpu 2.2GHz POWER5 system, model 9117-570, with 4GB ram. > Firmware is 240_332, 2.6.23 boots ok with the same config. > > There is a series of mm related pat

crash in kmem_cache_init

2008-01-15 Thread Olaf Hering
Current linus tree crashes in kmem_cache_init, as shown below. The system is a 8cpu 2.2GHz POWER5 system, model 9117-570, with 4GB ram. Firmware is 240_332, 2.6.23 boots ok with the same config. There is a series of mm related patches in 2.6.24-rc1: commit

Re: kmem_cache_init failure (was Re: 2.6.21-mm1)

2007-05-09 Thread Frederik Deweerdt
On Wed, May 09, 2007 at 02:26:08PM +0200, Frederik Deweerdt wrote: > On Wed, May 09, 2007 at 11:00:46AM +0200, Andi Kleen wrote: > > On Wed, May 09, 2007 at 09:40:24AM +0200, Jan Beulich wrote: > > > >I've bisected it down to the x86_64-mm-cpa-kerneltext.patch and the > > > > > > > >+ if (!p

Re: kmem_cache_init failure (was Re: 2.6.21-mm1)

2007-05-09 Thread Frederik Deweerdt
On Wed, May 09, 2007 at 11:00:46AM +0200, Andi Kleen wrote: > On Wed, May 09, 2007 at 09:40:24AM +0200, Jan Beulich wrote: > > >I've bisected it down to the x86_64-mm-cpa-kerneltext.patch and the > > > > > >+ if (!pte_present(*kpte)) > > >+ return 0; > > > > I the most recent

Re: kmem_cache_init failure (was Re: 2.6.21-mm1)

2007-05-09 Thread Andi Kleen
On Wed, May 09, 2007 at 09:40:24AM +0200, Jan Beulich wrote: > >I've bisected it down to the x86_64-mm-cpa-kerneltext.patch and the > > > >+ if (!pte_present(*kpte)) > >+ return 0; > > I the most recent version of the patch I sent to Andi this line is gone > (again), > as I r

kmem_cache_init failure (was Re: 2.6.21-mm1)

2007-05-09 Thread Jan Beulich
>I've bisected it down to the x86_64-mm-cpa-kerneltext.patch and the > >+ if (!pte_present(*kpte)) >+ return 0; I the most recent version of the patch I sent to Andi this line is gone (again), as I realized it was wrong on i386 (namely for DEBUG_PAGEALLOC) and its respective v

Re: kmem_cache_init failure (was Re: 2.6.21-mm1)

2007-05-08 Thread Frederik Deweerdt
On Wed, May 09, 2007 at 12:12:29AM +0200, Andi Kleen wrote: > On Tue, May 08, 2007 at 07:22:33PM +0200, Frederik Deweerdt wrote: > > On Sat, May 05, 2007 at 01:49:55AM -0700, Andrew Morton wrote: > > > > > > ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.21/2.6.21-mm1/ > > > >

Re: kmem_cache_init failure (was Re: 2.6.21-mm1)

2007-05-08 Thread Andi Kleen
On Tue, May 08, 2007 at 07:22:33PM +0200, Frederik Deweerdt wrote: > On Sat, May 05, 2007 at 01:49:55AM -0700, Andrew Morton wrote: > > > > ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.21/2.6.21-mm1/ > > > Hi all, > > My computer fails early at boot with a stack along the li

Re: kmem_cache_init failure (was Re: 2.6.21-mm1)

2007-05-08 Thread Andrew Morton
computer fails early at boot with a stack along the lines of: > > kmem_cache_zalloc > kmem_cache_create > kmem_cache_init > start_kernel > > eip is at cache_calloc_refill+0x3e1 which is the > slabp->colouroff = colouroff; in alloc_slabmgmt() > > I've bisec

kmem_cache_init failure (was Re: 2.6.21-mm1)

2007-05-08 Thread Frederik Deweerdt
On Sat, May 05, 2007 at 01:49:55AM -0700, Andrew Morton wrote: > > ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.21/2.6.21-mm1/ > Hi all, My computer fails early at boot with a stack along the lines of: kmem_cache_zalloc kmem_cache_create kmem_cache_init start_ke

Re: [PATCH] Catch kmalloc() failure in kmem_cache_init()

2007-04-02 Thread Pekka J Enberg
On Mon, 2 Apr 2007, Johannes Weiner wrote: > This patch makes kmem_cache_init() trigger a BUG when the calls to > kmalloc() fail. Altough the kernel would not boot anyway on failure, > this adds some clarity to the code. Indeed. This keeps popping up again and again, so Acked-by: Pek

[PATCH] Catch kmalloc() failure in kmem_cache_init()

2007-04-02 Thread Johannes Weiner
Hi, This patch makes kmem_cache_init() trigger a BUG when the calls to kmalloc() fail. Altough the kernel would not boot anyway on failure, this adds some clarity to the code. Signed-off-by: Johannes Weiner <[EMAIL PROTECTED]> diff --git a/mm/slab.c b/mm/slab.c index 57f7aa4..6d7e486

[PATCH] Catch kmalloc failure in kmem_cache_init() (was: [QUESTION] check for mem in slab)

2007-04-02 Thread Johannes Weiner
Hi, > On 3/30/07, Heiko Carstens <[EMAIL PROTECTED]> wrote: > >> in file mm/slab.c and routine kmem_cache_init() I found there > >> is no checking for allocated memory on line: > >> > >> /* 4) Replace the bootstrap head arrays */ > >&

kmem_cache_init ()

2001-05-13 Thread alad
Hi, I was unable to understand the logic of the following sanity checks in slab.c/kmem_cache_init (). if (kmem_cache_diff(c_firstp,c_magic) != kmem_slab_diff(s_nextp,s_magic) || kmem_cache_diff(c_firstp,c_inuse) != kmem_slab_diff(s_nextp,s_inuse) || kmem_cache_offset(c_lastp) - ((unsigned