Re: [PATCH v5 01/14] memory-hotplug: try to offline the memory twice to avoid dependence

2013-02-06 Thread Tang Chen
On 02/06/2013 10:24 PM, Glauber Costa wrote: And one more question, a memory section is 128MB in Linux. If we reserve part of the them for page_cgroup, then anyone who wants to allocate a contiguous memory larger than 128MB, it will fail, right ? Is it OK ? No, it is not. Another take on this:

Re: [PATCH v5 01/14] memory-hotplug: try to offline the memory twice to avoid dependence

2013-02-06 Thread Tang Chen
On 02/06/2013 05:17 PM, Tang Chen wrote: Hi all, On 02/06/2013 11:07 AM, Tang Chen wrote: Hi Glauber, all, An old thing I want to discuss with you. :) On 01/09/2013 11:09 PM, Glauber Costa wrote: memory can't be offlined when CONFIG_MEMCG is selected. For example: there is a memory d

Re: [PATCH v5 01/14] memory-hotplug: try to offline the memory twice to avoid dependence

2013-02-06 Thread Tang Chen
Hi all, On 02/06/2013 11:07 AM, Tang Chen wrote: Hi Glauber, all, An old thing I want to discuss with you. :) On 01/09/2013 11:09 PM, Glauber Costa wrote: memory can't be offlined when CONFIG_MEMCG is selected. For example: there is a memory device on node 1. The address range is [1G,

Re: [PATCH v5 01/14] memory-hotplug: try to offline the memory twice to avoid dependence

2013-02-05 Thread Tang Chen
Hi Glauber, all, An old thing I want to discuss with you. :) On 01/09/2013 11:09 PM, Glauber Costa wrote: memory can't be offlined when CONFIG_MEMCG is selected. For example: there is a memory device on node 1. The address range is [1G, 1.5G). You will find 4 new directories memory8, memory9, m

Re: [PATCH v6 00/15] memory-hotplug: hot-remove physical memory

2013-01-31 Thread Tang Chen
Hi Simon, On 02/01/2013 11:06 AM, Simon Jeons wrote: How can distinguish map and use? I mean how can confirm memory is used by kernel instead of map? If the page is free, for example, it is in the buddy system, it is not in use. Even if it is direct mapped by kernel, the kernel logic should

Re: [PATCH v6 00/15] memory-hotplug: hot-remove physical memory

2013-01-31 Thread Tang Chen
Hi Simon, On 02/01/2013 10:17 AM, Simon Jeons wrote: For example: 64TB, what ever xxxTB, what ever logic address space: |_kernel___|_user_| \ \ / /

Re: [PATCH v6 00/15] memory-hotplug: hot-remove physical memory

2013-01-31 Thread Tang Chen
On 02/01/2013 09:36 AM, Simon Jeons wrote: On Fri, 2013-02-01 at 09:32 +0800, Jianguo Wu wrote: So if config NUMA, kernel memory will not be linear mapping anymore? For example, Node 0 Node 1 0 ~ 10G 11G~14G It has nothing to do with linear mapping, I think. kernel memory only at Node 0

Re: [PATCH v6 00/15] memory-hotplug: hot-remove physical memory

2013-01-31 Thread Tang Chen
Hi Simon, On 01/31/2013 04:48 PM, Simon Jeons wrote: Hi Tang, On Thu, 2013-01-31 at 15:10 +0800, Tang Chen wrote: 1. IIUC, there is a button on machine which supports hot-remove memory, then what's the difference between press button and echo to /sys? No important difference, I think.

Re: [PATCH v6 00/15] memory-hotplug: hot-remove physical memory

2013-01-30 Thread Tang Chen
On 01/31/2013 02:19 PM, Simon Jeons wrote: Hi Tang, On Thu, 2013-01-31 at 11:31 +0800, Tang Chen wrote: Hi Simon, Please see below. :) On 01/31/2013 09:22 AM, Simon Jeons wrote: Sorry, I still confuse. :( update node_states[N_NORMAL_MEMORY] to node_states[N_MEMORY] or node_states

Re: [PATCH v6 00/15] memory-hotplug: hot-remove physical memory

2013-01-30 Thread Tang Chen
Hi Simon, Please see below. :) On 01/31/2013 09:22 AM, Simon Jeons wrote: Sorry, I still confuse. :( update node_states[N_NORMAL_MEMORY] to node_states[N_MEMORY] or node_states[N_NORMAL_MEMOR] present 0...ZONE_MOVABLE? node_states is what? node_states[N_NORMAL_MEMOR] or node_states[N_MEMORY]?

Re: [PATCH v6 00/15] memory-hotplug: hot-remove physical memory

2013-01-30 Thread Tang Chen
On 01/30/2013 06:15 PM, Tang Chen wrote: Hi Simon, Please see below. :) On 01/29/2013 08:52 PM, Simon Jeons wrote: Hi Tang, On Wed, 2013-01-09 at 17:32 +0800, Tang Chen wrote: Here is the physical memory hot-remove patch-set based on 3.8rc-2. Some questions ask you, not has relationship

Re: [PATCH v6 00/15] memory-hotplug: hot-remove physical memory

2013-01-30 Thread Tang Chen
Hi Simon, Please see below. :) On 01/29/2013 08:52 PM, Simon Jeons wrote: Hi Tang, On Wed, 2013-01-09 at 17:32 +0800, Tang Chen wrote: Here is the physical memory hot-remove patch-set based on 3.8rc-2. Some questions ask you, not has relationship with this patchset, but is memory hotplug

Re: [PATCH v6 08/15] memory-hotplug: Common APIs to support page tables hot-remove

2013-01-29 Thread Tang Chen
On 01/30/2013 11:27 AM, Simon Jeons wrote: On Wed, 2013-01-30 at 10:16 +0800, Tang Chen wrote: On 01/29/2013 09:04 PM, Simon Jeons wrote: Hi Tang, On Wed, 2013-01-09 at 17:32 +0800, Tang Chen wrote: From: Wen Congyang When memory is removed, the corresponding pagetables should alse be

Re: [PATCH v6 00/15] memory-hotplug: hot-remove physical memory

2013-01-29 Thread Tang Chen
On 01/30/2013 10:48 AM, Simon Jeons wrote: On Wed, 2013-01-30 at 10:32 +0800, Tang Chen wrote: On 01/29/2013 08:52 PM, Simon Jeons wrote: Hi Tang, On Wed, 2013-01-09 at 17:32 +0800, Tang Chen wrote: Here is the physical memory hot-remove patch-set based on 3.8rc-2. Hi Simon, I&#x

Re: [PATCH v6 00/15] memory-hotplug: hot-remove physical memory

2013-01-29 Thread Tang Chen
On 01/29/2013 08:52 PM, Simon Jeons wrote: Hi Tang, On Wed, 2013-01-09 at 17:32 +0800, Tang Chen wrote: Here is the physical memory hot-remove patch-set based on 3.8rc-2. Hi Simon, I'll summarize all the info and answer you later. :) Thanks for asking. :) Some questions ask you, no

Re: [PATCH v6 08/15] memory-hotplug: Common APIs to support page tables hot-remove

2013-01-29 Thread Tang Chen
On 01/29/2013 09:04 PM, Simon Jeons wrote: Hi Tang, On Wed, 2013-01-09 at 17:32 +0800, Tang Chen wrote: From: Wen Congyang When memory is removed, the corresponding pagetables should alse be removed. This patch introduces some common APIs to support vmemmap pagetable and x86_64 architecture

Re: [PATCH Bug fix 0/5] Bug fix for physical memory hot-remove.

2013-01-27 Thread Tang Chen
On 01/25/2013 09:17 PM, Michal Hocko wrote: On Wed 23-01-13 06:29:31, Simon Jeons wrote: On Tue, 2013-01-22 at 19:42 +0800, Tang Chen wrote: Here are some bug fix patches for physical memory hot-remove. All these patches are based on the latest -mm tree. git://git.kernel.org/pub/scm/linux

Re: [PATCH Bug fix 0/5] Bug fix for physical memory hot-remove.

2013-01-27 Thread Tang Chen
On 01/26/2013 02:19 AM, Toshi Kani wrote: On Tue, 2013-01-22 at 19:42 +0800, Tang Chen wrote: Here are some bug fix patches for physical memory hot-remove. All these patches are based on the latest -mm tree. git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm And patch1 and

[PATCH 2/2] cpu-hotplug, memory-hotplug: Remove __cpuinit declaration of numa_clear_node().

2013-01-24 Thread Tang Chen
numa_clear_node() will be used by check_and_unmap_cpu_on_node() when we do node hotplug. So it is no longer a __cpuinit function. Do not declare it as a __cpuinit function, otherwise it will cause section mismatch warning when compiling. Signed-off-by: Tang Chen --- arch/x86/mm/numa.c |2

[PATCH 0/2] Fix section mismatch caused by node offline code.

2013-01-24 Thread Tang Chen
Since node offline code uses __apicid_to_node[] and numa_clear_node(), they are no longer init data or function. So do not declare them as __cpuinitdata or __cpuinit. Tang Chen (2): cpu_hotplug: Remove __cpuinitdata declaration of __apicid_to_node[]. cpu-hotplug,memory-hotplug: Remove

[PATCH 1/2] cpu_hotplug: Remove __cpuinitdata declaration of __apicid_to_node[].

2013-01-24 Thread Tang Chen
__apicid_to_node[] will be used by acpi_unmap_lsapic() when we do node hotplug. So it is no longer an init data. Do not declare __apicid_to_node[] as a __cpuinitdata, otherwise it will cause section mismatch warning when compiling. Signed-off-by: Tang Chen --- arch/x86/mm/numa.c |2 +- 1

Re: [PATCH Bug fix 0/5] Bug fix for physical memory hot-remove.

2013-01-23 Thread Tang Chen
On 01/24/2013 08:35 AM, Simon Jeons wrote: On Wed, 2013-01-23 at 21:17 +0800, Tang Chen wrote: On 01/23/2013 08:29 PM, Simon Jeons wrote: Hi Tang, I remember your big physical memory hot-remove patchset has already merged by Andrew, but where I can find it? Could you give me git tree address

Re: [PATCH Bug fix 0/5] Bug fix for physical memory hot-remove.

2013-01-23 Thread Tang Chen
On 01/23/2013 08:29 PM, Simon Jeons wrote: Hi Tang, I remember your big physical memory hot-remove patchset has already merged by Andrew, but where I can find it? Could you give me git tree address? Hi Simon, You can find all the physical memory hot-remove patches and related bugfix patches f

[PATCH Bug fix 4/4] Rename movablecore_map to movablemem_map.

2013-01-22 Thread Tang Chen
Since "core" could be confused with cpu cores, but here it is memory, so rename the boot option movablecore_map to movablemem_map. Signed-off-by: Tang Chen --- Documentation/kernel-parameters.txt |8 ++-- include/linux/memblock.h|2 +- include/

[PATCH Bug fix 2/4] Bug fix: Fix the doc format.

2013-01-22 Thread Tang Chen
Signed-off-by: Tang Chen --- mm/page_alloc.c | 23 ++- 1 files changed, 14 insertions(+), 9 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 00037a3..cd6f8a6 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -4372,7 +4372,7 @@ static unsigned long

[PATCH Bug fix 3/4] Bug fix: Remove the unused sanitize_zone_movable_limit() definition.

2013-01-22 Thread Tang Chen
When CONFIG_HAVE_MEMBLOCK_NODE_MAP is not defined, sanitize_zone_movable_limit() is also not used. So remove it. Signed-off-by: Tang Chen --- mm/page_alloc.c |5 - 1 files changed, 0 insertions(+), 5 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index cd6f8a6..2bd529e

[PATCH Bug fix 1/4] Bug fix: Use CONFIG_HAVE_MEMBLOCK_NODE_MAP to protect movablecore_map in memblock_overlaps_region().

2013-01-22 Thread Tang Chen
The definition of struct movablecore_map is protected by CONFIG_HAVE_MEMBLOCK_NODE_MAP but its use in memblock_overlaps_region() is not. So add CONFIG_HAVE_MEMBLOCK_NODE_MAP to protect the use of movablecore_map in memblock_overlaps_region(). Reported-by: Stephen Rothwell Signed-off-by: Tang

[PATCH Bug fix 0/4] Bug fix for movablecore_map boot option.

2013-01-22 Thread Tang Chen
Hi Andrew, patch1 ~ patch3 fix some problems of movablecore_map boot option. And since the name "core" could be confused, patch4 rename this option to movablemem_map. All these patches are based on the latest -mm tree. git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git

[PATCH Bug fix 4/5] cpu-hotplug, memory-hotplug: clear cpu_to_node() when offlining the node

2013-01-22 Thread Tang Chen
: KOSAKI Motohiro Cc: Andrew Morton Cc: Mel Gorman Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Peter Zijlstra Cc: Tang Chen Signed-off-by: Wen Congyang Signed-off-by: Tang Chen --- mm/memory_hotplug.c | 30 +- 1 files changed, 29 insert

[PATCH Bug fix 5/5] Do not use cpu_to_node() to find an offlined cpu's node.

2013-01-22 Thread Tang Chen
. Else, a online cpu on another node will be picked. Cc: Yasuaki Ishimatsu Cc: David Rientjes Cc: Jiang Liu Cc: Minchan Kim Cc: KOSAKI Motohiro Cc: Andrew Morton Cc: Mel Gorman Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Peter Zijlstra Signed-off-by: Tang Chen Sig

[PATCH Bug fix 3/5] cpu-hotplug, memory-hotplug: try offline the node when hotremoving a cpu

2013-01-22 Thread Tang Chen
: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Peter Zijlstra Cc: Tang Chen Signed-off-by: Wen Congyang Signed-off-by: Tang Chen --- drivers/acpi/processor_driver.c |2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/drivers/acpi/processor_driver.c

[PATCH Bug fix 1/5] cpu_hotplug: clear apicid to node when the cpu is hotremoved

2013-01-22 Thread Tang Chen
d. This patch only clears apicid-to-node mapping when the cpu is hotremoved. Cc: Yasuaki Ishimatsu Cc: David Rientjes Cc: Jiang Liu Cc: Minchan Kim Cc: KOSAKI Motohiro Cc: Andrew Morton Cc: Mel Gorman Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Peter Zijlstra Cc: Ta

[PATCH Bug fix 2/5] memory-hotplug: export the function try_offline_node()

2013-01-22 Thread Tang Chen
Cc: Minchan Kim Cc: KOSAKI Motohiro Cc: Andrew Morton Cc: Mel Gorman Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Peter Zijlstra Cc: Tang Chen Signed-off-by: Wen Congyang Signed-off-by: Tang Chen --- include/linux/memory_hotplug.h |2 ++ mm/memory_hotplug.c

[PATCH Bug fix 0/5] Bug fix for node offline

2013-01-22 Thread Tang Chen
akpm Tang Chen (1): Do not use cpu_to_node() to find an offlined cpu's node. Wen Congyang (4): cpu_hotplug: clear apicid to node when the cpu is hotremoved memory-hotplug: export the function try_offline_node() cpu-hotplug, memory-hotplug: try offline the node when hotremoving a

[PATCH Bug fix 5/5] Bug fix: Fix the doc format in drivers/firmware/memmap.c

2013-01-22 Thread Tang Chen
Make the comments in drivers/firmware/memmap.c kernel-doc compliant. Reported-by: Julian Calaby Signed-off-by: Tang Chen --- drivers/firmware/memmap.c | 12 ++-- 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/firmware/memmap.c b/drivers/firmware/memmap.c index

[PATCH Bug fix 4/5] Bug fix: Fix section mismatch problem of release_firmware_map_entry().

2013-01-22 Thread Tang Chen
The function release_firmware_map_entry() references the function __meminit firmware_map_find_entry_in_list(). So it should also have __meminit. And since the firmware_map_entry->kobj is initialized with memmap_ktype, the memmap_ktype should also be prefixed by __refdata. Signed-off-by: T

[PATCH Bug fix 3/5] Bug fix: Do not split pages when freeing pagetable pages.

2013-01-22 Thread Tang Chen
patch will fix the problem based on the above patches. Reported-by: Wen Congyang Signed-off-by: Tang Chen --- arch/x86/mm/init_64.c | 148 +++- 1 files changed, 46 insertions(+), 102 deletions(-) diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm

[PATCH Bug fix 2/5] Bug-fix: mempolicy: fix is_valid_nodemask()

2013-01-22 Thread Tang Chen
BLE should be valid zone for policies. is_valid_nodemask() need to be changed to match it. Fix: check all zones, even its zoneid > policy_zone. Use nodes_intersects() instead open code to check it. Reported-by: Wen Congyang Signed-off-by: Lai Jiangshan Signed-off-by: Tang Chen --- mm/mempolicy.c

[PATCH Bug fix 0/5] Bug fix for physical memory hot-remove.

2013-01-22 Thread Tang Chen
will panic the next time memory is hot-added. patch3: the old way of freeing pagetable pages was wrong. We should never split larger pages into small ones. Lai Jiangshan (1): Bug-fix: mempolicy: fix is_valid_nodemask() Tang Chen (3): Bug fix: Do not split pages when freeing

[PATCH Bug fix 1/5] Bug fix: consider compound pages when free memmap

2013-01-22 Thread Tang Chen
+0xef/0x166 [ 696.960128] RSP [ 697.001768] CR2: ea004000 [ 697.041336] ---[ end trace e7f94e3a34c442d4 ]--- [ 697.096474] Kernel panic - not syncing: Fatal exception Signed-off-by: Wen Congyang Signed-off-by: Tang Chen --- mm/sparse.c |2 +- 1 files changed, 1 insertions(+), 1 dele

Re: [BUG Fix Patch 1/6] Bug fix: Hold spinlock across find|remove /sys/firmware/memmap/X operation.

2013-01-15 Thread Tang Chen
On 01/16/2013 06:26 AM, Julian Calaby wrote: Hi Tang, One minor point. /* - * Search memmap entry + * firmware_map_find_entry: Search memmap entry. + * @start: Start of the memory range. + * @end: End of the memory range (exclusive). + * @type: Type of the memory range. + * + * This func

[BUG Fix Patch 6/6] Bug fix: Reuse the storage of /sys/firmware/memmap/X/ allocated by bootmem.

2013-01-15 Thread Tang Chen
link the map entries allocated by bootmem when they are removed, and a lock to protect it. And these entries will be reused when the memory is hot-added again. The idea is suggestted by Andrew Morton Signed-off-by: Tang Chen --- drivers/firmware/memmap.c |

[BUG Fix Patch 4/6] Bug fix: Do not free page split from hugepage one by one.

2013-01-15 Thread Tang Chen
Signed-off-by: Tang Chen Signed-off-by: Wen Congyang --- arch/x86/mm/init_64.c | 80 +++- 1 files changed, 71 insertions(+), 9 deletions(-) diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index e77d312..ff0206b 100644 --- a/arch/x86/mm/ini

[BUG Fix Patch 3/6] Bug fix: Do not free direct mapping pages twice.

2013-01-15 Thread Tang Chen
Direct mapped pages were freed when they were offlined, or they were not allocated. So we only need to free vmemmap pages, no need to free direct mapped pages. Signed-off-by: Tang Chen --- arch/x86/mm/init_64.c |9 +++-- 1 files changed, 7 insertions(+), 2 deletions(-) diff --git a

[BUG Fix Patch 5/6] Bug fix: Fix the wrong comments of map_entries.

2013-01-15 Thread Tang Chen
Now we have a map_entries_lock to protect map_entries list. So we need to update the comments. Signed-off-by: Tang Chen --- drivers/firmware/memmap.c |6 +- 1 files changed, 1 insertions(+), 5 deletions(-) diff --git a/drivers/firmware/memmap.c b/drivers/firmware/memmap.c index 940c4e9

[BUG Fix Patch 1/6] Bug fix: Hold spinlock across find|remove /sys/firmware/memmap/X operation.

2013-01-15 Thread Tang Chen
these two functions need to be careful to hold the lock when using these two functions. The suggestion is from Andrew Morton Signed-off-by: Tang Chen --- drivers/firmware/memmap.c | 25 + 1 files changed, 17 insertions(+), 8 deletions(-) diff --git a/drivers/firmware

[BUG Fix Patch 2/6] Bug fix: Do not calculate direct mapping pages when freeing vmemmap pagetables.

2013-01-15 Thread Tang Chen
We only need to update direct_pages_count[level] when we freeing direct mapped pagetables. Signed-off-by: Tang Chen --- arch/x86/mm/init_64.c | 17 +++-- 1 files changed, 7 insertions(+), 10 deletions(-) diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index e829113

[BUG Fix Patch 0/6] Bug fix for physical memory hot-remove.

2013-01-15 Thread Tang Chen
Hi Andrew, Here are some bug fix patches for physical hot-remove patches. And there are some new ones reported by others, I'll fix them soon. Thanks. :) Tang Chen (6): Bug fix: Hold spinlock across find|remove /sys/firmware/memmap/X operation. Bug fix: Do not calculate direct ma

Re: [PATCH v6 04/15] memory-hotplug: remove /sys/firmware/memmap/X sysfs

2013-01-09 Thread Tang Chen
Hi Andrew, On 01/10/2013 07:19 AM, Andrew Morton wrote: ... + entry = firmware_map_find_entry(start, end - 1, type); + if (!entry) + return -EINVAL; + + firmware_map_remove_entry(entry); ... The above code looks racy. After firmware_map_find_entry() does the

Re: [PATCH v6 02/15] memory-hotplug: check whether all memory blocks are offlined or not when removing memory

2013-01-09 Thread Tang Chen
Hi Andrew, On 01/10/2013 07:11 AM, Andrew Morton wrote: On Wed, 9 Jan 2013 17:32:26 +0800 Tang Chen wrote: We remove the memory like this: 1. lock memory hotplug 2. offline a memory block 3. unlock memory hotplug 4. repeat 1-3 to offline all memory blocks 5. lock memory hotplug 6. remove

Re: [PATCH v6 04/15] memory-hotplug: remove /sys/firmware/memmap/X sysfs

2013-01-09 Thread Tang Chen
Hi Andrew, On 01/10/2013 06:49 AM, Andrew Morton wrote: On Wed, 9 Jan 2013 17:32:28 +0800 Tang Chen wrote: When (hot)adding memory into system, /sys/firmware/memmap/X/{end, start, type} sysfs files are created. But there is no code to remove these files. The patch implements the function to

Re: [PATCH v6 05/15] memory-hotplug: introduce new function arch_remove_memory() for removing page table depends on architecture

2013-01-09 Thread Tang Chen
Hi Andrew, On 01/10/2013 06:50 AM, Andrew Morton wrote: On Wed, 9 Jan 2013 17:32:29 +0800 Tang Chen wrote: For removing memory, we need to remove page table. But it depends on architecture. So the patch introduce arch_remove_memory() for removing page table. Now it only calls __remove_pages

Re: [PATCH v6 00/15] memory-hotplug: hot-remove physical memory

2013-01-09 Thread Tang Chen
Hi Andrew, On 01/10/2013 07:33 AM, Andrew Morton wrote: On Wed, 9 Jan 2013 17:32:24 +0800 Tang Chen wrote: This patch-set aims to implement physical memory hot-removing. As you were on th patch delivery path, all of these patches should have your Signed-off-by:. But some were missing it

Re: [PATCH v6 00/15] memory-hotplug: hot-remove physical memory

2013-01-09 Thread Tang Chen
Hi Andrew, Thank you very much for your pushing. :) On 01/10/2013 06:23 AM, Andrew Morton wrote: This does sound like a significant problem. We should assume that mmecg is available and in use. In patch1, we provide a solution which is not good enough: Iterate twice to offline the memory. 1

Re: [PATCH v5 01/14] memory-hotplug: try to offline the memory twice to avoid dependence

2013-01-09 Thread Tang Chen
Hi Glauber, On 01/09/2013 11:09 PM, Glauber Costa wrote: We try to make all page_cgroup allocations local to the node they are describing now. If the memory is the first memory onlined in this node, we will allocate it from the other node. For example, node1 has 4 memory blocks: 8-11, and we o

[PATCH v6 15/15] memory-hotplug: Do not allocate pdgat if it was not freed when offline.

2013-01-09 Thread Tang Chen
when the node is online again. The problem is suggested by Kamezawa Hiroyuki The idea is from Wen Congyang NOTE: If we don't reset pgdat to 0, the WARN_ON in free_area_init_node() will be triggered. Signed-off-by: Tang Chen Reviewed-by: Wen Congyang --- mm/memory_hotplug.c |

[PATCH v6 11/15] memory-hotplug: Integrated __remove_section() of CONFIG_SPARSEMEM_VMEMMAP.

2013-01-09 Thread Tang Chen
Currently __remove_section for SPARSEMEM_VMEMMAP does nothing. But even if we use SPARSEMEM_VMEMMAP, we can unregister the memory_section. Signed-off-by: Yasuaki Ishimatsu Signed-off-by: Wen Congyang Signed-off-by: Tang Chen --- mm/memory_hotplug.c | 11 --- 1 files changed, 0

[PATCH v6 14/15] memory-hotplug: free node_data when a node is offlined

2013-01-09 Thread Tang Chen
From: Wen Congyang We call hotadd_new_pgdat() to allocate memory to store node_data. So we should free it when removing a node. Signed-off-by: Wen Congyang Reviewed-by: Kamezawa Hiroyuki --- mm/memory_hotplug.c | 30 +++--- 1 files changed, 27 insertions(+), 3 deleti

[PATCH v6 12/15] memory-hotplug: memory_hotplug: clear zone when removing the memory

2013-01-09 Thread Tang Chen
From: Yasuaki Ishimatsu When a memory is added, we update zone's and pgdat's start_pfn and spanned_pages in the function __add_zone(). So we should revert them when the memory is removed. The patch adds a new function __remove_zone() to do this. Signed-off-by: Yasuaki Ishimatsu Signed-off-by:

[PATCH v6 10/15] memory-hotplug: remove memmap of sparse-vmemmap

2013-01-09 Thread Tang Chen
sparc. Signed-off-by: Yasuaki Ishimatsu Signed-off-by: Jianguo Wu Signed-off-by: Wen Congyang Signed-off-by: Tang Chen --- arch/arm64/mm/mmu.c |3 +++ arch/ia64/mm/discontig.c |4 arch/powerpc/mm/init_64.c |4 arch/s390/mm/vmem.c |4 arch/sparc/mm

[PATCH v6 13/15] memory-hotplug: remove sysfs file of node

2013-01-09 Thread Tang Chen
This patch introduces a new function try_offline_node() to remove sysfs file of node when all memory sections of this node are removed. If some memory sections of this node are not removed, this function does nothing. Signed-off-by: Wen Congyang Signed-off-by: Tang Chen --- drivers/acpi

[PATCH v6 09/15] memory-hotplug: remove page table of x86_64 architecture

2013-01-09 Thread Tang Chen
This patch searches a page table about the removed memory, and clear page table for x86_64 architecture. Signed-off-by: Wen Congyang Signed-off-by: Jianguo Wu Signed-off-by: Jiang Liu Signed-off-by: Tang Chen --- arch/x86/mm/init_64.c | 10 ++ 1 files changed, 10 insertions(+), 0

[PATCH v6 02/15] memory-hotplug: check whether all memory blocks are offlined or not when removing memory

2013-01-09 Thread Tang Chen
memory. But we don't hold the lock in the whole operation. So we should check whether all memory blocks are offlined before step6. Otherwise, kernel maybe panicked. Signed-off-by: Wen Congyang Signed-off-by: Yasuaki Ishimatsu Signed-off-by: Tang Chen Acked-by: KAMEZAWA Hiroyuki --- dr

[PATCH v6 06/15] memory-hotplug: implement register_page_bootmem_info_section of sparse-vmemmap

2013-01-09 Thread Tang Chen
From: Yasuaki Ishimatsu For removing memmap region of sparse-vmemmap which is allocated bootmem, memmap region of sparse-vmemmap needs to be registered by get_page_bootmem(). So the patch searches pages of virtual mapping and registers the pages by get_page_bootmem(). Note: register_page_bootmem

[PATCH v6 08/15] memory-hotplug: Common APIs to support page tables hot-remove

2013-01-09 Thread Tang Chen
cleared. In this case, the page used as PT/PMD can be freed. Signed-off-by: Yasuaki Ishimatsu Signed-off-by: Jianguo Wu Signed-off-by: Wen Congyang Signed-off-by: Tang Chen --- arch/x86/include/asm/pgtable_types.h |1 + arch/x86/mm/init_64.c| 299

[PATCH v6 05/15] memory-hotplug: introduce new function arch_remove_memory() for removing page table depends on architecture

2013-01-09 Thread Tang Chen
From: Wen Congyang For removing memory, we need to remove page table. But it depends on architecture. So the patch introduce arch_remove_memory() for removing page table. Now it only calls __remove_pages(). Note: __remove_pages() for some archtecuture is not implemented (I don't know how t

[PATCH v6 00/15] memory-hotplug: hot-remove physical memory

2013-01-09 Thread Tang Chen
. Patch3: new patch, no logical change, just remove reduntant codes. Patch9: merge the patch from wujianguo into this patch. flush tlb on all cpu after the pagetable is changed. Patch12: new patch, free node_data when a node is offlined. Tang Chen (6): memory-hotplug: mo

[PATCH v6 03/15] memory-hotplug: remove redundant codes

2013-01-09 Thread Tang Chen
From: Wen Congyang offlining memory blocks and checking whether memory blocks are offlined are very similar. This patch introduces a new function to remove redundant codes. Signed-off-by: Wen Congyang Signed-off-by: Tang Chen Reviewed-by: Kamezawa Hiroyuki --- mm/memory_hotplug.c | 129

[PATCH v6 01/15] memory-hotplug: try to offline the memory twice to avoid dependence

2013-01-09 Thread Tang Chen
From: Wen Congyang memory can't be offlined when CONFIG_MEMCG is selected. For example: there is a memory device on node 1. The address range is [1G, 1.5G). You will find 4 new directories memory8, memory9, memory10, and memory11 under the directory /sys/devices/system/memory/. If CONFIG_MEMCG i

[PATCH v6 04/15] memory-hotplug: remove /sys/firmware/memmap/X sysfs

2013-01-09 Thread Tang Chen
bootmem. So the patch makes memory leak. But I think the memory leak size is very samll. And it does not affect the system. Signed-off-by: Wen Congyang Signed-off-by: Yasuaki Ishimatsu Signed-off-by: Tang Chen Reviewed-by: Kamezawa Hiroyuki --- drivers/firmware/memmap.c| 96

[PATCH v6 07/15] memory-hotplug: move pgdat_resize_lock into sparse_remove_one_section()

2013-01-09 Thread Tang Chen
0x190 [ 457.764552] [] ? __init_kthread_worker+0x70/0x70 [ 457.839427] [] ret_from_fork+0x7c/0xb0 [ 457.903914] [] ? __init_kthread_worker+0x70/0x70 [ 457.978784] ---[ end trace 25e85300f542aa01 ]--- Signed-off-by: Tang Chen Signed-off-by: Lai Jiangshan Signed-off-by: Wen Congyang Ack

Re: [PATCH v5 02/14] memory-hotplug: check whether all memory blocks are offlined or not when removing memory

2012-12-26 Thread Tang Chen
On 12/26/2012 11:10 AM, Kamezawa Hiroyuki wrote: > (2012/12/24 21:09), Tang Chen wrote: >> From: Yasuaki Ishimatsu >> >> We remove the memory like this: >> 1. lock memory hotplug >> 2. offline a memory block >> 3. unlock memory hotplug >> 4. repea

Re: [PATCH v5 04/14] memory-hotplug: remove /sys/firmware/memmap/X sysfs

2012-12-26 Thread Tang Chen
On 12/26/2012 11:30 AM, Kamezawa Hiroyuki wrote: >> @@ -41,6 +42,7 @@ struct firmware_map_entry { >> const char *type; /* type of the memory range */ >> struct list_headlist; /* entry for the linked list */ >> struct kobject kobj; /* kobject for eac

Re: [PATCH v5 03/14] memory-hotplug: remove redundant codes

2012-12-26 Thread Tang Chen
Hi Kamezawa-san, Thanks for the reviewing. Please see below. :) On 12/26/2012 11:20 AM, Kamezawa Hiroyuki wrote: > (2012/12/24 21:09), Tang Chen wrote: >> From: Wen Congyang >> >> offlining memory blocks and checking whether memory blocks are offlined >> are very simil

Re: [PATCH v5 07/14] memory-hotplug: move pgdat_resize_lock into sparse_remove_one_section()

2012-12-25 Thread Tang Chen
On 12/26/2012 11:47 AM, Kamezawa Hiroyuki wrote: > (2012/12/24 21:09), Tang Chen wrote: >> In __remove_section(), we locked pgdat_resize_lock when calling >> sparse_remove_one_section(). This lock will disable irq. But we don't need >> to lock the whole function.

Re: [PATCH v5 06/14] memory-hotplug: implement register_page_bootmem_info_section of sparse-vmemmap

2012-12-25 Thread Tang Chen
On 12/25/2012 04:09 PM, Jianguo Wu wrote: + + if (!cpu_has_pse) { + next = (addr + PAGE_SIZE)& PAGE_MASK; + pmd = pmd_offset(pud, addr); + if (pmd_none(*pmd)) + continue; +

Re: [PATCH v5 08/14] memory-hotplug: Common APIs to support page tables hot-remove

2012-12-25 Thread Tang Chen
On 12/26/2012 11:11 AM, Tang Chen wrote: On 12/26/2012 10:49 AM, Tang Chen wrote: On 12/25/2012 04:17 PM, Jianguo Wu wrote: + +static void __meminit free_pagetable(struct page *page, int order) +{ + struct zone *zone; + bool bootmem = false; + unsigned long magic; + + /* bootmem page has

Re: [PATCH v5 08/14] memory-hotplug: Common APIs to support page tables hot-remove

2012-12-25 Thread Tang Chen
On 12/26/2012 10:49 AM, Tang Chen wrote: On 12/25/2012 04:17 PM, Jianguo Wu wrote: + +static void __meminit free_pagetable(struct page *page, int order) +{ + struct zone *zone; + bool bootmem = false; + unsigned long magic; + + /* bootmem page has reserved flag */ + if (PageReserved(page

Re: [PATCH v5 08/14] memory-hotplug: Common APIs to support page tables hot-remove

2012-12-25 Thread Tang Chen
On 12/25/2012 04:17 PM, Jianguo Wu wrote: + +static void __meminit free_pagetable(struct page *page, int order) +{ + struct zone *zone; + bool bootmem = false; + unsigned long magic; + + /* bootmem page has reserved flag */ + if (PageReserved(page)) { +

[PATCH v5 12/14] memory-hotplug: memory_hotplug: clear zone when removing the memory

2012-12-24 Thread Tang Chen
From: Yasuaki Ishimatsu When a memory is added, we update zone's and pgdat's start_pfn and spanned_pages in the function __add_zone(). So we should revert them when the memory is removed. The patch adds a new function __remove_zone() to do this. Signed-off-by: Yasuaki Ishimatsu Signed-off-by:

[PATCH v5 13/14] memory-hotplug: remove sysfs file of node

2012-12-24 Thread Tang Chen
This patch introduces a new function try_offline_node() to remove sysfs file of node when all memory sections of this node are removed. If some memory sections of this node are not removed, this function does nothing. Signed-off-by: Wen Congyang Signed-off-by: Tang Chen --- drivers/acpi

[PATCH v5 11/14] memory-hotplug: Integrated __remove_section() of CONFIG_SPARSEMEM_VMEMMAP.

2012-12-24 Thread Tang Chen
Currently __remove_section for SPARSEMEM_VMEMMAP does nothing. But even if we use SPARSEMEM_VMEMMAP, we can unregister the memory_section. Signed-off-by: Yasuaki Ishimatsu Signed-off-by: Wen Congyang Signed-off-by: Tang Chen --- mm/memory_hotplug.c | 11 --- 1 files changed, 0

[PATCH v5 14/14] memory-hotplug: free node_data when a node is offlined

2012-12-24 Thread Tang Chen
From: Wen Congyang We call hotadd_new_pgdat() to allocate memory to store node_data. So we should free it when removing a node. Signed-off-by: Wen Congyang --- mm/memory_hotplug.c | 20 +++- 1 files changed, 19 insertions(+), 1 deletions(-) diff --git a/mm/memory_hotplug.c b

[PATCH v5 08/14] memory-hotplug: Common APIs to support page tables hot-remove

2012-12-24 Thread Tang Chen
cleared. In this case, the page used as PT/PMD can be freed. Signed-off-by: Yasuaki Ishimatsu Signed-off-by: Jianguo Wu Signed-off-by: Wen Congyang Signed-off-by: Tang Chen --- arch/x86/include/asm/pgtable_types.h |1 + arch/x86/mm/init_64.c| 297

[PATCH v5 10/14] memory-hotplug: remove memmap of sparse-vmemmap

2012-12-24 Thread Tang Chen
sparc. Signed-off-by: Yasuaki Ishimatsu Signed-off-by: Jianguo Wu Signed-off-by: Wen Congyang Signed-off-by: Tang Chen --- arch/arm64/mm/mmu.c |3 +++ arch/ia64/mm/discontig.c |4 arch/powerpc/mm/init_64.c |4 arch/s390/mm/vmem.c |4 arch/sparc/mm

[PATCH v5 07/14] memory-hotplug: move pgdat_resize_lock into sparse_remove_one_section()

2012-12-24 Thread Tang Chen
rwise the WARN_ON_ONCE() in smp_call_function_many() will be triggered. Signed-off-by: Tang Chen Signed-off-by: Lai Jiangshan Signed-off-by: Wen Congyang --- mm/memory_hotplug.c |4 mm/sparse.c |5 - 2 files changed, 4 insertions(+), 5 deletions(-) diff --git

[PATCH v5 06/14] memory-hotplug: implement register_page_bootmem_info_section of sparse-vmemmap

2012-12-24 Thread Tang Chen
From: Yasuaki Ishimatsu For removing memmap region of sparse-vmemmap which is allocated bootmem, memmap region of sparse-vmemmap needs to be registered by get_page_bootmem(). So the patch searches pages of virtual mapping and registers the pages by get_page_bootmem(). Note: register_page_bootmem

[PATCH v5 05/14] memory-hotplug: introduce new function arch_remove_memory() for removing page table depends on architecture

2012-12-24 Thread Tang Chen
From: Wen Congyang For removing memory, we need to remove page table. But it depends on architecture. So the patch introduce arch_remove_memory() for removing page table. Now it only calls __remove_pages(). Note: __remove_pages() for some archtecuture is not implemented (I don't know how t

[PATCH v5 09/14] memory-hotplug: remove page table of x86_64 architecture

2012-12-24 Thread Tang Chen
This patch searches a page table about the removed memory, and clear page table for x86_64 architecture. Signed-off-by: Wen Congyang Signed-off-by: Jianguo Wu Signed-off-by: Jiang Liu Signed-off-by: Tang Chen --- arch/x86/mm/init_64.c | 10 ++ 1 files changed, 10 insertions(+), 0

[PATCH v5 03/14] memory-hotplug: remove redundant codes

2012-12-24 Thread Tang Chen
From: Wen Congyang offlining memory blocks and checking whether memory blocks are offlined are very similar. This patch introduces a new function to remove redundant codes. Signed-off-by: Wen Congyang --- mm/memory_hotplug.c | 101 --- 1 files c

[PATCH v5 02/14] memory-hotplug: check whether all memory blocks are offlined or not when removing memory

2012-12-24 Thread Tang Chen
From: Yasuaki Ishimatsu We remove the memory like this: 1. lock memory hotplug 2. offline a memory block 3. unlock memory hotplug 4. repeat 1-3 to offline all memory blocks 5. lock memory hotplug 6. remove memory(TODO) 7. unlock memory hotplug All memory blocks must be offlined before removing m

[PATCH v5 01/14] memory-hotplug: try to offline the memory twice to avoid dependence

2012-12-24 Thread Tang Chen
From: Wen Congyang memory can't be offlined when CONFIG_MEMCG is selected. For example: there is a memory device on node 1. The address range is [1G, 1.5G). You will find 4 new directories memory8, memory9, memory10, and memory11 under the directory /sys/devices/system/memory/. If CONFIG_MEMCG i

[PATCH v5 04/14] memory-hotplug: remove /sys/firmware/memmap/X sysfs

2012-12-24 Thread Tang Chen
From: Yasuaki Ishimatsu When (hot)adding memory into system, /sys/firmware/memmap/X/{end, start, type} sysfs files are created. But there is no code to remove these files. The patch implements the function to remove them. Note: The code does not free firmware_map_entry which is allocated by boot

[PATCH v5 00/14] memory-hotplug: hot-remove physical memory

2012-12-24 Thread Tang Chen
emory block. Patch3: new patch, no logical change, just remove reduntant codes. Patch9: merge the patch from wujianguo into this patch. flush tlb on all cpu after the pagetable is changed. Patch12: new patch, free node_data when a node is offlined. Tang Chen (5): memory-hotplug:

Re: [Patch v4 09/12] memory-hotplug: remove page table of x86_64 architecture

2012-12-06 Thread Tang Chen
On 11/27/2012 06:00 PM, Wen Congyang wrote: For hot removing memory, we sholud remove page table about the memory. So the patch searches a page table about the removed memory, and clear page table. (snip) +void __meminit +kernel_physical_mapping_remove(unsigned long start, unsigned long end)

Re: [Patch v4 08/12] memory-hotplug: remove memmap of sparse-vmemmap

2012-12-06 Thread Tang Chen
Hi Wu, I met some problems when I was digging into the code. It's very kind of you if you could help me with that. :) If I misunderstood your code, please tell me. Please see below. :) On 12/03/2012 10:23 AM, Jianguo Wu wrote: Signed-off-by: Jianguo Wu Signed-off-by: Jiang Liu --- include/li

Re: [Patch v4 08/12] memory-hotplug: remove memmap of sparse-vmemmap

2012-12-04 Thread Tang Chen
Hi Wu, On 12/04/2012 08:20 PM, Jianguo Wu wrote: (snip) Seems that we have different ways to handle pages allocated by bootmem or by regular allocator. Is the checking way in [PATCH 09/12] available here ? +/* bootmem page has reserved flag */ +if (PageReserved(page)) { .. +}

Re: [Patch v4 03/12] memory-hotplug: remove redundant codes

2012-12-04 Thread Tang Chen
On 12/04/2012 05:22 PM, Tang Chen wrote: On 11/27/2012 06:00 PM, Wen Congyang wrote: offlining memory blocks and checking whether memory blocks are offlined are very similar. This patch introduces a new function to remove redundant codes. CC: David Rientjes CC: Jiang Liu CC: Len Brown CC

Re: [Patch v4 12/12] memory-hotplug: free node_data when a node is offlined

2012-12-04 Thread Tang Chen
Morton CC: KOSAKI Motohiro CC: Yasuaki Ishimatsu Signed-off-by: Wen Congyang Reviewed-by: Tang Chen --- mm/memory_hotplug.c | 20 +++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 449663e..d1451ab 100644 --- a

Re: [Patch v4 11/12] memory-hotplug: remove sysfs file of node

2012-12-04 Thread Tang Chen
CC: Len Brown CC: Christoph Lameter Cc: Minchan Kim CC: Andrew Morton CC: KOSAKI Motohiro CC: Yasuaki Ishimatsu Signed-off-by: Wen Congyang Reviewed-by: Tang Chen --- drivers/acpi/acpi_memhotplug.c | 8 +- include/linux/memory_hotplug.h | 2 +- mm/memory_hotplug.c| 58

  1   2   >