On Thu, Jul 19, 2007 at 10:07:59AM -0700, Nishanth Aravamudan wrote:
> But I do think a second reason to do this is to make hugetlbfs behave
> like a normal fs -- that is read(), write(), etc. work on files in the
> mountpoint. But that is simply my opinion.
Mine as well.
-- wli
-
To unsubscribe
On Mon, Apr 30, 2007 at 10:43:10AM -0700, William Lee Irwin III wrote:
>> + Allocates the stack physically discontiguously and from high
>> + memory. Furthermore an unmapped guard page follows the stack.
>> + This is not for end-users. It's intended to trigger fatal
>> + system
On Thu, May 03, 2007 at 06:38:21PM -0700, Paul Jackson wrote:
> Adding Christoph Lameter <[EMAIL PROTECTED]> to the cc list, as he knows
> more about hugetlb pages than I do.
> This patch strikes me as a bit odd.
> Granted, it's solving what could be a touchy problem with a fairly
> simple solution
On Thu, May 03, 2007 at 12:39:30AM -0700, Bill Irwin wrote:
> As an aside, it looks like failures here need to eventually propagate
> to __cpu_up(). irq_ctx_init() needs to return a status, and its callers
> need to check it. irq_ctx_init() probably also needs to be __cpuinit.
Ign
Bill Irwin wrote:
>> I had the same question about yours and just brute-force merged. Not a
>> big deal for me to rediff against whatever everyone's working off of.
On Thu, May 03, 2007 at 12:07:29AM -0700, Jeremy Fitzhardinge wrote:
> I picked up one version of your patches
On Wed, May 02, 2007 at 10:48:09PM -0700, Bill Irwin wrote:
> Updated patch follows. Please add your Signed-off-by: if it meets your
> approval; I am operating on the assumption I should never do so myself.
> I'm a bit unsure of how to handle cpu 0 vs. potential freeing of per_c
Bill Irwin wrote:
>> Updated patch follows. Please add your Signed-off-by: if it meets your
>> approval;
On Wed, May 02, 2007 at 11:01:05PM -0700, Jeremy Fitzhardinge wrote:
> What does it apply to? I'm getting conflicts if I replace my patch with
> this. Or does it repl
On Wed, May 02, 2007 at 06:56:09PM -0700, Jeremy Fitzhardinge wrote:
> This fixes two bugs:
> - the stack allocation must be marked __cpuinit, since it gets called
>on resume as well.
> - presumably the interrupt stack should be freed on unplug if its
>going to get reallocated on every pl
cpu));
>> +
>> +for (i = 0; i < THREAD_SIZE/PAGE_SIZE; ++i)
>> +__free_page(per_cpu(irqstack_pages, cpu)[i]);
>> +}
On Wed, May 02, 2007 at 07:25:34PM -0700, Bill Irwin wrote:
[...]
Not sure if cpu 0 can ever be offlined, but it is remapped and so on.
So
On Wed, May 02, 2007 at 06:56:09PM -0700, Jeremy Fitzhardinge wrote:
> +static void __cpuinit __free_irqstack(int cpu, void *stk)
> +{
> + int i;
> +
> + if (!cpu)
> + return;
> +
> + unmap_vm_area(per_cpu(irqstack_area, cpu));
> +
> + for (i = 0; i < THREAD_SIZE/PAGE_SI
William Lee Irwin III wrote:
>> +unsigned long __kvaddr_to_paddr(unsigned long kvaddr)
>> +{
>> +if (high_memory)
>> +BUG_ON(kvaddr >= VMALLOC_START);
>> +else
>> +BUG_ON(kvaddr >= (unsigned long)__va(MAXMEM));
>> +return kvaddr - PAGE_OFFSET;
>> +}
On Wed, May
On Wed, May 02, 2007 at 09:28:46AM -0700, Jeremy Fitzhardinge wrote:
>>> I think this should be fixed now. Eric made all those writes
>>> unconditional (to fix a problem with PSE superpages not being created).
>>> The patch is in Andi's queue.
Bill Irwin <[EM
Bill Irwin wrote:
>> Brain dump before crashing for the night:
>> The patch refuses to clobber already-present pagetable entries of
>> whatever origin. There are pagetables prior to this setup covering the
>> address range just above PAGE_OFFSET. If this theory is correc
>> memory. Probably it does. Probably we'd prefer that it didn't ;)
On Wed, May 02, 2007 at 01:46:17AM -0700, Bill Irwin wrote:
> Brown paper bag time. I don't know how it got past me.
Brain dump before crashing for the night:
The patch refuses to clobber already-pr
On Wed, May 02, 2007 at 12:51:40AM -0700, Andrew Morton wrote:
> Thanks for the report. I can reproduce it.
> Bisection shows that x86_64-mm-paravirt-initial-pagetable.patch caused
> this.
> I didn't check whether the patch actually permits us to read kernel
> memory. Probably it does. Probably
On Tue, May 01, 2007 at 05:36:06PM -0500, Matt Mackall wrote:
>>> Can we register them lazily at request_irq time?
On Tue, May 01, 2007 at 03:51:25PM -0700, Bill Irwin wrote:
>> These IRQ stacks are per-cpu, not per-IRQ. It may make sense to
>> implement per-IRQ stacks,
At some point in the past, I wrote:
>> These IRQ stacks are per-cpu, not per-IRQ. It may make sense to
>> implement per-IRQ stacks, in which case dynamic allocation at the time
>> of request_irq() will make sense.
On Wed, May 02, 2007 at 12:07:45AM +0100, Alan Cox wrote:
> This depends if active I
At some point in the past, I wrote:
>> I just so happen to think we should implement a variety of CPU resource
>> limits beyond what we now do, so this, too, interests me.
On Wed, May 02, 2007 at 12:04:58AM +0100, Alan Cox wrote:
> Agreed - and make them all 64bit while doing the cleanup. One thin
On Mon, Apr 30, 2007 at 08:15:11PM +0100, Christoph Hellwig wrote:
>> So if you want to invest some time into getting this into mergeable
>> shape I'd suggest you redo the patch series in the following way:
>> patch 1: dynamic allocated irq stacks
On Tue, May 01, 2007 at 05:36:06PM -0500, Matt Ma
On Tue, May 01, 2007 at 06:27:24PM -0400, Theodore Tso wrote:
> There are two ways of implementing this. One is to have the
> JVM periodically poll using a pthread_getrusage() interface. A better
> choice might be some kind of per-thread CPU limit, that would result
> in a thread-specific S
On 5/1/07, Bill Irwin <[EMAIL PROTECTED]> wrote:
>> The basic
>> idea is to try to do it similarly to how everyone else does so userspace
>> (I suppose this would include glibc) don't have to bend over backward to
>> accommodate it. Or basically to do what ever
On 5/1/07, Bill Irwin <[EMAIL PROTECTED]> wrote:
>> A sort of note for me to refer back to when I get the rest of the way
>> here. AIX does this with getrusage(RUSAGE_THREAD,...), Solaris with
>> getrusage(RUSAGE_LWP,...),
On Tue, May 01, 2007 at 11:39:46AM -07
On Mon, Apr 09, 2007 at 04:53:15PM -0700, Andrew Morton wrote:
>> Seems sane. Could we please get it tested and get a full description in
>> place? Something which provides enough detail for the manpage maintainers.
>> Also, a quick comparison between Linux's RUSAGE_THREAD and $other-os's
>> impl
At some point in the past, I wrote:
>> +static void * __init __alloc_irqstack(int cpu)
>> +{
>> +if (!cpu)
>> +return __alloc_bootmem(THREAD_SIZE, THREAD_SIZE,
>> +__pa(MAX_DMA_ADDRESS));
>> +
>> +return (void *)__get_free_pages(GFP_KE
Bill Irwin a écrit :
>> as a stopgap measure, but I'm not all that interested in grabbing patch
>> credits where others could do it easily enough. Either of the config
>> alterations is fine by me as they now stand; maybe Eric Dumazet might
>> care to do something lik
Bill Irwin wrote:
>> It's a shame that the resource scalability implications of vmallocspace
>> allocations prevent this from being useful in production. One could, in
>> principle, establish guard pages within ZONE_NORMAL, but for 4KB stacks
>> it's somewhat awkw
Bill Irwin wrote:
>> This patch introduces CONFIG_DEBUG_STACK, which vmalloc()'s task and IRQ
>> stacks in order to establish guard pages. In such a manner any stack
>> overflow that references pages immediately adjacent to the stack is
>> immediately trapped with a
On Tue, May 01, 2007 at 06:26:23AM +0200, Eric Dumazet wrote:
>> Hum... We lose a usefull 2G/2G split. Should'nt we use a patch to change
>> PAGE_OFFSET to 0x800 instead of 0x7800 and keep 2G/2G split ?
On Tue, May 01, 2007 at 08:21:32AM +0200, Andi Kleen wrote:
> I dropped the patch for
On Mon, Apr 30, 2007 at 04:20:07PM -0700, Andrew Morton wrote:
> proper-prototype-for-hugetlb_get_unmapped_area.patch
...
> convert-hugetlbfs-to-use-vm_ops-fault.patch
...
> get_unmapped_area-handles-map_fixed-in-hugetlbfs.patch
...
> get_unmapped_area-doesnt-need-hugetlbfs-hacks-anymore.patch
This patch introduces CONFIG_DEBUG_STACK, which vmalloc()'s task and IRQ
stacks in order to establish guard pages. In such a manner any stack
overflow that references pages immediately adjacent to the stack is
immediately trapped with a fault, which precludes silent memory corruption
or difficult-t
IRQ stacks are a valuable stability feature. This patch makes them
unconditional, as there is no circumstance under which they do not
improve stability and they have no meaningful performance impact.
Signed-off-by: William Irwin <[EMAIL PROTECTED]>
Index: stack-paranoia/include/asm-i386/irq.h
==
Dynamically allocate IRQ stacks in order to conserve memory when using
IRQ stacks. cpu_possible_map is not now initialized in such a manner as
to provide a meaningful indication of how many CPU's might be in the
system, and features to appear in the sequel also require indirection,
so they themselv
This is a series of patches to improve i386 stack handling in several
manners. The net result is to improve system stability via IRQ stacks
while remaining memory efficient and to improve stack debugging.
[1/3] dynamically allocate irq stacks
This conserves memory while using IRQ stacks.
[
On Mon, Apr 30, 2007 at 08:15:11PM +0100, Christoph Hellwig wrote:
> So if you want to invest some time into getting this into mergeable
> shape I'd suggest you redo the patch series in the following way:
> patch 1: dynamic allocated irq stacks
> patch 2: make irqstacks unconditional, but allow s
On Mon, 30 Apr 2007, William Lee Irwin III wrote:
>> -static char softirq_stack[NR_CPUS * THREAD_SIZE]
>> -__attribute__((__aligned__(THREAD_SIZE)));
>> +static DEFINE_PER_CPU(char *, softirq_stack);
>> +static DEFINE_PER_CPU(char *, hardirq_stack);
>>
>> -static char hardirq_stack[NR
On Mon, Apr 30, 2007 at 10:38:19AM -0700, William Lee Irwin III wrote:
Here's what I did for i386 for someone concerned about blowing the
stack.
On Mon, Apr 30, 2007 at 10:45:10AM -0700, William Lee Irwin III wrote:
>>> Add checks to __pa() so it goes BUG() on vmallocspace addresses.
On
On Mon, Apr 30, 2007 at 08:15:11PM +0100, Christoph Hellwig wrote:
> So if you want to invest some time into getting this into mergeable
> shape I'd suggest you redo the patch series in the following way:
> patch 1: dynamic allocated irq stacks
> patch 2: make irqstacks unconditional, but allow s
On Sun, Mar 18, 2007 at 03:50:10AM +0100, Nick Piggin wrote:
> Yes, that should be the case. So would this mean that nonlinear protections
> don't work on regular files? I guess that's OK if Oracle and UML both use
> tmpfs/shm?
Sometimes ramfs is also used in the Oracle case. I presume that's even
On Tue, Mar 13, 2007 at 05:29:17PM -0700, Nish Aravamudan wrote:
> Ok, truly bizarre, I found that I was not running stock 2.6.20.3, but
> had your small hugetlb patch on top.
> So I went back and patched 2.6.20.1 with your patch, rebooted, got a
> soft lockup. Went back to stock 2.6.20.1 and did n
On Wed, Mar 07, 2007 at 05:26:48PM -0600, Adam Litke wrote:
> :) Enter my remove-is_file_hugepages() patches (which I posted a few
> weeks ago). I'll rework them and repost soon. That should help to make
> all of this cleaner.
Those were great. I've wanted something like them for a long, long ti
Bill Irwin <[EMAIL PROTECTED]> writes:
>> A comment to prepare others for the impending doubletake might be nice.
>> Or maybe just open-coding the equality check for &hugetlbfs_file_operations
>> in is_file_shm_hugepages() if others find it as jarring as I. Please
>&
On Tue, 2007-03-06 at 22:44 -0800, Bill Irwin wrote:
>> What do you see as the obstacle to eliminating nested IRQ's?
On Wed, Mar 07, 2007 at 04:34:52AM -0800, Arjan van de Ven wrote:
> political will, or maybe just the lack of convincing people so far
Political issues are sign
On Wed, Mar 07, 2007 at 11:47:42AM +0100, Peter Zijlstra wrote:
>> Well, now they don't, but it could be done or even exploited as a DoS.
On Wed, Mar 07, 2007 at 12:00:36PM +0100, Nick Piggin wrote:
> But so could nonlinear page reclaim. I think we need to restrict nonlinear
> mappings to root if
On Wed, 7 Mar 2007 01:29:03 -0800 Bill Irwin <[EMAIL PROTECTED]> wrote:
>> Guess what major real-life application not only uses nonlinear daily
>> but would even be very happy to see it extended with non-vma-creating
>> protections and more?
On Wed, Mar 07, 2007 at 01:39:4
On Wed, Mar 07, 2007 at 10:22:52AM +0100, Ingo Molnar wrote:
>>> ok. What do you think about the sys_remap_file_pages_prot() thing that
>>> Paolo has done in a nicely split up form - does that complicate things
>>> in any fundamental way? That is what is useful to
On Wed, Mar 07, 2007 at 10:28:21AM +0100, Nick Piggin wrote:
> Depending on whether anyone wants it, and what features they want, we
> could emulate the old syscall, and make a new restricted one which is
> much less intrusive.
> For example, if we can operate only on MAP_ANONYMOUS memory and speci
* Nick Piggin <[EMAIL PROTECTED]> wrote:
>> After these patches, I don't think there is too much burden. The main
>> thing left really is just the objrmap stuff, but that is just handled
>> with a minimal 'dumb' algorithm that doesn't cost much.
On Wed, Mar 07, 2007 at 10:22:52AM +0100, Ingo Mol
On Wed, 7 Mar 2007 09:27:55 +0100 Ingo Molnar <[EMAIL PROTECTED]> wrote:
>> btw., if we decide that nonlinear isnt worth the continuing maintainance
>> pain, we could internally implement/emulate sys_remap_file_pages() via a
>> call to mremap() and essentially deprecate it, without breaking the A
On Tue, Mar 06, 2007 at 10:51:01PM -0800, Andrew Morton wrote:
> Does anybody really pass a NULL `type' arg into filemap_nopage()?
The major vs. minor fault accounting patch that introduced the argument
didn't make non-NULL type arguments a requirement. It's essentially an
optional second return v
At some point in the past, I wrote:
>> I'm certainly in favor of the move; IRQ stacks could be made
>> rather deep and cheaply at that. I may get around to writing it this
>> week if no one else does it first.
On Tue, Mar 06, 2007 at 08:28:35PM -0800, Arjan van de Ven wrote:
> the irq stacks aren'
On Tue, 6 Mar 2007, Chuck Ebbert wrote:
>> In the 4k/4k stack i386 kernel, is there any fundamental reason it
>> can't be 4k/8k? We seem to be mostly hitting problems in overflowing
>> the IRQ stack... I think. Overhead would only be 4k per CPU for that.
On Tue, Mar 06, 2007 at 07:43:41PM +, H
On Saturday 03 March 2007, Bill Irwin wrote:
>> If you have a known-working kernel version, git-bisect might help you
>> track down where it was introduced. Given the messages prior to the
>> hugetlbpage.c BUG_ON I'd say that this is something else besides the
>>
Bill Irwin wrote:
>> Chuck, is any of this of any use to you?
On Tue, Mar 06, 2007 at 01:59:39PM -0500, Chuck Ebbert wrote:
> I said "simple." :)
> In the 4k/4k stack i386 kernel, is there any fundamental reason it
> can't be 4k/8k? We seem to be mostly hitting pro
On Wed, Feb 28, 2007 at 09:41:44PM +0100, Andi Kleen wrote:
> I suppose one could have a CONFIG_DEBUG_STACK_OVERFLOW that gets
> the stacks from vmalloc which would catch any overflow with its
> guard pages. This is you would need to change __pa() to handle
> that too because there might be still s
On Thu, 2007-03-01 at 16:08 -0800, Bill Irwin wrote:
>> Looks like I should grab these testcases for the sake of due diligence
>> (not to say I intend to alter maintenance style from primarily review,
>> approval, and bugfixing, not that I've been doing as much of any of th
On Fri, Mar 02, 2007 at 04:51:15PM +0300, Alexander Y. Fomichev wrote:
> I'm hit a bug on 2.6.21-rc1 at startup of mysql with 'large-pages' flag set.
> (at this point mysql trying to allocate pages from hugetlb pool by sysv
> shm syscalls). Seems like it could be triggered by previous badness
> an
At some point in the past, Mel Gorman wrote:
>> I can't think of a workload that totally makes a mess out of list-based.
>> However, list-based makes no guarantees on availability. If a system
>> administrator knows they need between 10,000 and 100,000 huge pages and
>> doesn't want to waste mem
On Fri, Mar 02, 2007 at 01:23:28PM -0500, Rik van Riel wrote:
> With 32 CPUs diving into the page reclaim simultaneously,
> each trying to scan a fraction of memory, this is disastrous
> for performance. A 256GB system should be even worse.
Thundering herds of a sort pounding the LRU locks from d
On Fri, 02 Mar 2007 12:43:42 -0500 Rik van Riel <[EMAIL PROTECTED]> wrote:
>> I can't share all the details, since a lot of the problems are customer
>> workloads.
>> One particular case is a 32GB system with a database that takes most
>> of memory. The amount of actually freeable page cache memor
On Thu, Mar 01, 2007 at 10:12:50AM +, Mel Gorman wrote:
> These are figures based on kernels patches with Andy Whitcrofts reclaim
> patches. You will see that the zone-based kernel is getting success rates
> closer to 40% as one would expect although there is still something amiss.
Yes, combin
On Thu, Mar 01, 2007 at 03:46:08PM -0800, Adam Litke wrote:
> static inline int is_file_hugepages(struct file *file)
> {
> - return file->f_op == &hugetlbfs_file_operations;
> + if (file->f_op == &hugetlbfs_file_operations)
> + return 1;
> + if (is_file_shm_hugepages(file)
On Wed, Feb 28, 2007 at 02:13:29PM -0600, Adam Litke wrote:
> Hey. While testing 2.6.21-rc2 with libhugetlbfs, the shm-fork test case
> causes the kernel to oops. To reproduce: Execute 'make check' in the
> latest libhugetlbfs source on a 2.6.21-rc2 kernel with 100 huge pages
> allocated. Using
On Thu, Mar 01, 2007 at 10:52:01PM +, Simon Arlott wrote:
> Whenever jiffies is started at a multiple of 5*HZ or wraps, calc_load is
> run exactly on the second which is when tasks using round_jiffies will
> be scheduled to run. This has a bad effect on the load average, making
> it tend tow
On Feb 28 2007 15:20, Bill Irwin wrote:
>> I don't know about the rest of the world, but halting the system in the
>> case of memory corruption sounds like an extremely good idea to me.
On Thu, Mar 01, 2007 at 12:36:47AM +0100, Jan Engelhardt wrote:
> Just because a rather
On Wed, Feb 28, 2007 at 09:41:44PM +0100, Andi Kleen wrote:
> Likely already too late then -- if critical state is overwritten
> you crashed before. Also a lot of stack intensive codes
> relatively large unused holes so it might miss the canary completely
> Anyways if you want a crash on context sw
65 matches
Mail list logo
