Andrew Morton wrote:
Martin Hicks <[EMAIL PROTECTED]> wrote:
This patch introduces a new sysctl for NUMA systems that tries to drop
as much of the page cache as possible from a set of nodes. The
motivation for this patch is for setting up High Performance Computing
jobs, where initial memory placement is very important to overall
performance.
- Using a write to /proc for this seems a bit hacky. Why not simply add
a new system call for it?
We did it this way because it was easier to get it into SLES9 that way.
But there is no particular reason that we couldn't use a system call.
It's just that we figured adding system calls is hard.
- Starting a kernel thread for each node might be overkill. Yes, it
would take longer if one process was to do all the work, but does this
operation need to be very fast?
It is possible that this call might need to be executed at the start of
each batch job in the system. The reason for using a kernel thread was
that there was no good way to start concurrency due to a write to /proc.
If it does, then userspace could arrange for that concurrency by
starting a number of processes to perform the toss, each with a different
nodemask.
That works fine as well if we can get a system call number assigned and
avoids the hackiness of both /proc and the kernel threads.
- Dropping "as much pagecache as possible" might be a bit crude. I
wonder if we should pass in some additional parameter which specifies how
much of the node's pagecache should be removed.
Or, better, specify how much free memory we will actually require on
this node. The syscall terminates when it determines that enough
pagecache has been removed.
Our thoughts exactly. This is clearly a "big hammer" and we want to
make a lighter hammer to free up a certain number of pages. Indeed,
we would like to have these calls occur automatically from __alloc_pages()
when we try to allocate local storage and find that there isn't any.
For our workloads, we want to free up unmapped, clean pagecache, if that
is what is keeping us from allocating a local page. Not all workloads
want that, however, so we would probably use a sysctl() to enable/disable
this.
However, the first step is to do this manually from user space.
- To make the syscall more general, we should be able to reclaim mapped
pagecache and anonymous memory as well.
So what it comes down to is
sys_free_node_memory(long node_id, long pages_to_make_free, long what_to_free)
where `what_to_free' consists of a bunch of bitflags (unmapped pagecache,
mapped pagecache, anonymous memory, slab, ...).
Do we have to implement all of those or just allow for the possibility of
that
being implemented in the future? E. g. in our case we'd just implement the
bit that says "unmapped pagecache".
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
--
Best Regards,
Ray
-----------------------------------------------
Ray Bryant
512-453-9679 (work) 512-507-7807 (cell)
[EMAIL PROTECTED] [EMAIL PROTECTED]
The box said: "Requires Windows 98 or better",
so I installed Linux.
-----------------------------------------------
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
