On Wed, 12 Aug 2026 22:37:06 -0400 Gregory Price <[email protected]> wrote:
> On Thu, Aug 06, 2026 at 05:09:31PM +0900, Rakie Kim wrote:
> > Package-aware weighted interleave places a task's weighted-interleave
> > pages on the NUMA nodes of its local package, so that interleave traffic
> > does not have to cross the interconnect to another package. This keeps
> > each node's weight aligned with the bandwidth the task actually gets
> > from it, so effective bandwidth holds up on a system that has more than
> > one package. (A package is a CPU socket together with the memory
> > attached to it.)
> >
>
> The only major question I have before I dig into the actual patches is
> whether you actually need this on a *per-task* basis, rather than simply
> a *per-process* basis - because that's all this really buys you.
>
> You can already do what is described here by simply using a combination
> of cpuset and mempolicy
>
> cpuset.mems = 0,2
> mempolicy = weighted interleave --all
>
> If this is actually required on a per-task basis, then I agree this
> concept is reasonable. I just want to make we're grounded on a real
> use case before we go adding this complexity.
>
> ~Gregory
Hello Gregory,
Thank you for reviewing this series. My description was not precise
enough, and it seems to have confused several people. Let me go over
the example again.
I labelled the rows "from CPU 0" and "from CPU 1", which reads as two
separate tasks, each pinned to one place. What the rows were meant to
show is which package the allocation is requested from:
node0 node1 node2 node3
from package 0: 2 0 1 0
from package 1: 0 2 0 1
Both rows come from one policy at the same time. Which row applies is
decided per allocation, by the package the requesting CPU is in.
The cpuset.mems = 0,2 you describe is the node set of package 0, so
the tasks in that cgroup use node0 and node2 wherever they run.
The feature I am proposing does not fix a package that way. It uses
the nodes of the package the request comes from.
To build the feature I am proposing with cpuset, I think you would
need a cgroup per package and would have to place each thread in the
one matching the package its CPU is in.
cpuset.mems does not follow the CPU, so unless cpuset.cpus is set
alongside it, a thread running on a CPU of package 1 would use node0
and node2. For a program with tens or hundreds of threads, keeping
that placement right does not look easy.
To put it plainly, what this series aims for is that when a process
with many threads runs across several packages, each thread's
allocations are weighted-interleaved within the nodes of the package
that thread is on.
There is one process and one policy, but the nodes actually used
differ with where each thread sits. That is why I think this has to be
per task rather than per process.
The workloads I had in mind are memory-intensive server workloads:
vector databases as used in AI applications, or in-memory key-value
caching services that sit in front of large services and keep their
data in memory.
They run with a great many threads, spread across several packages and
running at the same time, and they want as much memory bandwidth as
they can get.
That is also why the numbers were measured with MLC. It produces high
bandwidth from many threads, much like those workloads, while still
giving a quantitative result.
The runs used "numactl -w all mlc" with no CPU binding, so the threads
were spread over both packages, and in that configuration the
bandwidth was higher than with the existing weighted interleave.
That is the reasoning behind my thinking that this belongs at the task
level. In the next version I will strengthen the documentation,
including why this feature is needed.
Thank you for taking the time to review this and for the question. It
made it much clearer to me what the cover letter still needs to
explain.
Rakie Kim
