Quoting Lee Schermerhorn ([EMAIL PROTECTED]):
> On Tue, 2007-08-14 at 14:56 -0700, Christoph Lameter wrote:
> > On Tue, 14 Aug 2007, Lee Schermerhorn wrote:
> >
> > > > Ok then you did not have a NUMA system configured. So its okay for the
> > > > dummies to ignore the stuff. CONFIG_NODES_SHIFT is a constant and does
> > > > not
> > > > change. The first bit is always set.
> > >
> > > The first bit [node 0] is only set for the N_ONLINE [and N_POSSIBLE]
> > > mask. We could add the static init for the other masks, but since
> > > non-numa platforms are going through the __build_all_zonelists, they
> > > might as well set the MEMORY bits explicitly. Or, maybe you'll
> > > disagree ;-).
> >
> > The bitmaps can be completely ignored if !NUMA.
> >
> > In the non NUMA case we define
> >
> > static inline int node_state(int node, enum node_states state)
> > {
> > return node == 0;
> > }
> >
> > So its always true for node 0. The "bit" is set.
>
> The issue is with the N_*_MEMORY masks. They don't get initialized
> properly because node_set_state() is a no-op if !NUMA. So, where we
> look for intersections with or where we AND with the N_*_MEMORY masks we
> get the empty set.
>
> >
> > We are trying to get cpusets to work with !NUMA?
>
>
> Well, yes. In Serge's case, he's trying to use cpusets with !NUMA.
> He'll have to comment on the reasons for that. Looking at all of the
So I can lock a container to a cpu on a non-numa machine.
> #ifdefs and init/Kconfig, CPUSET does not depend on NUMA--only SMP and
> CONTAINERS [altho' methinks CPUSET should select CONTAINERS rather than
> depend on it...]. So, you can use cpusets to partition of cpus in
> non-NUMA configs.
>
> In the more general case, tho', I'm looking at all uses of the
> node_online_map and for_each_online_node, for instances where they
> should be replaced with one of the *_MEMORY masks. IMO, generic code
> that is compiled independent of any CONFIG option, like NUMA, should
> just work, independent of the config. Currently, as Serge has shown,
> this is not the case. So, I think we should fix the *_MEMORY maps to be
> correctly populated in both the NUMA and !NUMA cases. A couple of
> options:
>
> 1) just use node_set() when populating the masks,
>
> 2) initialize all masks to include at least cpu/node 0 in the !NUMA
> case.
>
> Serge chose #1 to fix his problem. I followed his lead to fix the other
> 2 places where node_set_state() was being used to initialize the NORMAL
> memory node mask and the CPU node mask. This will add a few unnecessary
> instructions to !NUMA configs, so we could change to #2.
>
> Thoughts?
Paul, is the mems stuff in cpusets only really useful for NUMA cases?
(I think it is... but am not sure) If so I suppose one alternative
could be to just disable that when !NUMA. But disabling cpusets when
!NUMA is completely wrong.
I personally would think that 1) is still the best option. Otherwise
the action
echo $SOME_CPU > /cpusets/set1/cpu
echo $SOME_CPU > /cpusets/set1/mems
works on a numa machine, and is wrong on a non-numa machine. With
option 1, the second part doesn't actually restrict the memory, but
at least /cpusets/set1/mems exists and $SOME_CPU doesn't have to be 0 to
be valid.
-serge
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
