On Mon, 14 Jul 2014, Dave Hansen wrote:
> > + if (node == NUMA_NO_NODE) {
> > + node = page_to_nid(page);
> > + } else {
> > + int distance = node_distance(page_to_nid(page), node);
> > +
> > + /*
> > + * Do not migrate to memory that would not be reclaimed
> > + * from.
> > + */
> > + if (distance > RECLAIM_DISTANCE)
> > + goto out_unmap;
> > + }
>
> Isn't the reclaim behavior based on zone_reclaim_mode and not
> RECLAIM_DISTANCE directly? And isn't that reclaim behavior disabled by
> default?
>
Seems that RECLAIM_DISTANCE has taken on a life of its own independent of
zone_reclaim_mode as a heuristic, such as its use in creating sched
domains which would be unrelated.
> I think you should at least be consulting zone_reclaim_mode.
>
Good point, and it matches what the comment is saying about whether we'd
actually reclaim from the remote node to allocate thp on fault or not.
I'll add it.
After this change, we'll also need to consider the behavior of thp at
fault and whether remote HPAGE_PMD_SIZE memory when local memory is
low/fragmented is better than local PAGE_SIZE memory. In my page fault
latency testing on true NUMA machines it's convincing that it's not.
This makes me believe that, somewhat similar to this patch, when we
allocate thp memory at fault and zone_reclaim_mode is non-zero that we
should set only nodes with numa_node_id() <= RECLAIM_DISTANCE and then
otherwise fallback to the PAGE_SIZE fault path.
I've been hesitant to make that exact change, though, because it's a
systemwide setting and I really hope to avoid a prctl() that controls
zone reclaim for a particular process. Perhaps the NUMA balancing work
makes this more dependable.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
