On Tue, Sep 16, 2014 at 01:17:44AM -0700, Dave Hansen wrote: > On 09/15/2014 08:29 PM, Peter Zijlstra wrote:
> > What is that cluster-on-die thing? I've heard it before but never could > > find anything on it. > > You split each socket in to halves. Each half gets slightly lower > latency to the memory on its half's memory controller. But, at an > increased latency cost to go to every other bit of memory on the system. > I've got some data squirreled away on exactly what the latencies are, > but it really is a substantial increase for memory, especially for the > memory that is still on package but is now "off-node". So it really splits the local memory DIMMs in two as well, ok. > I went back and re-read the text around there. I don't see any mention > of nodes or NUMA. It's just talking about the topology within the CPU > from what I can tell. > > Good thing it's just in "Documentation/ABI/testing/" I guess. > > Either way, are you saying that a core's "physical_package_id" should > depend on what BIOS options get set, and that different cores in the > same physical package should have different "physical_package_id"s? Yah, I more or less changed my mind halfway through the reply. I suppose it makes sense to have this mask as defined. It just doesn't match which how the mask is used in the scheduler (which is the main in-kernel user or the thing). > > The idea is that core_siblings (or rather cpu_core_mask) is a mask of > > all cores on a node. > > Why would we need that in the CPU topology information if we can get at > it easily here? > > /sys/devices/system/cpu/cpu0/node0/cpulist Good point, the 'problem' is that we currently have the static order of the masks, if we were to flip the MC and NUMA masks we need a condition to do that on and make sure everything is aware of that. CoD not being detectable sucks arse for sure :/ Also, I think we want to rename the mask to not be MC but PKG or so. > s390 has this concept of a "book" although it's not obvious at all where > this fits in to the hierarchy to me. The help text and comments are > pretty funny: > > Book scheduler support improves the CPU scheduler's decision > making when dealing with machines that have several books. > > Wow, really? Book scheduling helps with books? I would have never > guessed. Wish I knew what a book was. :) Google helped a bit: > > http://sysmagazine.com/posts/150430/ > > but it's still not clear. I think a book is below a node? > > SMT <= LLC <= MC <= BOOK <= NODE > > Or is a book a collection of nodes? > > SMT <= LLC <= MC <= NODE <= BOOK s390 is 'special' you should know that :-) Since s390 is a virtualized platform they cannot do NUMA since the vCPU can move around freely. So what they do to 'hide' stuff is big L4 caches, which they call BOOKS. > > The scheduler assumes: SMT <= LLC <= MC <= NODE and if setting the MC > > mask to cover multiple nodes works, its by accident. > > This makes a lot of sense to enforce inside the scheduler. But, do we > really need to expose all the naming all the way out to userspace? Dunno.. lets start out by not doing that (the safe option). -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/