Re: [perf-discuss] NUMA and interconnect transfers

2008-01-26 Thread Rafael Vanoni
Richard L. Hamilton wrote: BTW: Does Solaris/x86 support something like >> marking certain pages as non-cacheable (for example in some cases it may >> be nice to turn caching off if the traffic caused by cache coherency (or >> even the plain usage of cache lines for data which a

Re: [perf-discuss] NUMA and interconnect transfers

2008-01-26 Thread Richard L. Hamilton
> > > BTW: Does Solaris/x86 support something like > marking certain pages as > > > non-cacheable (for example in some cases it may > be nice to turn caching > > > off if the traffic caused by cache coherency (or > even the plain usage of > > > cache lines for data which are only read or > written

Re: [perf-discuss] NUMA and interconnect transfers

2008-01-18 Thread Roland Mainz
Rafael Vanoni wrote: > Roland Mainz wrote: > > Rafael Vanoni wrote: > >> Roland Mainz wrote: > >>> Jonathan Chew wrote: > Rafael Vanoni wrote: > > [snip] > >>> BTW: How does the topology code handle DR ? > >> Take a look at > >> http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts

Re: [perf-discuss] NUMA and interconnect transfers

2008-01-14 Thread Rafael Vanoni
Roland Mainz wrote: > Rafael Vanoni wrote: >> Roland Mainz wrote: >>> Jonathan Chew wrote: Rafael Vanoni wrote: > [snip] >>> BTW: How does the topology code handle DR ? >> Take a look at >> http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/os/lgrp.c#522. >> That's the pl

Re: [perf-discuss] NUMA and interconnect transfers

2008-01-14 Thread Roland Mainz
Rafael Vanoni wrote: > Roland Mainz wrote: > > Jonathan Chew wrote: > >> Rafael Vanoni wrote: [snip] > > BTW: How does the topology code handle DR ? > > Take a look at > http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/os/lgrp.c#522. > That's the platform independent portio

Re: [perf-discuss] NUMA and interconnect transfers

2008-01-14 Thread Rafael Vanoni
Roland Mainz wrote: > Jonathan Chew wrote: >> Rafael Vanoni wrote: >>> Hey everyone >>> >>> Is the kernel aware of the status of the interconnect between different >>> NUMA nodes ? >> No, not currently. It just assumes that there is some interconnect >> between the nodes and may know the latency b

Re: [perf-discuss] NUMA and interconnect transfers

2008-01-14 Thread Roland Mainz
Jonathan Chew wrote: > Rafael Vanoni wrote: > > Hey everyone > > > > Is the kernel aware of the status of the interconnect between different > > NUMA nodes ? > > No, not currently. It just assumes that there is some interconnect > between the nodes and may know the latency between them when the s

Re: [perf-discuss] NUMA and interconnect transfers

2008-01-14 Thread Rayson Ho
The Open MPI project has a framework called carto, which is used to represent the physical topology of a host: http://www.open-mpi.org/community/lists/devel/2008/01/2972.php See the OnHostTopologyDescription page, and also the attachment: https://svn.open-mpi.org/trac/ompi/wiki/OnHostTopologyDe

Re: [perf-discuss] NUMA and interconnect transfers

2007-12-12 Thread Jonathan Chew
Rayson Ho wrote: > On Dec 11, 2007 9:33 PM, Jonathan Chew <[EMAIL PROTECTED]> wrote: > >> Yes. If you're interested, there is also an update to that presentation at >> > > I think I've read that one too, and the chapter in the Solaris > Internals book... all 3 are interesting :-D > > >

Re: [perf-discuss] NUMA and interconnect transfers

2007-12-11 Thread Rayson Ho
On Dec 11, 2007 9:33 PM, Jonathan Chew <[EMAIL PROTECTED]> wrote: > Yes. If you're interested, there is also an update to that presentation at I think I've read that one too, and the chapter in the Solaris Internals book... all 3 are interesting :-D > Thanks for the pointers. I had the funny f

Re: [perf-discuss] NUMA and interconnect transfers

2007-12-11 Thread Jonathan Chew
Rayson Ho wrote: > On Dec 11, 2007 5:53 PM, Jonathan Chew <[EMAIL PROTECTED]> wrote: > >> No, not currently. It just assumes that there is some interconnect >> between the nodes and may know the latency between them when the system >> is not loaded. >> > > Jonathan, I just found out that y

Re: [perf-discuss] NUMA and interconnect transfers

2007-12-11 Thread Rafael Vanoni
Jonathan Chew wrote: > Rafael Vanoni wrote: >> Hey everyone >> >> Is the kernel aware of the status of the interconnect between >> different NUMA nodes ? >> > > No, not currently. It just assumes that there is some interconnect > between the nodes and may know the latency between them when th

Re: [perf-discuss] NUMA and interconnect transfers

2007-12-11 Thread Rayson Ho
On Dec 11, 2007 5:53 PM, Jonathan Chew <[EMAIL PROTECTED]> wrote: > No, not currently. It just assumes that there is some interconnect > between the nodes and may know the latency between them when the system > is not loaded. Jonathan, I just found out that you are the author of the MPO (Memory P

Re: [perf-discuss] NUMA and interconnect transfers

2007-12-11 Thread Jonathan Chew
Rafael Vanoni wrote: > Hey everyone > > Is the kernel aware of the status of the interconnect between different > NUMA nodes ? > No, not currently. It just assumes that there is some interconnect between the nodes and may know the latency between them when the system is not loaded. > For

[perf-discuss] NUMA and interconnect transfers

2007-12-11 Thread Rafael Vanoni
Hey everyone Is the kernel aware of the status of the interconnect between different NUMA nodes ? For instance, when an app is transferring data from one node to another, saturating the interconnect between these two. Are we aware of this ? I haven't seen any code that collects telemetry from