Eric Saxe wrote: > Roland Mainz wrote: > > Eric Saxe wrote: > >> Roland Mainz wrote: > >>> Is there any special handling of process groups to make sure that > >>> processes (and their LWPs) are kept together ? > >>> > >>> Think about an (imaginary (and simplified)) machine with 4 strands per > >>> core, 4 cores per socket, 4 sockets per board and 4 boards per cabinet > >>> and 4 cabinets where each layer increases the latency factor by 2 - in > >>> this case it would be desireable to keep processes which often > >>> communicate with each other closely together. > >>> > >>> Which "factors" does Solaris consider - "process group", "uid", "process > >>> hieracy", plain statistics or something else ? > >>> > >>> > >> On some platforms, Solaris will try to home threads from the same > >> process to the same lgroup (which > >> physically would be a board, node, etc). The platform specific code > >> plays a role in tuning the extent > >> to which this happens (how aggressive that coalescing policy is). It > >> boils down to optimizing for latency > >> vs. aggregate bandwidth. > >> > >> You can use liblgrp(3LIB) , or plgrp(1) to implement these types of > >> policies yourself on any NUMA platform. > >> > >> Beyond NUMA, we've been thinking about similar policy at the CMT level > >> as well (shared caches, etc). > >> Optimizing at this level can be a challenge as things tend to be quite a > >> bit more dynamic. ;P > > > > Yes, known problem... my question is more about the problem what happens > > when there are many free CPUs/boards with "equal" priority. Is there > > anything which looks at other attributes than the hardware to shepherd > > the threads/processes to stay together or migrate in groups ? > > Not currently. The lgroup "affinities" the kernel assigns (and the > assigning of affinities for the > same lgroup to threads of the same process) is all there is ATM. What > other attributes did you > have in mind?
I was thinking about things like "hardware layout", "scheduler class" or "user id" (and maybe zone/project id etc.). For example if you only have two users and more than two free boards you could put each user on a different board since they are unlikeky to produce heavy communication with each other (and rapid changes in load, number of threads or memory usage would only happen within the borders of one board and not affect the other user unless the resources of the one board are exceeded). Another idea would be to put processes of the "RT" (realtime class) or "BT" (batch class (RFE for that follows)) on seperate CPU _sockets_ if there are any free/idle ones. ---- Bye, Roland -- __ . . __ (o.\ \/ /.o) [EMAIL PROTECTED] \__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer /O /==\ O\ TEL +49 641 7950090 (;O/ \/ \O;) _______________________________________________ perf-discuss mailing list perf-discuss@opensolaris.org
