Hi David,
In addition to the various variables you can tune to help the scheduler
make better decisions, here are some other things you can do at
different levels:
- Turn off kernel cage. Put the following line in /etc/system and
reboot.
set kcage_on=0
Without the kernel cage constraint the memory allocation should
spread out much better. (I haven't done much performace work in
this area for a while, but I hope I am still right. :) )
- across-board interleaving
I was not directly involved in the development of this particular
platform so I don't know for sure whether the firmware provides the
same options. But assuming that it closely reassembles the x800
series, you can log onto the system controller, connect to the
platform's console and do:
sc0:SC> setupdomain
There should be an option for changing memory interleaving. Set
it to "across-boards" and reboot the system.
Depending on the DIMM configuration on various boards, a logical
bank might not actually span all boards. Use "cfgadm -av" to see
the resulting configuration.
With across board memory interleaving, you will be left with a UMA
machine, so you won't be able to take advantage of all the cool
stuff that Eric and his team has implemented over the last few
years. I would be interested in knowing how the performance
compares.
Sherry
--
[EMAIL PROTECTED], Solaris Kernel Development, http://blogs.sun.com/sherrym
On Thu, Sep 01, 2005 at 08:40:41AM -0700, David McDaniel (damcdani) wrote:
> Very, very enlightening, Eric. Its really terrific to have this kind
> of channel for dialog.
> The "return to home base" behavior you describe is clearly consistent
> with what I see and makes perfect sense.
> Let me followup with a question. In this application, processes have
> not only their "own" memory, ie heap, stack program text and data, etc,
> but they also share a moderately large (~ 2-5GB today) amount of memory
> in the form of mmap'd files. From Sherry Moore's previous posts, I'm
> assuming that at startup time that would actually be all allocated in
> one board. Since I'm contemplating moving processes onto psrsets off
> that board, would it be plausible to assume that I might get slightly
> better net throughput if I could somehow spread that across all the
> boards? I know its speculation of the highest order, so maybe my real
> question is whether that's even worth testing.
> In any case, I'd love to turn the knob you mention and I'll look on
> the performance community page and see what kind of trouble I can get
> into. If there are any particular items you think I should check out,
> guidance is welcome.
> Regards
> -d
_______________________________________________
perf-discuss mailing list
perf-discuss@opensolaris.org
