Thanks Sherry, will do, though probably not for a couple of days due to other issues. In the meantime, I have seen references for years wrt "kernel cage" but have never groked what it was really about. Any good "kcage for dummies" places I might look for enlightenment? -d
> -----Original Message----- > From: Sherry Moore [mailto:[EMAIL PROTECTED] > Sent: Thursday, September 01, 2005 11:46 AM > To: David McDaniel (damcdani) > Cc: Eric C. Saxe; perf-discuss@opensolaris.org > Subject: Re: [perf-discuss] Re: Puzzling scheduler behavior > > Hi David, > > In addition to the various variables you can tune to help the > scheduler make better decisions, here are some other things > you can do at different levels: > > - Turn off kernel cage. Put the following line in /etc/system and > reboot. > > set kcage_on=0 > > Without the kernel cage constraint the memory allocation should > spread out much better. (I haven't done much performace work in > this area for a while, but I hope I am still right. :) ) > > - across-board interleaving > > I was not directly involved in the development of this particular > platform so I don't know for sure whether the firmware > provides the > same options. But assuming that it closely reassembles the x800 > series, you can log onto the system controller, connect to the > platform's console and do: > > sc0:SC> setupdomain > > There should be an option for changing memory interleaving. Set > it to "across-boards" and reboot the system. > > Depending on the DIMM configuration on various boards, a logical > bank might not actually span all boards. Use "cfgadm -av" to see > the resulting configuration. > > With across board memory interleaving, you will be left with a UMA > machine, so you won't be able to take advantage of all the cool > stuff that Eric and his team has implemented over the last few > years. I would be interested in knowing how the performance > compares. > > Sherry > -- > [EMAIL PROTECTED], Solaris Kernel Development, > http://blogs.sun.com/sherrym > > On Thu, Sep 01, 2005 at 08:40:41AM -0700, David McDaniel > (damcdani) wrote: > > Very, very enlightening, Eric. Its really terrific to > have this kind > > of channel for dialog. > > The "return to home base" behavior you describe is clearly > > consistent with what I see and makes perfect sense. > > Let me followup with a question. In this application, > processes have > > not only their "own" memory, ie heap, stack program text and data, > > etc, but they also share a moderately large (~ 2-5GB today) > amount of > > memory in the form of mmap'd files. From Sherry Moore's previous > > posts, I'm assuming that at startup time that would actually be all > > allocated in one board. Since I'm contemplating moving > processes onto > > psrsets off that board, would it be plausible to assume > that I might > > get slightly better net throughput if I could somehow spread that > > across all the boards? I know its speculation of the > highest order, so > > maybe my real question is whether that's even worth testing. > > In any case, I'd love to turn the knob you mention and > I'll look on > > the performance community page and see what kind of trouble > I can get > > into. If there are any particular items you think I should > check out, > > guidance is welcome. > > Regards > > -d > _______________________________________________ perf-discuss mailing list perf-discuss@opensolaris.org
