We have not started our JVM's with numactl. I am not sure what (if any) benefit there has been to turning on NUMA in the BIOS. Turning it on could have in fact reduced performance. I suspect that Java is only using memory from one of the processors (since less than half of the physical memory is assigned to the JVM) and the other processors memory is being used for file system cache.
Clearly there is probably some room for improvement here. We have not invested much time in this as of yet. If anyone else has knowledge in this area please chime in! -Eric On Sun, Oct 24, 2010 at 10:36 PM, David Dabbs <dmda...@gmail.com> wrote: > Eric, > > Thanks for the detailed post! Did you need to start your JVMs with numactl > in order to take advantage of NUMA? > I know the board, OS and JVM must be configured properly, but it's not > clear if the JVMs must be started with numactl. > > > > Thanks, > > David > > > > From: Eric Rosenberry [mailto:epros...@gmail.com] > Sent: Sunday, October 24, 2010 10:17 PM > To: user@cassandra.apache.org > Subject: Experiences with Cassandra hardware planning > > All- > > Over the past nine months I have been working to tune our hardware > configuration to optimally balance CPU/RAM/disk/iops/network per node for > our Cassandra workload. Thanks much to those here who have provided > helpful > advice. > > I wanted to share back to the community some of the learnings we have come > across including the hardware configuration we have been successful with > (YMMV). This is still a work in progress naturally. > > I have written up a detailed blog post about this here: > > http://www.bitplumber.net/2010/10/a-cassandra-hardware-stack-dell-c1100s-ocz > -vertex-2-ssds-with-sandforce-arista-7048s/ > > Here are the highlights: > • Dell C1100 "cloud series" servers with 10x 2.5 inch drive bays > • OCZ Technology Vertex 2 MLC SSD's with the Sandforce 1200 series > controllers > • Arista Networks 7048 1U Top of Rack switches running MLAG with LACP to > the > hosts > Let me know if you have any questions! > > -Eric > >
