On 5/9/2010 11:28 PM, Claus Assmann wrote:
PS: you might want to run some of those disk I/O benchmarks
to determine the number of IOPs your system can provide.
Thanks, everyone, for your help. I followed Nick's advice and went in
the server room to watch the lights, and they're really not blinken that
much. I had expected them to be on constantly.
I tried running bonnie -s 1024 while watching the output of iostat. The
iostat -w 5 -d output at "rest" (relaying mail, LA 0.9-1.5) is 8-16
KB/t, 0-15 t/s, 0.01-0.10 MB/s. During the run it seemed to max out at
~10 MB/s on writes and ~32 MB/s on reads.
I also ran Jeff Ross' first dd test:
> jr...@varley:/var/postgresql $ sudo time dd if=/dev/zero of=big_file
> bs=8k count=1024000
> 1024000+0 records in
> 1024000+0 records out
> 8388608000 bytes transferred in 276.573 secs (30330468 bytes/sec)
> 276.60 real 0.17 user 60.39 sys
# time dd if=/dev/zero of=big_file bs=8k count=1024000
1024000+0 records in
1024000+0 records out
8388608000 bytes transferred in 204.444 secs (41031301 bytes/sec)
3m25.64s real 0m0.57s user 0m49.42s system
So, I got slightly better performance out of these 10k disks. iostat
said I was doing about 20mb/sec. I'll have to run the /dev/null test
later so my users don't yell at me about delayed emails.
I have no milters running. I do have TLS enabled, but I wouldn't think
that'd have an impact on a 3 GHz machine, and if it did I'd expect high
CPU use. I'll keep looking, but at this point I'm thinking I'm not disk
bound.
For completeness, bioctl and dmesg pasted below. I'm running 4.7-stable
with patch 004. The dmesg says it's a Smart Array 64xx, but I'm pretty
sure it's a 6i, if that matters.
Volume Status Size Device
ciss0 0 Online 72833679360 sd0 RAID1
0 Online 72834973696 0:0.0 noencl <COMPAQ BD07288277 >
'3KT08MDM000075266AF3'
1 Online 72834973696 0:1.0 noencl <COMPAQ BD0728856A >
'AAL1P5A0F2B80541'
OpenBSD 4.7 (GENERIC.MP) #1: Thu May 6 01:12:41 EDT 2010
r...@pa508bld47-64.gmaccm.com:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 2146381824 (2046MB)
avail mem = 2079801344 (1983MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.3 @ 0xec000 (56 entries)
bios0: vendor HP version "P52" date 07/16/2007
bios0: HP ProLiant DL360 G4
acpi0 at bios0: rev 2
acpi0: tables DSDT FACP SPCR MCFG APIC
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(TM) CPU 3.00GHz, 3000.60 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR,NXE,LONG
cpu0: 1MB 64b/line 8-way L2 cache
cpu0: apic clock running at 200MHz
cpu1 at mainbus0: apid 6 (application processor)
cpu1: Intel(R) Xeon(TM) CPU 3.00GHz, 3000.11 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR,NXE,LONG
cpu1: 1MB 64b/line 8-way L2 cache
ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
ioapic1 at mainbus0: apid 9 pa 0xfec10000, version 20, 24 pins
ioapic1: misconfigured as apic 0, remapped to apid 9
ioapic2 at mainbus0: apid 10 pa 0xfec82000, version 20, 24 pins
ioapic3 at mainbus0: apid 11 pa 0xfec82400, version 20, 24 pins
acpiprt0 at acpi0: bus 1 (IP2P)
acpiprt1 at acpi0: bus 2 (ICHR)
acpiprt2 at acpi0: bus 7 (PCXA)
acpiprt3 at acpi0: bus 10 (PCXB)
acpiprt4 at acpi0: bus 6 (PTB0)
acpiprt5 at acpi0: bus 13 (PTA0)
acpiprt6 at acpi0: bus 3 (PTC0)
acpiprt7 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0
acpicpu1 at acpi0
acpitz0 at acpi0: critical temperature 31 degC
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel E7520 Host" rev 0x0c
ppb0 at pci0 dev 2 function 0 "Intel E7520 PCIE" rev 0x0c
pci1 at ppb0 bus 13
ppb1 at pci0 dev 4 function 0 "Intel E7520 PCIE" rev 0x0c
pci2 at ppb1 bus 6
ppb2 at pci2 dev 0 function 0 "Intel PCIE-PCIE" rev 0x09
pci3 at ppb2 bus 7
ppb3 at pci2 dev 0 function 2 "Intel PCIE-PCIE" rev 0x09
pci4 at ppb3 bus 10
ppb4 at pci0 dev 6 function 0 "Intel E7520 PCIE" rev 0x0c
pci5 at ppb4 bus 3
ppb5 at pci0 dev 28 function 0 "Intel 6300ESB PCIX" rev 0x02
pci6 at ppb5 bus 2
ciss0 at pci6 dev 1 function 0 "Compaq Smart Array 64xx" rev 0x01: apic
9 int 0 (irq 5)
ciss0: 1 LD, HW rev 1, FW 2.84/2.84, 64bit fifo
scsibus0 at ciss0: 1 targets
sd0 at scsibus0 targ 0 lun 0: <HP, LOGICAL VOLUME, 2.84> SCSI2 0/direct
fixed
sd0: 69459MB, 512 bytes/sec, 142253280 sec total
bge0 at pci6 dev 2 function 0 "Broadcom BCM5704C" rev 0x10, BCM5704 B0
(0x2100): apic 9 int 1 (irq 5), address 00:12:79:93:07:ff
brgphy0 at bge0 phy 1: BCM5704 10/100/1000baseT PHY, rev. 0
bge1 at pci6 dev 2 function 1 "Broadcom BCM5704C" rev 0x10, BCM5704 B0
(0x2100): apic 9 int 2 (irq 5), address 00:12:79:93:07:fe
brgphy1 at bge1 phy 1: BCM5704 10/100/1000baseT PHY, rev. 0
uhci0 at pci0 dev 29 function 0 "Intel 6300ESB USB" rev 0x02: apic 8 int
16 (irq 5)
uhci1 at pci0 dev 29 function 1 "Intel 6300ESB USB" rev 0x02: apic 8 int
19 (irq 5)
"Intel 6300ESB WDT" rev 0x02 at pci0 dev 29 function 4 not configured
"Intel 6300ESB APIC" rev 0x02 at pci0 dev 29 function 5 not configured
ehci0 at pci0 dev 29 function 7 "Intel 6300ESB USB" rev 0x02: apic 8 int
23 (irq 5)
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb6 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0x0a
pci7 at ppb6 bus 1
vga1 at pci7 dev 3 function 0 "ATI Rage XL" rev 0x27
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
"Compaq iLO" rev 0x01 at pci7 dev 4 function 0 not configured
"Compaq iLO" rev 0x01 at pci7 dev 4 function 2 not configured
pcib0 at pci0 dev 31 function 0 "Intel 6300ESB LPC" rev 0x02
pciide0 at pci0 dev 31 function 1 "Intel 6300ESB IDE" rev 0x02: DMA,
channel 0 configured to compatibility, channel 1 configured to compatibility
atapiscsi0 at pciide0 channel 0 drive 0
scsibus1 at atapiscsi0: 2 targets
cd0 at scsibus1 targ 0 lun 0: <TEAC, DV-28E-N, C.6B> ATAPI 5/cdrom removable
cd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
pciide0: channel 1 disabled (no drives)
usb1 at uhci0: USB revision 1.0
uhub1 at usb1 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb2 at uhci1: USB revision 1.0
uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
com1: probed fifo depth: 0 bytes
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot
wsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
midi0 at pcppi0: <PC speaker>
spkr0 at pcppi0
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec
mtrr: Pentium Pro MTRR support
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
root on sd0a swap on sd0b dump on sd0b
