Normally I build VMware ESXi servers with enterprise-class WD SATA drives and I/O performance in FreeBSD VMs on the servers is fine. Whenever I build a VMware ESXi server with a RAID controller, IO performance is awful in FreeBSD VMs. I've previously seen this effect with VMware ESXi 3ware 9690SA-8I and 9650 RAID controllers, and now I'm seeing similar performance with a Dell 6/iR controller.
Any suggestions would be appreciated. Guy Details of the current environment: VMware ESXi 5.1 on Dell R610 4GB RAM, SAS 6/iR controller, 2x500GB disks in RAID1 set (default stripe size) and 1x1TB (no RAID). From VMware's client, I see I/O rates in the sub-MBps range and latencies peaking occasionally at 80 ms. FreeBSD 9.2 (RC2) amd64 in a VM with 2GB RAM assigned, virtual disks assigned from both the RAID1 set and 1TB (no RAID) drive, UFS+soft updates file systems. The virtual drives show up in FreeBSD attached to an mpt virtual controller: mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x1400-0x14ff mem 0xd0040000-0xd005ffff,0xd0020000-0xd003ffff irq 17 at device 16.0 on pci0 mpt0: MPI Version=1.2.0.0 I don't see anything else sharing the interrupt - vmstat -i shows: irq17: mpt0 77503 27 gstat is showing an abysmal 6 to 16 ops/s for requests on the virtual disks. I've used gpart to setup the GPT partition table on the virtual disk assigned from the 1TB drive with alignment for the first UFS partition at 1MB to try to optimize alignment: Geom name: da0 modified: false state: OK fwheads: 255 fwsectors: 63 last: 268435422 first: 34 entries: 128 scheme: GPT Providers: 1. Name: da0p1 Mediasize: 524288 (512k) Sectorsize: 512 Stripesize: 0 Stripeoffset: 20480 Mode: r0w0e0 rawuuid: d9e6e3e8-1bdb-11e3-b7c5-000c29cbf143 rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f label: gpboot length: 524288 offset: 20480 type: freebsd-boot index: 1 end: 1063 start: 40 2. Name: da0p2 Mediasize: 2147483648 (2.0G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 1048576 Mode: r1w1e2 rawuuid: fbd6cf40-1bdb-11e3-b7c5-000c29cbf143 rawtype: 516e7cb6-6ecf-11d6-8ff8-00022d09712b label: gprootfs length: 2147483648 offset: 1048576 type: freebsd-ufs index: 2 end: 4196351 start: 2048 3. Name: da0p3 Mediasize: 4294967296 (4.0G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 2148532224 Mode: r1w1e1 rawuuid: 0658208d-1bdc-11e3-b7c5-000c29cbf143 rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b label: gpswap length: 4294967296 offset: 2148532224 type: freebsd-swap index: 3 end: 12584959 start: 4196352 4. Name: da0p4 Mediasize: 130995437056 (122G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 2148532224 Mode: r1w1e2 rawuuid: 0ca5bc32-1bdc-11e3-b7c5-000c29cbf143 rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b label: gpusrfs length: 130995437056 offset: 6443499520 type: freebsd-swap index: 4 end: 268435422 start: 12584960 Consumers: 1. Name: da0 Mediasize: 137438953472 (128G) Sectorsize: 512 Mode: r3w3e8 sysctl vfs shows: vfs.ufs.dirhash_reclaimage: 5 vfs.ufs.dirhash_lowmemcount: 179 vfs.ufs.dirhash_docheck: 0 vfs.ufs.dirhash_mem: 0 vfs.ufs.dirhash_maxmem: 3481600 vfs.ufs.dirhash_minsize: 2560 vfs.ufs.rename_restarts: 0 vfs.nfs.downdelayinitial: 12 vfs.nfs.downdelayinterval: 30 vfs.nfs.keytab_enctype: 1 vfs.nfs.skip_wcc_data_onerr: 1 vfs.nfs.nfs3_jukebox_delay: 10 vfs.nfs.reconnects: 0 vfs.nfs.bufpackets: 4 vfs.nfs.debuglevel: 0 vfs.nfs.callback_addr: vfs.nfs.realign_count: 0 vfs.nfs.realign_test: 0 vfs.nfs.nfs_directio_allow_mmap: 1 vfs.nfs.nfs_keep_dirty_on_error: 0 vfs.nfs.nfs_directio_enable: 0 vfs.nfs.clean_pages_on_close: 1 vfs.nfs.commit_on_close: 0 vfs.nfs.prime_access_cache: 0 vfs.nfs.access_cache_timeout: 60 vfs.nfs.diskless_rootpath: vfs.nfs.diskless_valid: 0 vfs.nfs.nfs_ip_paranoia: 1 vfs.nfs.defect: 0 vfs.nfs.iodmax: 20 vfs.nfs.iodmin: 0 vfs.nfs.iodmaxidle: 120 vfs.devfs.rule_depth: 1 vfs.devfs.generation: 113 vfs.nfsd.disable_checkutf8: 0 vfs.nfsd.server_max_nfsvers: 4 vfs.nfsd.server_min_nfsvers: 2 vfs.nfsd.nfs_privport: 0 vfs.nfsd.async: 0 vfs.nfsd.enable_locallocks: 0 vfs.nfsd.issue_delegations: 0 vfs.nfsd.commit_miss: 0 vfs.nfsd.commit_blks: 0 vfs.nfsd.mirrormnt: 1 vfs.nfsd.minthreads: 1 vfs.nfsd.maxthreads: 1 vfs.nfsd.threads: 0 vfs.nfsd.request_space_used: 0 vfs.nfsd.request_space_used_highest: 0 vfs.nfsd.request_space_high: 13107200 vfs.nfsd.request_space_low: 8738133 vfs.nfsd.request_space_throttled: 0 vfs.nfsd.request_space_throttle_count: 0 vfs.nfsd.fha.enable: 1 vfs.nfsd.fha.bin_shift: 22 vfs.nfsd.fha.max_nfsds_per_fh: 8 vfs.nfsd.fha.max_reqs_per_nfsd: 0 vfs.nfsd.fha.fhe_stats: No file handle entries. vfs.pfs.trace: 0 vfs.pfs.vncache.misses: 0 vfs.pfs.vncache.hits: 0 vfs.pfs.vncache.maxentries: 0 vfs.pfs.vncache.entries: 0 vfs.acl_nfs4_old_semantics: 0 vfs.flushwithdeps: 0 vfs.unmapped_buf_allowed: 1 vfs.barrierwrites: 1 vfs.notbufdflashes: 0 vfs.flushbufqtarget: 100 vfs.mappingrestarts: 0 vfs.getnewbufrestarts: 337501 vfs.getnewbufcalls: 349444 vfs.hifreebuffers: 1524 vfs.lofreebuffers: 762 vfs.numfreebuffers: 13601 vfs.dirtybufthresh: 3084 vfs.hidirtybuffers: 3427 vfs.lodirtybuffers: 1713 vfs.numdirtybuffers: 11 vfs.recursiveflushes: 341175 vfs.altbufferflushes: 0 vfs.bdwriteskip: 0 vfs.dirtybufferflushes: 0 vfs.hirunningspace: 3538944 vfs.lorunningspace: 2359296 vfs.bufdefragcnt: 0 vfs.buffreekvacnt: 339965 vfs.bufreusecnt: 347292 vfs.hibufspace: 222625792 vfs.lobufspace: 222560256 vfs.maxmallocbufspace: 11131289 vfs.bufmallocspace: 0 vfs.maxbufspace: 223281152 vfs.unmapped_bufspace: 290652160 vfs.bufspace: 291602432 vfs.runningbufspace: 131072 vfs.vmiodirenable: 1 vfs.cache.numfullpathfound: 47 vfs.cache.numfullpathfail4: 0 vfs.cache.numfullpathfail2: 0 vfs.cache.numfullpathfail1: 0 vfs.cache.numfullpathcalls: 47 vfs.cache.numupgrades: 32 vfs.cache.numneghits: 2853 vfs.cache.numnegzaps: 16 vfs.cache.numposhits: 356750 vfs.cache.numposzaps: 282 vfs.cache.nummisszap: 10 vfs.cache.nummiss: 35615 vfs.cache.numchecks: 377860 vfs.cache.dotdothits: 19 vfs.cache.dothits: 146 vfs.cache.numcalls: 395710 vfs.cache.numcache: 29909 vfs.cache.numneg: 336 vfs.ncsizefactor: 2 vfs.ncnegfactor: 16 vfs.read_min: 1 vfs.read_max: 64 vfs.write_behind: 1 vfs.typenumhash: 1 vfs.lookup_shared: 1 vfs.usermount: 0 vfs.worklist_len: 3 vfs.timestamp_precision: 0 vfs.reassignbufcalls: 515051 vfs.vlru_allow_cache_src: 0 vfs.freevnodes: 27829 vfs.wantfreevnodes: 27833 vfs.numvnodes: 29507 vfs.ffs.doreallocblks: 1 vfs.ffs.doasyncfree: 1 vfs.ffs.compute_summary_at_mount: 0
signature.asc
Description: Message signed with OpenPGP using GPGMail
