We are using the following equipment: - 12 x WD RE3 1TB SATA - 1 x lsi 1068e HBA - supermicro expander - xeon 5520 / 12gb memory
We're having very slow read performance on our san/nas. We have one raidz2 pool of 12 devices. We use the pool for iscsi ( xenserver virtual machines) + cifs share. The pool feels very unresponsive and especially the cifs share is slooooow. bonnie benchmark: Version 1.03b ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP san 25000M 335044 49 71691 16 128556 13 204.9 1 Is it true that a raidz2 pool has a read capacity equal to the slowest disk's IOPs per second ?? A 128KB block in a 12-wide raidz2 vdev will be split into 128/(12-2) = 12.8KB chunks and this affects performance? What bothers me is iostat, asvc_t shows most of the time values <10, but then jumps to >10 even when the disks are relative idle. extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 4.1 11.2 32.0 21.7 0.0 0.0 0.0 2.4 0 3 c0t10d0 4.1 12.4 33.7 22.9 0.0 0.0 0.0 2.6 0 3 c0t11d0 3.8 12.7 32.8 23.5 0.0 0.0 0.0 2.2 0 3 c0t12d0 4.0 12.0 33.7 22.6 0.0 0.0 0.0 2.6 0 3 c0t13d0 4.2 11.0 32.3 21.9 0.0 0.0 0.0 2.5 0 3 c0t14d0 4.2 11.6 32.8 23.3 0.0 0.0 0.0 2.2 0 3 c0t15d0 3.8 11.9 32.8 22.7 0.0 0.0 0.0 2.3 0 3 c0t16d0 4.1 11.7 33.9 23.3 0.0 0.0 0.0 2.1 0 2 c0t18d0 4.1 12.3 32.7 22.7 0.0 0.0 0.0 2.2 0 3 c0t19d0 3.8 11.1 32.2 21.6 0.0 0.0 0.0 2.3 0 3 c0t17d0 3.8 10.6 32.6 20.9 0.0 0.0 0.0 2.5 0 3 c0t21d0 4.1 11.2 34.0 21.9 0.0 0.0 0.0 2.2 0 3 c0t20d0 0.0 6.3 0.0 26.5 0.0 0.0 0.0 0.2 0 0 c1t1d0 extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 0.7 0.4 0.7 2.6 0.0 0.0 0.0 10.3 0 1 c0t10d0 0.7 0.4 0.6 2.6 0.0 0.0 0.0 12.4 0 1 c0t11d0 0.6 0.4 0.4 2.6 0.0 0.0 0.0 10.2 0 1 c0t12d0 0.4 0.4 0.3 2.6 0.0 0.0 0.0 6.5 0 0 c0t13d0 0.5 0.4 0.3 2.6 0.0 0.0 0.0 3.1 0 0 c0t14d0 0.5 0.4 0.4 2.6 0.0 0.0 0.0 10.7 0 1 c0t15d0 0.5 0.4 0.4 2.6 0.0 0.0 0.0 10.6 0 1 c0t16d0 0.5 0.4 0.5 2.6 0.0 0.0 0.0 9.1 0 0 c0t18d0 0.5 0.4 0.5 2.6 0.0 0.0 0.0 10.7 0 1 c0t19d0 0.7 0.4 0.6 2.6 0.0 0.0 0.0 12.3 0 1 c0t17d0 0.7 0.4 0.5 2.5 0.0 0.0 0.0 11.6 0 1 c0t21d0 0.7 0.4 0.6 2.6 0.0 0.0 0.0 10.2 0 1 c0t20d0 extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 0.4 0.0 0.2 0.0 0.0 0.0 0.0 18.0 0 0 c0t10d0 0.6 0.0 0.5 0.0 0.0 0.0 0.0 11.4 0 1 c0t11d0 0.6 0.0 0.6 0.0 0.0 0.0 0.0 13.4 0 0 c0t12d0 0.6 0.0 0.6 0.0 0.0 0.0 0.0 20.9 0 1 c0t13d0 0.7 0.0 0.6 0.0 0.0 0.0 0.0 16.4 0 1 c0t14d0 0.7 0.0 0.5 0.0 0.0 0.0 0.0 13.5 0 1 c0t15d0 0.7 0.0 0.5 0.0 0.0 0.0 0.0 11.1 0 1 c0t16d0 0.6 0.0 0.3 0.0 0.0 0.0 0.0 10.8 0 1 c0t18d0 0.3 0.0 0.2 0.0 0.0 0.0 0.0 16.7 0 0 c0t19d0 0.7 0.0 0.4 0.0 0.0 0.0 0.0 17.1 0 1 c0t17d0 0.6 0.0 0.5 0.0 0.0 0.0 0.0 13.5 0 1 c0t21d0 0.5 0.0 0.4 0.0 0.0 0.0 0.0 14.4 0 1 c0t20d0 -- This message posted from opensolaris.org _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss