Without attachment since 40K is max.
Looks like 194MB/s is max I can write on zpool on controller c6
iostat -xnMCez 10 10
extended device statistics ---- errors ---
r/s w/s Mr/s Mw/s wait actv wsvc_t asvc_t %w %b s/w h/w
trn tot device
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.3 0 0 1 0 0 1 c0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.3 0 0 1 0
0 1 c0t0d0
1.1 4.4 0.0 0.3 0.0 0.5 0.1 87.0 0 2 0 0 0 0 c1
0.5 2.0 0.0 0.1 0.0 0.2 0.1 88.3 0 1 0 0
0 0 c1t0d0
0.5 2.4 0.0 0.1 0.0 0.3 0.0 85.8 0 1 0 0
0 0 c1t1d0
328.3 349.8 12.0 14.4 0.0 33.3 0.0 49.1 0 350 86 0 0 86 c6
[...9 more instances..]
cat iostat.txt | grep -w c6
0.1 1877.8 0.0 190.5 0.0 682.5 0.0 363.5 0 3677 86 0
0 86 c6
0.0 1453.5 0.0 132.6 0.0 509.2 0.0 350.3 0 2562 86 0
0 86 c6
0.7 1817.3 0.0 189.9 0.0 907.4 0.0 499.1 0 3643 86 0
0 86 c6
0.0 1483.8 0.0 138.0 0.0 514.9 0.0 347.0 0 2686 86 0
0 86 c6
0.8 1467.2 0.0 138.5 0.0 669.9 0.0 456.3 0 2672 86 0
0 86 c6
0.4 1812.5 0.0 193.3 0.0 979.5 0.0 540.3 0 3735 86 0
0 86 c6
0.5 1487.6 0.0 140.2 0.0 506.2 0.0 340.2 0 2747 86 0
0 86 c6
0.4 1480.1 0.0 140.7 0.0 742.1 0.0 501.3 0 2718 86 0
0 86 c6
0.3 1862.8 0.0 194.2 0.0 882.5 0.0 473.6 0 3755 86 0
0 86 c6
---------- Forwarded message ----------
From: Asif Iqbal <[EMAIL PROTECTED]>
Date: Nov 17, 2007 5:00 PM
Subject: Re: [perf-discuss] zpool io to 6140 is really slow
To: adrian cockcroft <[EMAIL PROTECTED]>
Cc: [EMAIL PROTECTED], perf-discuss@opensolaris.org,
[EMAIL PROTECTED]
Looks like max IO write I get is ~ 194MB/s on controller c6, where the
zpool is on
On Nov 17, 2007 3:29 PM, adrian cockcroft <[EMAIL PROTECTED]> wrote:
> What do you get from iostat? Try something like
>
> % iostat -xnMCez 10 10
>
> (extended, named, Mbyte, controller, errors, nonzero, interval 10
> secs, 10 measurements)
>
> Post the results and you may get more commentary...
>
> Adrian
>
>
> On 11/17/07, Asif Iqbal <[EMAIL PROTECTED]> wrote:
> > (Including storage-discuss)
> >
> > I have 6 6140s with 96 disks. Out of which 64 of them are Seagate
> > ST3300007FC (300GB - 10000 RPM FC-AL)
> >
> > I created 16k seg size raid0 luns using single fcal disks. Then
> > created a zpool with 8 4+1 raidz1 using those luns, out of single
> > disks. Also set the zfs nocache flush to `1' to
> > take advantage of the 2G NVRAM cache of the controllers.
> >
> > I am using one port per controller. Rest of them are down (not in
> > use). Each controller port
> > speed is 4Gbps.
> >
> > All luns have one controller as primary and second one as secondary
> >
> > I am getting only 125MB/s according to the zpool IO.
> >
> > I should get ~ 512MB/s per IO.
> >
> > Also is it possible to get 2GB/s IO by using the leftover ports of the
> > controllers?
> >
> > Is it also possible to get 4GB/s IO by aggregating the controllers (w/
> > 8 ports totat)?
> >
> >
> >
> > On Nov 16, 2007 5:30 PM, Asif Iqbal <[EMAIL PROTECTED]> wrote:
> > > I have the following layout
> > >
> > > A 490 with 8 1.8Ghz and 16G mem. 6 6140s with 2 FC controllers using
> > > A1 anfd B1 controller port 4Gbps speed.
> > > Each controller has 2G NVRAM
> > >
> > > On 6140s I setup raid0 lun per SAS disks with 16K segment size.
> > >
> > > On 490 I created a zpool with 8 4+1 raidz1s
> > >
> > > I am getting zpool IO of only 125MB/s with zfs:zfs_nocacheflush = 1 in
> > > /etc/system
> > >
> > > Is there a way I can improve the performance. I like to get 1GB/sec IO.
> > >
> > > Currently each lun is setup as primary A1 and secondary B1 or vice versa
> > >
> > > I also have write cache eanble according to CAM
> > >
> > > --
> > > Asif Iqbal
> > > PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
> > >
> >
> >
> >
> > --
> > Asif Iqbal
> > PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
> > _______________________________________________
> > perf-discuss mailing list
> > perf-discuss@opensolaris.org
> >
>
--
Asif Iqbal
PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
--
Asif Iqbal
PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
_______________________________________________
perf-discuss mailing list
perf-discuss@opensolaris.org
