scsi vs ide performance on fsync's
We're doing some mysql benchmarking. For some reason it seems that ide
drives are currently beating a scsi raid array and it seems to be related
to fsync's. Bonnie stats show the scsi array to blow away ide as
expected, but mysql tests still have the idea beating on plain insert
speeds. Can anyone explain how this is possible, or perhaps explain how
our testing may be flawed?
Here's the bonnie stats:
IDE Drive:
Version 1.00g --Sequential Output-- --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
jeremy 300M 9026 94 17524 12 8173 9 7269 83 23678 7 102.9 0
--Sequential Create-- Random Create
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 469 98 1476 98 16855 89 459 98 7132 99 688 25
SCSI Array:
Version 1.00g --Sequential Output-- --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
orville300M 8433 100 134143 99 127982 99 8016 100 374457 99 1583.4 6
--Sequential Create-- Random Create
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 503 13 + +++ 538 13 490 13 + +++ 428 11
So...obviously from bonnie stats, the scsi array blows away the ide...but
using the attached c program, here's what we get for fsync stats using the
little c program I've attached:
IDE Drive:
jeremy:~# time ./xlog file.out fsync
real0m1.850s
user0m0.000s
sys 0m0.220s
SCSI Array:
[root@orville mysql_data]# time /root/xlog file.out fsync
real0m23.586s
user0m0.010s
sys 0m0.110s
I would appreciate any help understand what I'm seeing here and any
suggestions on how to improve the performance.
The SCSI adapter on the raid array is an Adaptec 39160, the raid
controller is a CMD-7040. Kernel 2.4.0 using XFS for the filesystem on
the raid array, kernel 2.2.18 on ext2 on the IDE drive. The filesystem is
not the problem, as I get almost the exact same results running this on
ext2 on the raid array.
Thanks
-jeremy
--
this is my sig.
#include
#include
#include
#include
#include
struct Entry
{
int count;
char string[50];
};
int main(int argc, char **argv)
{
int fd;
struct Entry *trans;
int x;
if((fd = creat(argv[1], 0666)) == -1)
{
printf("Could not open file %s\n", argv[1]);
return 1;
}
for(x=0; x < 2000; ++x)
{
trans = malloc(sizeof(struct Entry));
trans->count = x;
strcpy(trans->string, "Blah Blah Blah Blah Blah Blah Blah");
if(strcmp(argv[2],"fsync")== 0)
{
write(fd, (char *)trans, sizeof(struct Entry));
if(fdatasync(fd) != 0)
{
perror("Error");
}
}
else
{
write(fd, (char *)trans, sizeof(struct Entry));
}
free(trans);
}
close(fd);
}
Re: scsi vs ide performance on fsync's
On Fri, 2 Mar 2001, Steve Lord wrote: > > > > > > On Friday, March 02, 2001 12:39:01 PM -0600 Steve Lord <[EMAIL PROTECTED]> wrote: > > > > [ file_fsync syncs all dirty buffers on the FS ] > > > > > > So it looks like fsync is going to cost more for bigger devices. Given the > > > O_SYNC changes Stephen Tweedie did, couldnt fsync look more like this: > > > > > >down(&inode->i_sem); > > > filemap_fdatasync(ip->i_mapping); > > > fsync_inode_buffers(ip); > > > filemap_fdatawait(ip->i_mapping); > > >up(&inode->i_sem); > > > > > > > reiserfs might need to trigger a commit on fsync, so the fs specific fsync > > op needs to be called. But, you should not need to call file_fsync in the > > XFS fsync call (check out ext2's) > > > Right, this was just a generic example, the fsync_inode_buffers would be in > the filesystem specific fsync callout - this was more of a logical > example of what ext2 could do. XFS does completely different stuff in there > anyway. > > > > > For why ide is beating scsi in this benchmark...make sure tagged queueing > > is on (or increase the queue length?). For the xlog.c test posted, I would > > expect scsi to get faster than ide as the size of the write increases. > > I think the issue is the call being used now is going to get slower the > larger the device is, just from the point of view of how many buffers it > has to scan. Well, I tried making the device smaller, creating just a 9gig partition on the raid array and this made no different in the xlog results. -jeremy > > > > -chris > > Steve > > > -- this is my sig. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: scsi vs ide performance on fsync's
On 2 Mar 2001, Linus Torvalds wrote: > In article <[EMAIL PROTECTED]>, > Jeremy Hansen <[EMAIL PROTECTED]> wrote: > > > >The SCSI adapter on the raid array is an Adaptec 39160, the raid > >controller is a CMD-7040. Kernel 2.4.0 using XFS for the filesystem on > >the raid array, kernel 2.2.18 on ext2 on the IDE drive. The filesystem is > >not the problem, as I get almost the exact same results running this on > >ext2 on the raid array. > > Did you try a 2.4.x kernel on both? Finally got around to working on this. Right now I'm running 2.4.2-ac11 on both machines and getting the same results: SCSI: [root@orville /root]# time /root/xlog file.out fsync real0m21.266s user0m0.000s sys 0m0.310s IDE: [root@kahlbi /root]# time /root/xlog file.out fsync real0m8.928s user0m0.000s sys 0m6.700s This behavior has been noticed by others, so I'm hoping I'm not just crazy or that my test is somehow flawed. We're using MySQL with Berkeley DB for transaction log support. It was really confusing when a simple ide workstation was out performing our Ultra160 raid array. Thanks -jeremy > 2.4.0 has a bad elevator, which may show problems, so please check 2.4.2 > if the numbers change. Also, "fsync()" is very different indeed on 2.2.x > and 2.4.x, and I would not be 100% surprised if your IDE drive does > asynchronous write caching and your RAID does not... That would not show > up in bonnie. > > Also note how your bonnie file remove numbers for IDE seem to be much > better than for your RAID array, so it is not impossible that your RAID > unit just has a _huge_ setup overhead but good throughput, and that the > IDE numbers are better simply because your IDE setup is much lower > latency. Never mistake throughput for _speed_. > > Linus > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to [EMAIL PROTECTED] > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > -- this is my sig. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: scsi vs ide performance on fsync's
Ahh, now we're getting somewhere. IDE: jeremy:~# time ./xlog file.out fsync real0m33.739s user0m0.010s sys 0m0.120s so now this corresponds to the performance we're seeing on SCSI. So I guess what I'm wondering now is can or should anything be done about this on the SCSI side? Thanks -jeremy On Tue, 6 Mar 2001, Mike Black wrote: > Write caching is the culprit for the performance diff: > > On IDE: > time xlog /blah.dat fsync > 0.000u 0.190s 0:01.72 11.0% 0+0k 0+0io 91pf+0w > # hdparm -W 0 /dev/hda > > /dev/hda: > setting drive write-caching to 0 (off) > # time xlog /blah.dat fsync > 0.000u 0.220s 0:50.60 0.4% 0+0k 0+0io 91pf+0w > # hdparm -W 1 /dev/hda > > /dev/hda: > setting drive write-caching to 1 (on) > # time xlog /blah.dat fsync > 0.010u 0.230s 0:01.88 12.7% 0+0k 0+0io 91pf+0w > > On my SCSI setup: > # time xlog /usr5/blah.dat fsync > 0.020u 0.230s 0:30.48 0.8% 0+0k 0+0io 91pf+0w > > > > Michael D. Black Principal Engineer > [EMAIL PROTECTED] 321-676-2923,x203 > http://www.csihq.com Computer Science Innovations > http://www.csihq.com/~mike My home page > FAX 321-676-2355 > - Original Message - > From: "Andre Hedrick" <[EMAIL PROTECTED]> > To: "Linus Torvalds" <[EMAIL PROTECTED]> > Cc: "Douglas Gilbert" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]> > Sent: Tuesday, March 06, 2001 2:12 AM > Subject: Re: scsi vs ide performance on fsync's > > > On Mon, 5 Mar 2001, Linus Torvalds wrote: > > > Well, it's fairly hard for the kernel to do much about that - it's almost > > certainly just IDE doing write buffering on the disk itself. No OS > > involved. > > I am pushing for WC to be defaulted in the off state, but as you know I > have a bigger fight than caching on my hands... > > > I don't know if there is any way to turn of a write buffer on an IDE disk. > > You want a forced set of commands to kill caching at init? > > Andre Hedrick > Linux ATA Development > ASL Kernel Development > > - > ASL, Inc. Toll free: 1-877-ASL-3535 > 1757 Houret Court Fax: 1-408-941-2071 > Milpitas, CA 95035Web: www.aslab.com > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to [EMAIL PROTECTED] > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to [EMAIL PROTECTED] > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > -- this is my sig. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
anyone international patches merged with rh kernel rpm?
I'm looking for anyone that has merged the international crypto patches with Red Hat kernel rpm's. I've basically gotten the patch to apply cleanly, but then the build completely fails during module building. I don't have the know how to fix things. Thanks for any information. I'm currently using H.J. Lu's rpm's from: http://ftp.valinux.com/pub/support/hjl/redhat/6.2/SRPMS/ Thanks -jeremy - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: scsi vs ide performance on fsync's
So in the meantime as this gets worked out on a lower level, we've decided to take the fsync() out of berkeley db for mysql transaction logs and mount the filesystem -o sync. Can anyone perhaps tell me why this may be a bad idea? Thanks -jeremy On Tue, 6 Mar 2001, Jeremy Hansen wrote: > > Ahh, now we're getting somewhere. > > IDE: > > jeremy:~# time ./xlog file.out fsync > > real0m33.739s > user0m0.010s > sys 0m0.120s > > > so now this corresponds to the performance we're seeing on SCSI. > > So I guess what I'm wondering now is can or should anything be done about > this on the SCSI side? > > Thanks > -jeremy > > On Tue, 6 Mar 2001, Mike Black wrote: > > > Write caching is the culprit for the performance diff: > > > > On IDE: > > time xlog /blah.dat fsync > > 0.000u 0.190s 0:01.72 11.0% 0+0k 0+0io 91pf+0w > > # hdparm -W 0 /dev/hda > > > > /dev/hda: > > setting drive write-caching to 0 (off) > > # time xlog /blah.dat fsync > > 0.000u 0.220s 0:50.60 0.4% 0+0k 0+0io 91pf+0w > > # hdparm -W 1 /dev/hda > > > > /dev/hda: > > setting drive write-caching to 1 (on) > > # time xlog /blah.dat fsync > > 0.010u 0.230s 0:01.88 12.7% 0+0k 0+0io 91pf+0w > > > > On my SCSI setup: > > # time xlog /usr5/blah.dat fsync > > 0.020u 0.230s 0:30.48 0.8% 0+0k 0+0io 91pf+0w > > > > > > > > Michael D. Black Principal Engineer > > [EMAIL PROTECTED] 321-676-2923,x203 > > http://www.csihq.com Computer Science Innovations > > http://www.csihq.com/~mike My home page > > FAX 321-676-2355 > > - Original Message - > > From: "Andre Hedrick" <[EMAIL PROTECTED]> > > To: "Linus Torvalds" <[EMAIL PROTECTED]> > > Cc: "Douglas Gilbert" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]> > > Sent: Tuesday, March 06, 2001 2:12 AM > > Subject: Re: scsi vs ide performance on fsync's > > > > > > On Mon, 5 Mar 2001, Linus Torvalds wrote: > > > > > Well, it's fairly hard for the kernel to do much about that - it's almost > > > certainly just IDE doing write buffering on the disk itself. No OS > > > involved. > > > > I am pushing for WC to be defaulted in the off state, but as you know I > > have a bigger fight than caching on my hands... > > > > > I don't know if there is any way to turn of a write buffer on an IDE disk. > > > > You want a forced set of commands to kill caching at init? > > > > Andre Hedrick > > Linux ATA Development > > ASL Kernel Development > > > > - > > ASL, Inc. Toll free: 1-877-ASL-3535 > > 1757 Houret Court Fax: 1-408-941-2071 > > Milpitas, CA 95035Web: www.aslab.com > > > > - > > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > > the body of a message to [EMAIL PROTECTED] > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > Please read the FAQ at http://www.tux.org/lkml/ > > > > - > > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > > the body of a message to [EMAIL PROTECTED] > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > Please read the FAQ at http://www.tux.org/lkml/ > > > > -- this is my sig. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
hotmail not dealing with ECN
Just curious if others have noticed that hotmail is unable to deal with ECN and wondering if this is a standard that should be encouraged, as in should I tell hotmail that perhaps they should look into supporting it, or should I not waste my breath and echo 0 > /proc/sys/net/ipv4/tcp_ecn? thanks -jeremy -- this is my sig. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
hotmail can't deal with ECN
After mentioning ECN, this is the response I received from hotmail. -jeremy --- From: MSN Hotmail Support <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] Subject: RE: CST23725481ID - Ban on Ecropolis Jeremy, Some of the routers and load balencers that we use will drop any packet that contains any bits in the reserved section of TCP headers. There are firmware updates being planned by our vendor, but they are not available yet. Also, we have to wait until they are fully stable before we can use them on our site. We can't have our routers and load balancers falling over constantly under stress because of bugs in the firmware. When we can, we will update, until then, there is nothing that we can do about this. We apologize for any inconvenience. David MSN Hotmail Customer Support -- this is my sig. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: hotmail can't deal with ECN
On Thu, 25 Jan 2001, David S. Miller wrote: > > Albert D. Cahalan writes: > > How about some way to test before you do this? > > Example: an ecn.kernel.org host that replys to mail. > > "test"? I know exactly whats going to happen, and unless folks like > hotmail.com and others get their act together I'll certainly end up > removing *@*hotmail.com from the lists by the end of that day. Actually, this is a kick ass idea. How many hotmail addresses on lkml? This will definitely get the point across. They don't seem to get it from the email I received. It should have read "Oh, we know about this, but in order for us to do something, we'd actually have to do work and like upgrade things and that's a drag. We'd rather watch Benny Hill reruns and throw snot balls at our enterprise solutions all day" or something to that effect. What's funny is I spent two days thinking my segment was banned from hotmail for some totally unknown reason before the whole ECN thing popped into my head. -jeremy > That is the whole point of this experiment. > > Alan plans on doing similar things to ftp.linux.org.uk and other > machines he maintains. > > The behavior of these sites is simply intolerable, and I think > this is a wonderful way to get our point across. I cannot see it > being argued that these entities have not been given enough notice > of the problem. If they cannot be bothered to get fixed an issue like > this after nearly half a year, I cannot be bothered to feel bad for > them when all users at their site lose access to the lists. > > Later, > David S. Miller > [EMAIL PROTECTED] > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to [EMAIL PROTECTED] > Please read the FAQ at http://www.tux.org/lkml/ > -- this is my sig. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
