On Aug 27, 2009, at 4:30 AM, David Bond <david.b...@tag.no> wrote:
Hi,
I was directed here after posting in CIFS discuss (as i first
thought that it could be a CIFS problem).
I posted the following in CIFS:
When using iometer from windows to the file share on opensolaris
svn101 and svn111 I get pauses every 5 seconds of around 5 seconds
(maybe a little less) where no data is transfered, when data is
transfered it is at a fair speed and gets around 1000-2000 iops with
1 thread (depending on the work type). The maximum read response
time is 200ms and the maximum write response time is 9824ms, which
is very bad, an almost 10 seconds delay in being able to send data
to the server.
This has been experienced on 2 test servers, the same servers have
also been tested with windows server 2008 and they havent shown this
problem (the share performance was slightly lower than CIFS, but it
was consistent, and the average access time and maximums were very
close.
I just noticed that if the server hasnt hit its target arc size, the
pauses are for maybe .5 seconds, but as soon as it hits its arc
target, the iops drop to around 50% of what it was and then there
are the longer pauses around 4-5 seconds. and then after every pause
the performance slows even more. So it appears it is definately
server side.
This is with 100% random io with a spread of 33% write 66% read, 2KB
blocks. over a 50GB file, no compression, and a 5.5GB target arc size.
Also I have just ran some tests with different IO patterns and 100
sequencial writes produce and consistent IO of 2100IOPS, except when
it pauses for maybe .5 seconds every 10 - 15 seconds.
100% random writes produce around 200 IOPS with a 4-6 second pause
around every 10 seconds.
100% sequencial reads produce around 3700IOPS with no pauses, just
random peaks in response time (only 16ms) after about 1 minute of
running, so nothing to complain about.
100% random reads produce around 200IOPS, with no pauses.
So it appears that writes cause a problem, what is causing these
very long write delays?
A network capture shows that the server doesnt respond to the write
from the client when these pauses occur.
Also, when using iometer, the initial file creation doesnt have and
pauses in the creation, so it might only happen when modifying files.
Any help on finding a solution to this would be really appriciated.
What version? And system configuration?
I think it might be the issue where ZFS/ARC write caches more then the
underlying storage can handle writing in a reasonable time.
There is a parameter to control how much is write cached, I believe it
is zfs_write_override.
-Ross
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
