[perf-discuss] Latest release of Filebench.

FileBenchers, I uploaded version 1.4.8 of FileBench to sourceforge a couple of weeks ago, but I forgot to send out a notice. So, here it is. Sadly, this is the last work I will be doing on FileBench, at least for quite a while, as it is my last day at Sun. Hopefully others will carry on

Re: [perf-discuss] filebench randomwrite not scaling with multiple threads?

you using? Drew On 06/09/09 15:39, John Fitzgerald wrote: On 06/09/09 14:55, Andrew Wilson wrote: John, Where is the file you are writing to located? A single stream of 512KB will saturate most disk drives. Even if this is just tmpfs on the server (as the name you picked implies), you might

Re: [perf-discuss] filebench randomwrite not scaling with multiple threads?

John, Where is the file you are writing to located? A single stream of 512KB will saturate most disk drives. Even if this is just tmpfs on the server (as the name you picked implies), you might still be saturating your Ethernet with just a single thread. Reads may be getting local ca

Re: [perf-discuss] Trouble compiling filebench 1.4.7 on RHEL 5.2

Sorry, my mistake. The config.h file shouldn't be there at all. It was left over from the opensolaris version. With the SourceForge version config.h is generated by the "configure" script. Just remove it before running "configure". This may not solve all the problems, but it might reduce them

Re: [perf-discuss] Trouble compiling filebench 1.4.7 on RHEL 5.2

Demetri, The top level include file, filebench.h, does have a typedef for boolean_t which is supposed to be invoked if autoconfig determines that boolean_t is otherwise undefined. So, not sure why it's not being invoked. The lines in question are: #ifndef HAVE_BOOLEAN_T typedef enum { B_FAL

[perf-discuss] New version of FileBench on OpenSolaris and Sourceforge

FileBench users: Version 1.4.7 has now been pushed to the OpenSolaris gate, and also uploaded to SourceForge. Features and fixes since the last SourceForge update (v1.4.4) include: * Ability to open files within filesets in read only mode * Some sanity checks so that if you specify both an

Re: [perf-discuss] zfs filebench: cached workload versus uncached workload

ether the file cache starts out "hot", with portions of the files that were just created, or "cold". Drew On Apr 28, 2009, at 2:22 PM, johan...@sun.com wrote: On Tue, Apr 28, 2009 at 12:58:25PM -0700, Andrew Wilson wrote: I believe that Solaris always converts reads and wri

Re: [perf-discuss] zfs filebench: cached workload versus uncached workload

Since zfs uses a private cache (the ARC) for most of its file caching, rather than VM pages, the "caching" or not doesn't make that much difference. On the other hand, ufs does use VM pages, so the caching options makes quite a bit of difference. The fs_flush script is used to flush the ZFS

Re: [perf-discuss] New release of FIlebench on SourceForge

On 04/24/09 05:52 AM, Dearl Neal wrote: Hey Drew.. Thanks for the update on filebench. I have dowloaded and compiled it for use on a sun sparc system. One thing I have noticed about using this tool is that it does not clean up the ipc shared memory segements, active semaphores, or active message

Re: [perf-discuss] New release of FIlebench on SourceForge

There is code to clean those up, though it doesn't seem to always get run, especially if you quit early. But normally FB does clean all those things up. I suppose it is possible that my new shutdown code broke the cleanup. I'll investigate to see if I have the same cleanup failures you are

Re: [perf-discuss] filebench batch mode problem

Ted, The thisrun.f file looks fine. It is not ignoring runtime, as it is putting in a "sleep 60", which does causes it to run for 60 seconds. It sounds like go_filebench isn't running at all. You might try running thisrun.f directly from the shell: $thisrun.f See what happens. You do have

Re: [perf-discuss] Problem compiling filebench 1.4.4 on Sol 10 with Sun Studio 12

Frank, I looked at what my lex compiler is producing, and the parser_lex.c file is about 4000 lines. Note that I am NOT using Sun Studio 12. If I am interpreting the error message you are seeing correctly, it looks like your parser_lex.c file is ending after only 212 lines, hence the unexpe

Re: [perf-discuss] Problem compiling filebench 1.4.4 on Sol 10 with Sun Studio 12

ync as well on sf.net: http://sourceforge.net/project/stats/detail.php?group_id=133644&ugn=filebench&type=cvs&mode=12months On Mar 25, 2009, at 9:55 AM, Andrew Wilson wrote: Frank, I can't see anything wrong with the parser_lex.l code, however there are a couple of extra semic

Re: [perf-discuss] Problem compiling filebench 1.4.4 on Sol 10 with Sun Studio 12

Frank, I can't see anything wrong with the parser_lex.l code, however there are a couple of extra semicolons in parser_gram.y that were reported a few days ago that might be the source of the problem. They have actually been there for a while, but are not caught by the regular Solaris compiler

Re: [perf-discuss] New release of FIlebench on SourceForge

Make that: "remove the trailing ';' from lines 1447 and 1527" in the previous message. Lines 1448 and 1528 do need their trailing ';'. Drew On 03/20/09 06:45 AM, Tim Becker wrote: I'm attempting to compile filebench 1.4.4 on sparc with SunStudio 12 and am getting the following error cc: Warni

Re: [perf-discuss] New release of FIlebench on SourceForge

It appears my compiler didn't catch that bug. Remove the ';' from the end of line 1448 and 1528 in parser_gram.y. I forgot to remove them when I added another line to the expression. The trailing semicolon is supposed to only occur at the end of the last line in both cases. I'll fix that in the

[perf-discuss] New release of FIlebench on SourceForge

FileBench followers, I finally figured out all the nuts and bolts of getting new versions of FileBench into the build environment its SourceForge releases use. So, I have just put up a "Catch up" Release of FileBench v1.4.4, which includes the following: - Multi-client support - A Video Se

[perf-discuss] Code Review Request for FileBench

FileBench fans, This change cleans up 4 feature requests that have been hanging around for a while: 6584939 filebench should be smarter about how its opening files 6685196 need stricter

Re: [perf-discuss] filebench bug

n the flush script will FileBench from exporting / importing the pools. Why don't you try that and tell me if FileBench continues to fail. If it does fail, could you check to see if you can ls /testpool? Drew On Mar 12, 2009, at 4:24 PM, Asif Iqbal wrote: On Thu, Mar 12, 2009 at 5:09

Re: [perf-discuss] filebench bug

from doing this by changing the fs_flush script. Drew On 03/12/09 02:23 PM, Asif Iqbal wrote: On Thu, Mar 12, 2009 at 5:09 PM, Andrew Wilson wrote: First off, it is not a bug, it is a feature that is necessary to flush the zfs ARC. Otherwise you get rather excessive performance. This is done

Re: [perf-discuss] filebench bug

First off, it is not a bug, it is a feature that is necessary to flush the zfs ARC. Otherwise you get rather excessive performance. This is done through a script called fs_flush, though, and it is easy to comment out that part of the script. The script is found in filebench/scripts (usr/benchma

Re: [perf-discuss] Filebench Source Code

I should have added that the binary and workload files ARE installed as part of 2008.11 They are in /usr/benchmarks/filebench It is the source that is on Open Solaris under /usr/src/cmd/filebench I think the source on Open Solaris is for one rev later than what is in 2008.11. (1.4.3 vs. 1.4.

Re: [perf-discuss] Filebench Source Code

Sorry for the SorceForge Problems. Development on Filebench definitely has not flat lined, but is being done on OpenSolaris. Unfortunately, I haven't been able to update SorceForge lately. We hope to get that fixed soon. If you want the latest Filebench, it is available from OpenSolaris and

[perf-discuss] Code review of FileBench shutdown bug fixes requested

Dear OpenSolaris performance gurus, Several bugs related to problems shutting down FileBench have been reported, including the two cited here and one more that I may add to the push. Unfortunately the provided information is a bit sparse, and I can't reproduce them on my test machines. Howev

Re: [perf-discuss] System Statistics Collection in Filebench

Saurav, There is a way to have filebench run a separate statistics collection script while it is generating load, I believe. It should also be possible to execute the collection script on the server with rsh or equivalent. However, I will have to investigate a bit on how to do it and get back

[perf-discuss] FileBench Code review requested

In preparation for developing a set of file system plug-in modules, this change set has been created which introduces an I/O functions vector which both flowop_library.c and fileset.c use to make file system calls. It creates a new header file, fsplug.h which defines the vector and macros for a

Re: [perf-discuss] Can't get FileBench foreach to work

Demetri, As I said in my previous answer, the way the rest of us do what you want to do is through .prof scripts interpreted by filebench.pl. Each entry in the profile script causes go_filebench to be run, modified with any changes specified. For example: # # CDDL HEADER START # # The conte

Re: [perf-discuss] Can't get FileBench foreach to work

Demetri, I think you may be the only person (besides myself) that has tried to use "foreach". The problem isn't the "foreach" command though, it is that "run 10" shuts down processes when it finishes and re-running from the same "f" script doesn't work. I have two suggestions: 1) create a .p

[perf-discuss] Code review request for FileBench code to implement random file access within fiilesets

This code modifies FileBench to use avl trees to manage filesetentries, allowing direct selection of existing vs. non existing files and leaf directories and random access of files and leaf directories. The use of avl trees and separate trees for existing and non-existing files significantly re

Re: [perf-discuss] filebench source hints at raw device support

Demetri, It is defined in "config.h". If you are getting the sources from the SourceForge source tree, it is very obsolete. The tar package for 1.3.4 is the latest one on SourceForge, and does support the raw devices (files). Also, you can go to opensolaris.org and download from the source t

Re: [perf-discuss] filebench source hints at raw device support

On 11/21/08 15:09, Demetri S. Mouratis wrote: > Dear filebench gang, > > fileset.c shows some evidence of a way to use filebench against individual > raw device files instead of the 0001/0001 directory structure implied > by fileset names. I've been staring at it an hour or more, and I c

Re: [perf-discuss] Filebench installation on Suse/Linux?

Bob, that is a very useful bit of information. It does sound like the makefile wasn't quite fixed up correctly for Linux. Unfortunately, I am not at all familiar with Linux, so I generally have to rely on others to fix this stuff, but I will investigate, on the off chance it is something trivia

[perf-discuss] fb_onnv_videoserv-hgchild

FileBench lovers, I just put a webrev for some changes to FileBench up on the opensolaris site: http://cr.opensolaris.org/~dreww/video_server/ This adds two features: an example video server workload, and a set of file open / close / list flowops and workloads. It also fixes one small bug

Re: [perf-discuss] Filebench:Oltp profile still hanging?

Well, 4 minutes is about 3 and 1/2 minutes longer than my goal though. I'll see if I can speed it up a bit without causing other problems. FYI, the "problem" with shooting things down too fast is that certain shared memory items get removed while some processes are still referencing them, which

Re: [perf-discuss] Filebench:Oltp profile still hanging?

Bob, The OLTP workload is the only workload that uses system V semaphores. When the runtime expires, some of the processes are blocked on semaphores and do not "see" the quit indication. Unfortunately, they actually never see it, because the processes holding the semaphores do see it and exi

[perf-discuss] Code review and comments requested on new FileBench Mulit-Client feature.

Folks interested in FileBench, I have put together an INTERIM approach to doing multi-client benchmarking with Filebench. A description of how it works and how to use it is in the following pdf: http://www.solarisinternals.com/wiki/index.php/Image:FileBench_Multi_Client.pdf There are a numb

Re: [perf-discuss] about filesize for singlestreamwrite and multistreamwrite in filebench

zhihui, Filesize is only used to determine how large to make the file initially. If you do sequential writes to it, as those two workloads do, it will continue to grow. zhihui Chen wrote: I believe we have an RFC to add an attribute such as "maxfilesize" to filesets, which will prevent th

Re: [perf-discuss] filebench issue with large # of files and paralloc

Peter, You are correct about the purpose of fileset_freemem(). However, you don't need to chop any code out, just set the "cache" attribute, which turns off the call to fileset_freemem. I admit the name isn't very intuitive, and Eric and I have discussed the need to change it to something tha

[perf-discuss] fb_onnv_cf

I have posted a webrev of changes to FileBench to implement the Composite Flowop feature. The webrev is here: http://cr.opensolaris.org/~dreww/FileBench_Composite_Flowops/ You can go to: http://www.solarisinternals.com/wiki/index.php/Filebench_News And click on "FileBench Composite Flowops" to

Re: [perf-discuss] port filebench to linux

Denis, I can answer a couple of your questions: rae l wrote: > > BTW: there is still some inconsistence through all these documentation: > http://www.solarisinternals.com/wiki/index.php/FileBench > http://www.opensolaris.org/os/community/performance/filebench/ > and some docs in the source tree;

Re: [perf-discuss] Filebench uses too many threads during fileset creation

Pavel, It was supposed to only use 32. Thanks for reporting this. I'll look into that for you and try to get it fixed. Just out of curiosity, are you running this on a machine with some sort of RAIDed disk drives. If you only have one drive the paralloc doesn't seem to really help anyway, as

[perf-discuss] fb_onnv_improv

I have been fixing several bugs and finishing one RFE for FileBench. The specific improvements are: Flowop deletefile has been enhanced to use an "fd" attribute, if supplied to locate the file to delete, rather than always picking an arbitrary one. To use this, you will need to close the file f

[perf-discuss] fb_onnv_rawbug

FileBench fans, I have uploaded a new webrev for your reviewing pleasure at: http://cr.opensolaris.org/~dreww/rawfilebug/ This fixes a collection of recently discovered bugs, including: 6673349 seg fault when variables defined after define stmt - caused by checking for raw files too early.

[perf-discuss] New FileBench proposal for Composite Flowops

I have posted a proposal for addition of a Composite Flowop feature to FileBench. See: http://www.solarisinternals.com/wiki/images/3/36/FileBench_Composite_Flowops.pdf for more information. Drew ___ perf-discuss mailing list perf-discuss@opensolari

[perf-discuss] New FileBench code review requested

FileBench Users, I have modified FileBench so that it now features a couple of "run to completion" modes of operation. This is useful for benchmarks that do a specified amount of work and then quit rather than running for a specified time. Currently with the finishonbytes and finishoncount f

Re: [perf-discuss] Filebench with LARGE fileset

Logan, Could you say how many threads you were trying to run, and what the mean directory width was that you requested? Thanks, Drew Logan wrote: > I'm doing a performance test of an nfs server using Filebench. The test is > meant to emulate the performance of a cache such as squid hitting

Re: [perf-discuss] Code Review for new FileBench feature requested

; Nice work, > Thanks! > -j > > On Tue, Jan 29, 2008 at 11:09:52AM -0800, Andrew Wilson wrote: > >> I have developed a new "random variables" feature for FileBench. A >> general discussion of the approach is available at: >> >> http://www.solari

[perf-discuss] Code Review for new FileBench feature requested

I have developed a new "random variables" feature for FileBench. A general discussion of the approach is available at: http://www.solarisinternals.com/wiki/images/1/1d/FileBench_Random_Variables.pdf See the section headed "Current Proposal and Prototype" on page 7 of the document for a descript

[perf-discuss] Proposal for FileBench Random Variables

To all those interested in FileBench, I have been working on a syntax and implementation approach for adding a new type of variable, called the Random Variable, to FileBench. A Random type Variable returns a value from a defined random distribution with each access. The proposal is at: http:/

Re: [perf-discuss] New FileBench code uploaded to SourceForge

Not sure why you can't see it. Just to be sure I tried the link as posted in the reflected message and it worked fine for me. Anyway, for anyone else having problems, go to: SolarisInternals.com Then click on "Filebench News" in the "Recently Updated" column, then on "unification of files and

[perf-discuss] New FileBench code uploaded to SourceForge

The FileBench source has been updated with fixes to several bugs, and unification of files and filesets. More info is available on SolarisInternals.com at: Unification of Files and Filesets. Happy Benchmarking! Drew ___ perf-discuss mailing list

Re: [perf-discuss] Filebench Package...

Ken,   The package that you get from SourceForge when you click on the "download Filebench" button is the latest available version. There is a link to the site from the "quick start" section of the FileBench page on the SolarisInternals Wiki, BTW. The actual SourceForge page is: FileBench Not

Re: [perf-discuss] [storage-discuss] zpool io to 6140 is really slow

situation. Drew Andrew Wilson wrote: What kind of workload are you running. If you are you doing these measurements with some sort of "write as fast as possible" microbenchmark, once the 4 GB of nvram is full, you will be limited by backend performance (FC disks and their in

Re: [perf-discuss] [storage-discuss] zpool io to 6140 is really slow

What kind of workload are you running. If you are you doing these measurements with some sort of "write as fast as possible" microbenchmark, once the 4 GB of nvram is full, you will be limited by backend performance (FC disks and their interconnect) rather than the host / controller bus. Since

[perf-discuss] Webrev posted for Filebench fixes / enhancements

FileBench affectionados, I have just uploaded a webrev of a set of bug fixes and other modifications to FileBench to the OpenSolaris.org webrev site. This set of changes started out as modification to essentially merge "files" into "filesets", but ended up addressing a number of related issues.

Re: [perf-discuss] Runtime Parameter Hard Rule or Guidance?

A very good question. I assume you are running one of the suites of workloads, such as "filemicro"? If so, some workloads will finish early, depending on kind of application the particular workload is emulating. For example, the "copyfiles" workload finishes when it has copied all the files. Th