Today I evaluated the various ways I could download a file efficiently. So I played with <get>, using both the HTTP and FILE protocol, and with <copy> (see build file below). The file I'm downloading is 38,448,695 bytes long. The output of running each target twice is also shown below.
So what is this all about? Well, the HTTP protocol is slower than the FILE protocol, but I expected that, because the HTTP server is serving files that reside on another NetApps filer machine (used to be faster when serving from its own disks). Note though that even though the file: protocol is faster, is not "smart" as implemented by <get>, i.e. it always redownloads, even when the local copy is up-to-date. (note that the bigshot HTTP server is serving the same file from afiler1). I then tried out copy, which I know to be smart, and expected to be as fast as <get> with the file: protocol. But to my surprise, it's not at all, and is in fact more than TWICE slower!!! A quick look at the code reveals that <get> uses a 100*1024 bytes buffer, when <copy> uses a 8*1024 bytes buffer. This seems to have quite an impact on download performance!!! Given this information, I'd like: 1) <get> to be made "smarter" for the file protocol 2) <copy> to be made faster by adding the ability to control the buffer size used, as was recently added to <checksum> I believe. In my particular case, I'd prefer (1), to be able to switch easily from http: to file: without having to change my build files (just the URL prefix used). I'll probably try it myself soon. (2) would be good too, and is probably simple to implement. Just a head's up. I don't recall anybody noticing something like this. Cheers, --DD P:\com_lgc\buildtools>build -f get.xml get.http Buildfile: get.xml get.http: [GET] Getting: http://bigshot.ad.lgc.com/dsbuilds/dsinfra/HEAD/latest/module-dsinfra.zip BUILD SUCCESSFUL Total time: 1 minute 7 seconds P:\com_lgc\buildtools>build -f get.xml get.http Buildfile: get.xml get.http: [GET] Getting: http://bigshot.ad.lgc.com/dsbuilds/dsinfra/HEAD/latest/module-dsinfra.zip Not modified - so not downloaded BUILD SUCCESSFUL Total time: 1 second P:\com_lgc\buildtools>build -f get.xml get.file Buildfile: get.xml get.file: [GET] Getting: file://afiler1.ad.lgc.com/dsbuilds/dsinfra/HEAD/latest/module-dsinfra.zip BUILD SUCCESSFUL Total time: 25 seconds P:\com_lgc\buildtools>build -f get.xml get.file Buildfile: get.xml get.file: [GET] Getting: file://afiler1.ad.lgc.com/dsbuilds/dsinfra/HEAD/latest/module-dsinfra.zip BUILD SUCCESSFUL Total time: 25 seconds P:\com_lgc\buildtools>build -f get.xml copy.file Buildfile: get.xml copy.file: [GET] Copying 1 file to P:\com_lgc\buildtools BUILD SUCCESSFUL Total time: 1 minute 8 seconds P:\com_lgc\buildtools>build -f get.xml copy.file Buildfile: get.xml BUILD SUCCESSFUL Total time: 1 second <?xml version="1.0"?> <project name="GET" default="help"> <target name="help" description="Display project description and targets"> <echo>=== ${ant.project.name} Build ===</echo> <java classname="org.apache.tools.ant.Main"> <arg value="-emacs" /> <arg value="-projecthelp" /> <arg value="-buildfile" /> <arg value="${ant.file}" /> </java> </target> <property name="dsinfra/" value="dsinfra/HEAD/latest/module-dsinfra.zip" /> <property name="dsinfra\" value="dsinfra\HEAD\latest\module-dsinfra.zip" /> <target name="get.http" description="Get dsinfra.zip, HTTP protocol"> <get src="http://bigshot.ad.lgc.com/dsbuilds/${dsinfra/}"; dest="dsinfra-http.zip" useTimeStamp="true" /> </target> <target name="get.file" description="Get dsinfra.zip, FILE protocol"> <get src="file:////afiler1.ad.lgc.com/dsbuilds/${dsinfra/}" dest="dsinfra-file.zip" useTimeStamp="true" /> </target> <target name="copy.file" description="Copy dsinfra.zip"> <copy file="\\afiler1.ad.lgc.com\dsbuilds\${dsinfra\}" tofile="dsinfra-copy.zip" preserveLastModified="true" /> </target> </project> --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
