Re: [S3tools-general] Much slower transfer when throttling

2009-07-12 Thread Michal Ludvig
Hi Nigel, > In my use, the 0.001 throttle > is always sufficient to make the transfer work, so I'd vote for either > making the settings you mention above the default or configurable. The right value really depends on what your connectivity to Amazon is. Whatever the defaults are there will be so

Re: [S3tools-general] Much slower transfer when throttling

2009-07-10 Thread Nigel
On Wed, Jul 8, 2009 at 6:19 PM, Michal Ludvig wrote: > Hi Nigel, > > > Perhaps it should be sleeping for .001 seconds upon first retry, instead > > of .01 seconds? Even better might be to just retry once with no sleep > > and then sleep for .001 seconds per chunk on the second retry. > > give th

Re: [S3tools-general] Much slower transfer when throttling

2009-07-08 Thread Nigel
Thanks, Michal -- I'll give this a try tomorrow! Chris On Wed, Jul 8, 2009 at 6:19 PM, Michal Ludvig wrote: > Hi Nigel, > > > Perhaps it should be sleeping for .001 seconds upon first retry, instead > > of .01 seconds? Even better might be to just retry once with no sleep > > and then sleep fo

Re: [S3tools-general] Much slower transfer when throttling

2009-07-08 Thread Michal Ludvig
Hi Nigel, > Perhaps it should be sleeping for .001 seconds upon first retry, instead > of .01 seconds? Even better might be to just retry once with no sleep > and then sleep for .001 seconds per chunk on the second retry. give the attached patch a try - it'll make the first re-try without thrott

[S3tools-general] Much slower transfer when throttling

2009-07-08 Thread Nigel
Hello, We're using s3cmd to transfer large files in 10mb chunks. About 80% of the chunks work normally, taking about 90 seconds each (~10-11 MB/s). But about 20% of the chunks fail with "(104, 'Connection reset by peer')", followed by "Retrying on lower speed (throttle=0.01)". The retry reliabl