Re: [S3tools-general] Patch to enable parallel get, put and sync operations

2010-10-25 Thread Michal Ludvig
Hi Philip, thanks for the contribution. Your patch is too large to be integrated just before s3cmd 1.0.0 comes out but I will definitely keep it in mind for future. I've got some big plans on parallel processing in s3cmd and parallel transfer is indeed a part of that. Other possible improvem

[S3tools-general] Patch to enable parallel get, put and sync operations

2010-08-28 Thread Philip Corliss
I've just finished testing a patch to enable the transfer operations to be threaded. This speeds up operations dramatically (4-8x in my tests), especially when you have many small files. Hopefully this can be integrated into the main release. You can find the patch at http://github.com/pcorliss/s3