On Sat, Feb 11, 2017 at 1:58 AM wrote:
> Are there any other methodologies where I can optimize compresssion part
and upload part
Many ISPs have upload speed several times slower than download. The
symptoms you described would be consistent with that. But maybe you have a
better, symmetric conne
On Sat, Feb 11, 2017 at 1:58 AM wrote:
> Are there any other methodologies where I can optimize compresssion part
> and upload part
>
>
Start by establishing a baseline. If you just grab the files from S3 and do
nothing to to them, how fast is that? If you are not CPU-bound, chances are
that you
Just confirming, you are uploading each file as you're downloading it. Not
downloading completely, then decompressing completely, followed by uploading.
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop rec
Hello,
I have written a GO program which downloads a 5G compressed CSV from Amazon
S3, decompresses it and uploads the decompressed CSV (20G) to Amazon S3.
Amazon S3 provides a default concurrent uploader/downloader and I am using
a multithreaded approach to download files in parallel, decompre
