vvcephei commented on pull request #9070:
URL: https://github.com/apache/kafka/pull/9070#issuecomment-663321224


   Hey all, after waiting at least 10 hours cumulatively for file uploads (so 
far) for the 2.5.1 release candidates, I've been feeling motivated to try and 
improve the situation.
   
   Currently, it takes about 2 hours to upload the release artifacts, and 
almost all of that time is just establishing an ssh connection over and over 
again. Each connection takes about 6 seconds to establish, and then a few 
milliseconds to transfer the file.
   
   I was a little surprised to see this since it looked like the code was 
supposed to be using sftp in batch mode, but was actually doing a batch of one 
command each time.
   
   I found in https://github.com/apache/kafka/pull/5374 that this was actually 
intentional, because the full batch of commands was taking too long and making 
people think the command had hung.
   
   I'm proposing to instead expand the toolchain to use `tar`, `rsync`, and 
`ssh`. These seem like reasonable requirements, and I've double-checked the 
commands on a Mac, and they seem to do the same thing as they do on Linux.
   
   I don't know how long the batch upload was taking before, but the rsyncing 
the compressed artifact is taking me about one minute. Considering the 
compilation step a little earlier "hangs" with no output for at least two 
minutes, this seems to be ok.
   
   With this change, the release script takes about five minutes to run instead 
of two hours.
   
   WDYT?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Reply via email to