Daniel Stenberg <dan...@haxx.se> 于2020年6月22日周一 上午1:29写道： > > On Sun, 21 Jun 2020, Hongyi Zhao via curl-library wrote: > > > As you can see, it seems libcurl returns the random URL for the target file. > > I would rather say it like this: > > The *web server* returns a random URL for the target file. > > > I want to know whether libcurl has the capability of obtaining all the > > potential mirror's URLs. > > libcurl is not a magician, it just delivers to the client what the server > responds. In this case it seems the server does some round robin or something > and delivers different Location: URLs for different requests. > > libcurl doesn't know anything about where the target URL may exist, only the > server knows that and it apparently delivers one of them for each request. >
OK. But I find that these random URLs have very different network performance for downloading the target file. See the following for more information: $ curl -x socks5://127.0.0.1:18887 https://mirrors.tuna.tsinghua.edu.cn/ctan/systems/texlive/Images/texlive2020-20200406.iso > /dev/null % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 5 3813M 5 199M 0 0 10.7M 0 0:05:55 0:00:18 0:05:37 13.4M^C $ curl -x socks5://127.0.0.1:18887 http://ctan.mirror.colo-serv.net/systems/texlive/Images/texlive2020-20200406.iso > /dev/null % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 3813M 0 32.3M 0 0 1120k 0 0:58:04 0:00:29 0:57:35 1173k^C So, try to pick out the high-speed mirror URL is crucial for performance. I want to do the following jobs based on libcurl's multi-threading capability: 1. During the whole downloading process, let different workers/threads try to re-establish the connection to the initial URL and get the redirected target URL. So that libcurl can find as many as possible mirror URLs. 2. Based on the speed of the different target URL, kill the low-speed workers/threads and reuse the URL which presents high-speed. Any hints for doing the trick with libcurl? -- Hongyi Zhao <hongyi.z...@gmail.com> ------------------------------------------------------------------- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
