Thanks this really is a great explanation. So new connections will be created if there are no idle connection available.
The following article suggests setting the Transport.MaxIdleConnsPerHost to the same value as Transport.MaxIdleConns to allow any host to use all connections if necessary. http://tleyden.github.io/blog/2016/11/21/tuning-the-go-http-client-library-for-load-testing/ On Tuesday, August 21, 2018 at 5:18:05 AM UTC-7, Kevin Conway wrote: > > > Are requests sent serially ... so that other concurrent requests to > the same host are blocked > > When sending requests using HTTP/1, each connection will handle only one > request at a time except when using pipelining. When sending requests using > HTTP/2, each connection may manage any number of requests. The > http.Transport does a lot to abstract this behavior so it looks the same to > a developer. As far as blocking, the http.Transport will never "block" > while waiting for a connection to return to the pool. Unlike the connection > pool in Java, for example, the http.Transport connection pool does not > limit the number of open connections. If there are no idle connections in > the pool then the http.Transport will make another one on-demand and use > that for the request. > > > forget to Close() a Response.Body > > This can definitely become a problem and is a common mistake made by new > go devs. A good answer to this question with more detail is here: > https://stackoverflow.com/a/33238755 > > > Transport.IdleConnTimeout will close any connections automatically for > me right? > > When configuring your connection pool, it is important to keep in mind > that all of the logic for removing an old connection from the pool and > closing it is based on *idle* time. For example, IdleConnTimeout of 30s > will cause connections that have gone unused for 30s to close. There is, as > of go1.10, still no way to define a maximum lifetime of a connection that > is in-use. > > > pool of clients ... to allow for parallel requests > > Each client already manages any number of connections for HTTP/1 calls and > handles multiple, concurrent HTTP/2 calls on a single connection per host. > You do not need to do anything else to get the behavior you want. > > > I could increase the Transport.MaxIdleConns or > Transport.MaxIdleConnsPerHost > > Like the connection timeout, it is important to recognize that the > connection limits are based on *idle* connections. There is no setting, as > of go1.10, that allows you to set a maximum number of active connections > for HTTP/1. The HTTP/2 support in http.Transport already has a built-in > limit because it enforces use of a single connection for all concurrent > requests so you don't have to do anything special for HTTP/2. However, > there is no limit to the number of connections http.Transport will make for > a HTTP/1 calls. > > > > On Tue, Aug 21, 2018 at 12:32 AM <golang...@gmail.com <javascript:>> > wrote: > >> Also do you have any recommendations for deriving appropriate values for >> Transport.MaxIdleConns >> or Transport.MaxIdleConnsPerHost? >> >> >> On Monday, August 20, 2018 at 12:59:07 AM UTC-7, golang...@gmail.com >> wrote: >>> >>> The http.Transport caches connections for future re-use >>> https://golang.org/pkg/net/http/#Transport >>> >>> Are requests sent serially by which I mean only one request and response >>> can be handled by this persistent connection at a time, so that other >>> concurrent requests to the same host are blocked until a response is >>> received and closed? >>> >>> If I forget to Close() a Response.Body, does that leave the connection >>> open so that it can't be shared? Do other concurrent requests open a new >>> connection during that time? >>> >>> And if I do forget, by default the Transport.IdleConnTimeout will close >>> any connections automatically for me right? >>> >>> I understand that persistent connections save on protocol overhead, but >>> if it can only handle one request at a time wouldn't in some cases it be >>> better to have a pool of clients each with their own persistent connection >>> to the same host to allow for parallel requests? >>> >>> Alternatively, I could increase the Transport.MaxIdleConns or >>> Transport.MaxIdleConnsPerHost right? Which would trade throughput for >>> server resources to maintain those connections. >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golang-nuts...@googlegroups.com <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.