Seeing this issue with golang 1.8.3 in Kubernetes but without a reverse proxy. Just KubeDNS -> Kubernetes service IP -> other Golang service.
When compiled with go 1.7 we saw "dial tcp: no suitable address found" errors and so after looking at this issue <https://github.com/golang/go/issues/16739> upgraded to go 1.8.3 hoping it would fix it. Instead it seems to have just been replaced with this error. dial tcp: lookup other-service.default.svc.cluster.local on 10.244.0.10:53: dial udp 10.244.0.10:53: operation was canceled On Tuesday, December 20, 2016 at 3:55:17 PM UTC-8, Matt Duch wrote: > > I'm running into issues when using a ReverseProxy that sits in front of a > service that serves static asset files (in this case, 100's in a few > seconds). Occasionally (1 out ~500 files) will error out with: > http: proxy error: dial tcp: lookup myapp.default.svc.cluster.local on > 172.18.0.16:53: dial udp 172.18.0.16:53: operation was canceled > That error occurs within a second of the request being sent, so it > shouldn't trigger any timeouts. > > When using a default ReverseProxy initialized using: > proxy := httputil.NewSingleHostReverseProxy(&url.URL{Scheme: scheme,Host: > host}) > > In a configuration like: > browser -> proxy -> golang http service > > Where the proxy and http service (go 1.7.4) are running in docker > containers (using kubernetes), and the DNS resolver is kube-dns (kubernetes > 1.4). > Changing the proxy to use a non-default transport (identical to > http.DefaultTransport) but changing DialContext -> Dial: > > v.Proxy.Transport = &http.Transport{ > > Proxy: http.ProxyFromEnvironment, > > Dial: (&net.Dialer{ > > Timeout: 30 * time.Second, > > KeepAlive: 30 * time.Second, > > }).Dial, > > MaxIdleConns: 100, > > IdleConnTimeout: 90 * time.Second, > > TLSHandshakeTimeout: 10 * time.Second, > > ExpectContinueTimeout: 1 * time.Second, > > } > > solves the problem. > > There are no errors in dnsmasq and kube-dns (the containers that provide > DNS information). > I've stress tested the DNS setup (hundreds of thousands of requests, with > one hundred concurrently) without issues (from a go binary). > I've hit the backend service directly with 1000 concurrent requests, no > issues (from a go binary as well). > If I switch from using a hostname in NewSingleHostReverseProxy to the IP > address of the backend service, the problem is also solved. > > This seems to be a unique to named hosts + DialContext + ReverseProxy. Any > ideas as to what might be happening? > -- id: 7898659753248090 -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
