Yeah, I'm not very comfortable with networking so it's possible I'm not using the right terms...
I want to be able to allow my users to use, with Selenium (and therefore Chrome), a proxy with authentication. The problem is that I cannot send the proxy credentials to the Chrome instance or interact with the proxy connection alert box. So basically, what I want to do, is a localhost proxy managed by my program which will receive all Chrome CONNECT requests, inject the Proxy-Authorization header to authenticate requests and send them back to the user remote proxy. I managed to get something with a TCP tunnel, I relied on the source code of io.Copy source code. I managed to intercept the request from the incoming net.Conn, decode the http request from the tcp packet, add the Proxy-Authorization at the end, re-encode the packet and finally write it to the outgoing net.Conn (to the remote proxy ) But I'm getting a 400 Server Error on the remote proxy side... You can check all of that here: https://github.com/hbollon/IGopher/tree/proxy/internal/proxy (It's WIP stuff, so it's not cleaned or optimized yet. I just pushed it so you could check it out) Hope I am clear enough ... Anyways, thanks for your help! :) Le mercredi 17 mars 2021 à 09:30:22 UTC+1, vlad...@varank.in a écrit : > I think I didn't get what you're building right. Now, it looks like, > instead of implementing a custom RR's director, you need to configure its > Transport [1], which will be aware of your auth proxy in the middle. Have a > look at net/http.Transport.Proxy field [2] for that. > > [1]: https://pkg.go.dev/net/http/httputil?utm_source=godoc#ReverseProxy > [2]: https://pkg.go.dev/net/http?utm_source=godoc#Transport > > On Tuesday, March 16, 2021 at 11:17:49 AM UTC+1 hugo....@gmail.com wrote: > >> Thank you for your advice that I applied. >> But now I have a *407 Proxy Authentication Required* error, while the >> header is added to the request... >> Here is the output: >> >> INFO[0019] Pre-Edited request: &{Method:CONNECT URL://google.com:443 >> Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 >> Header:map[Proxy-Connection:[Keep-Alive] User-Agent:[curl/7.68.0]] >> Body:<nil> GetBody:<nil> ContentLength:0 TransferEncoding:[] Close:false >> Host:google.com:443 Form:map[] PostForm:map[] MultipartForm:<nil> >> Trailer:map[] RemoteAddr:127.0.0.1:45382 RequestURI:google.com:443 >> TLS:<nil> Cancel:<nil> Response:<nil> ctx:0xc000814240} function=func1 >> line=60 >> >> INFO[0019] Edited Request: &{Method:CONNECT URL:http://51.178.xx.xx:3128/ >> Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 >> Header:map[Proxy-Authorization:[Basic <auth>] Proxy-Connection:[Keep-Alive] >> User-Agent:[curl/7.68.0]] Body:<nil> GetBody:<nil> ContentLength:0 >> TransferEncoding:[] Close:false Host:google.com:443 Form:map[] >> PostForm:map[] MultipartForm:<nil> Trailer:map[] RemoteAddr: >> 127.0.0.1:45382 RequestURI:google.com:443 TLS:<nil> Cancel:<nil> >> Response:<nil> ctx:0xc000814240} function=func1 line=69 >> >> INFO[0019] Scheme: http, Host: 51.178.xx.xx:3128, Port: 3128 >> function=func1 line=70 >> >> INFO[0019] Response: &{Status:407 Proxy Authentication Required >> StatusCode:407 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 >> Header:map[Content-Language:[en] Content-Length:[3520] >> Content-Type:[text/html;charset=utf-8] Date:[Tue, 16 Mar 2021 10:03:44 GMT] >> Mime-Version:[1.0] Server:[squid/3.5.27] Vary:[Accept-Language] Via:[1.1 >> vps799016 (squid/3.5.27)] X-Cache:[MISS from vps799xxx] >> X-Cache-Lookup:[NONE from vps799xxx:3128] >> X-Squid-Error:[ERR_CACHE_ACCESS_DENIED 0]] Body:0xc0004de180 >> ContentLength:3520 TransferEncoding:[] Close:false Uncompressed:false >> Trailer:map[] Request:0xc0005ee100 TLS:<nil>} function=PrintResponse >> line=33 >> >> >> PS: Is it possible on Google Groups to format code snippets? Markdown not >> seems to be supported >> >> Le mardi 16 mars 2021 à 09:04:31 UTC+1, vlad...@varank.in a écrit : >> >>> Hey there, >>> >>> Seems the issue hides in the chunk, where you overwrite reverse proxy's >>> "Director" method, which NewSingleHostReverseProxy creates internally. >>> Since your own director doesn't set the client request's Schema and Host, >>> you have to either do that manually or make sure you call the original >>> director. >>> >>> Try doing the following: >>> >>> proxyDirector := proxy.Director // ← keep the original director >>> d := func(req *http.Request) { >>> logrus.Infof("Pre-Edited request: %+v\n", req) >>> >>> proxyDirector(req) // ← call the original director to make sure >>> the request will go through the proxy >>> >>> // Inject proxy authentication headers to outgoing request into >>> new Header >>> basicAuth := "Basic " + >>> base64.StdEncoding.EncodeToString([]byte(remoteServerAuth)) >>> req.Header.Set("Proxy-Authorization", basicAuth) >>> logrus.Infof("Edited Request: %+v\n", req) >>> logrus.Infof("Scheme: %s, Host: %s, Port: %s\n", req.URL.Scheme, >>> req.URL.Host, req.URL.Port()) >>> } >>> proxy.Director = d >>> >>> Also, have a look at the implementation of NewSingleHostReverseProxy >>> https://go.googlesource.com/go/+/go1.16/src/net/http/httputil/reverseproxy.go#142 >>> >>> Cheers, >>> V. >>> >>> On Monday, March 15, 2021 at 11:37:51 PM UTC+1 hugo....@gmail.com wrote: >>> >>>> Hi! >>>> I'm actually building an automation tool based on Selenium with Go >>>> called IGopher and I have had a few requests to implement native proxy >>>> support. >>>> However, I am facing an issue with those with authentication... >>>> I can't send the proxy credentials to Chrome and without them it asks >>>> through an alert box for authentication that I can hardly interact with >>>> through Selenium (I'm not even sure it's possible in headless mode) . >>>> >>>> So I thought of an intermediary proxy system hosted locally by my >>>> program which will add the *Proxy-Authorization* header and transfer >>>> the request to the remote proxy: >>>> >>>> [image: IGopher_proxies.jpg] >>>> >>>> Something like this: proxy-login-automator >>>> <https://github.com/sjitech/proxy-login-automator> >>>> >>>> I'm not very familiar with proxies to be honest, but I tried this >>>> approach using *NewSingleHostReverseProxy*: >>>> ```go >>>> var ( >>>> localServerHost string >>>> remoteServerHost string >>>> remoteServerAuth string >>>> ) >>>> >>>> // ProxyConfig store all remote proxy configuration >>>> type ProxyConfig struct { >>>> IP string `yaml:"ip"` >>>> Port int `yaml:"port"` >>>> Username string `yaml:"username"` >>>> Password string `yaml:"password"` >>>> Enabled bool `yaml:"activated"` >>>> } >>>> >>>> func PrintResponse(r *http.Response) error { >>>> logrus.Infof("Response: %+v\n", r) >>>> return nil >>>> } >>>> >>>> // LaunchForwardingProxy launch forward server used to inject proxy >>>> authentication header >>>> // into outgoing requests >>>> func LaunchForwardingProxy(localPort uint16, remoteProxy ProxyConfig) >>>> error { >>>> localServerHost = fmt.Sprintf("localhost:%d", localPort) >>>> remoteServerHost = fmt.Sprintf( >>>> "http://%s:%d";, >>>> remoteProxy.IP, >>>> remoteProxy.Port, >>>> ) >>>> remoteServerAuth = fmt.Sprintf( >>>> "%s:%s", >>>> remoteProxy.Username, >>>> remoteProxy.Password, >>>> ) >>>> >>>> remote, err := url.Parse(remoteServerHost) >>>> if err != nil { >>>> panic(err) >>>> } >>>> >>>> proxy := httputil.NewSingleHostReverseProxy(remote) >>>> d := func(req *http.Request) { >>>> logrus.Infof("Pre-Edited request: %+v\n", req) >>>> // Inject proxy authentication headers to outgoing request into new >>>> Header >>>> basicAuth := "Basic " + >>>> base64.StdEncoding.EncodeToString([]byte(remoteServerAuth)) >>>> req.Header.Set("Proxy-Authorization", basicAuth) >>>> logrus.Infof("Edited Request: %+v\n", req) >>>> logrus.Infof("Scheme: %s, Host: %s, Port: %s\n", req.URL.Scheme, >>>> req.URL.Host, req.URL.Port()) >>>> } >>>> proxy.Director = d >>>> proxy.ModifyResponse = PrintResponse >>>> http.ListenAndServe(localServerHost, proxy) >>>> >>>> return nil >>>> } >>>> ``` >>>> >>>> With this code snippet, I'm able to intercept the request and update >>>> the header. >>>> However, resending the CONNECT request fails with the following output: >>>> >>>> ``` >>>> INFO[0028] Pre-Edited request: &{Method:CONNECT URL://google.com:443 >>>> Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 >>>> Header:map[Proxy-Connection:[Keep-Alive] User-Agent:[curl/7.68.0]] >>>> Body:<nil> GetBody:<nil> ContentLength:0 TransferEncoding:[] Close:false >>>> Host:google.com:443 Form:map[] PostForm:map[] MultipartForm:<nil> >>>> Trailer:map[] RemoteAddr:127.0.0.1:35610 RequestURI:google.com:443 >>>> TLS:<nil> Cancel:<nil> Response:<nil> ctx:0xc000164300} function=func1 >>>> line=59 >>>> >>>> INFO[0028] Edited Request: &{Method:CONNECT URL://google.com:443 >>>> Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 >>>> Header:map[Proxy-Authorization:[Basic <auth>] >>>> Proxy-Connection:[Keep-Alive] >>>> User-Agent:[curl/7.68.0]] Body:<nil> GetBody:<nil> ContentLength:0 >>>> TransferEncoding:[] Close:false Host:google.com:443 Form:map[] >>>> PostForm:map[] MultipartForm:<nil> Trailer:map[] RemoteAddr: >>>> 127.0.0.1:35610 RequestURI:google.com:443 TLS:<nil> Cancel:<nil> >>>> Response:<nil> ctx:0xc000164300} function=func1 line=63 >>>> >>>> INFO[0028] Scheme: , Host: google.com:443, Port: 443 >>>> function=func1 line=64 >>>> >>>> *2021/03/15 21:35:11 http: proxy error: unsupported protocol scheme ""* >>>> ``` >>>> >>>> What am I doing wrong? Is there a way to send a CONNECT request without >>>> scheme in Go? >>>> Maybe I'm doing something wrong or my approach to this problem is >>>> wrong. Are there better methods to achieve my goals? >>>> >>>> If you have an idea to complete what I did or any other method, please >>>> let me know! :) >>>> >>>> You can find all IGopher sources here: GitHub repository >>>> <https://github.com/hbollon/IGopher> (Proxy stuff excluded) >>>> >>> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/83ed087c-120f-4b9e-ba2d-b9e181c11398n%40googlegroups.com.
