Re: [go-nuts] Re: Get request send me hmtl content but works fine from python and curl

[Brian Candler]

My suggestion is to change the URL to something you control, with http not 
https, and see how the requests differ using tcpdump.  Here's what I get:

---- python3 ----
import requests

r=requests.get("http://localhost/foo";, params = {'version':'5', 
"phoneList":"XXXXXX", "output":"json"}, 
headers={"loginId":"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX  "})
print(r.status_code)
print(r.text)

-->
GET /foo?version=5&phoneList=XXXXXX&output=json HTTP/1.1
Host: localhost
User-Agent: python-requests/2.24.0
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
loginId: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

---- go 1.14.6 ----

Code as per original posting, but with URL changed to http://localhost/foo

-->
GET /foo HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Loginid: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Accept-Encoding: gzip

So the first problem is that the URL query parameters are not getting 
encoded.

Uncommenting the line "req.URL.RawQuery = q.Encode()", I get:

-->
GET /foo?output=json&phoneList=XXXXXX&version=5 HTTP/1.1
Host: localhost
User-Agent: Go-http-client/1.1
Loginid: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Accept-Encoding: gzip

That looks reasonable to me.  Now change the URL to 
"https://www.dncscrub.com/app/main/rpc/scrub";  (at this point I cannot 
tcpdump the packets of course, but just show the application response)

- with go, the response is a 200 OK with HTML containing login information
- with python, I get exactly the same

Now changing to https://www.lrn.com 

- with go the response is a 200 and a huge HTML/CSS page
- with python I get exactly the same

$ python3 test3.py | wc
   1288   12333  241224
$ go run . | wc
    1289   12336  241286

So I'm afraid I cannot see any difference between go and python, whereas 
you said "responses are totally different from two languages".  If they are 
still different for you, this is weird.

For completeness, here are the last two programs I ran.

---- python3 ----
import requests

r=requests.get("https://www.lrn.com";, params = {'version':'5', 
"phoneList":"XXXXXX", "output":"json"}, 
headers={"loginId":"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX  "})
print(r.status_code)
print(r.text)

---- go ----
package main

import (
        "fmt"
        "io/ioutil"
        "net/http"
        "os"
)

func main() {

        client := &http.Client{}
        //req, err := http.NewRequest("GET", "http://localhost/foo";, nil)
        req, err := http.NewRequest("GET", "https://www.lrn.com";, nil)
        if err != nil {
                os.Exit(1)
        }

        q := req.URL.Query()
        q.Add("phoneList", "XXXXXX")
        q.Add("output", "json")
        q.Add("version", "5")
        req.URL.RawQuery = q.Encode()
        req.Header.Set("loginId", "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX")

        fmt.Println(req.URL.String())

        resp, err := client.Do(req)

        if err != nil {
                fmt.Println("Errored when sending request to the server")
                return
        }

        defer resp.Body.Close()
        resp_body, _ := ioutil.ReadAll(resp.Body)

        fmt.Println(resp.Status)
        fmt.Println(string(resp_body))
}

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9f34c72e-4186-48fa-b8de-d4165fd59059o%40googlegroups.com.