[go-nuts] promoted methods confusion
Is there any proposal in Go 2.0 to remove the promotion of methods on embedded struct fields or do you think it's a good idea ? For example in the code below if someone marshals `Pack`, only `Animal` is actually marshalled. Cat field is skipped. If he wants to marshal Cat as well the developer must write an additional wrapper method(MarshalJSON) on `Pack` and any other struct that embeds `Animal`. This becomes an even bigger issue if you create types dynamically (using reflect). Actually I believe reflect.StructOf is actually broken because of this feature (i.e. it panics if the embedded type has methods). type Pack struct{ Animal Cat Cat } func (a *Animal)MarshalJSON()([]byte, error){ // custom animal marshalling } -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/4366cedc-5833-4bb5-add5-8d96b86a83c5n%40googlegroups.com.
[go-nuts] Can generics help me make my library safer?
I have developed a library that depends very much on reflect package. It caches a specific type and return a function that encodes(does something with) with that kind /type of data. Think of defining database schema using types and generating functions to validate/update/insert data. I reduced it to a basic example below. Can the generics feature from Go help me make it any safer? Ideally I would like to make the program below* not to compile *due the errors of invalid params passed to *PrintT* instead to throw dynamically at runtime. package main import ( "errors" "fmt" "reflect" ) type T struct { F1 string F2 int } type T2 struct { F1 float32 F2 bool } var PrintT = Function(T{}) func main() { if err := PrintT("one", 1); err != nil { fmt.Printf("err %v", err) } if err := PrintT("one", 1, "another"); err != nil { fmt.Printf("err %v", err) } if err := PrintT("one", "one"); err != nil { fmt.Printf("err %v", err) } } type ReturnFunc func(params ...interface{}) error func Function(v interface{}) ReturnFunc { var paramTypes []reflect.Type tv := reflect.TypeOf(v) for i := 0; i < tv.NumField(); i++ { paramTypes = append(paramTypes, tv.Field(i).Type) } fn := func(param ...interface{}) error { // validate input if len(param) != len(paramTypes) { return errors.New("invalid number of params passed") } for k, v := range param { if reflect.TypeOf(v) != paramTypes[k] { return errors.New("invalid type passed") } } // do something with the params fmt.Println(param...) return nil } return fn } https://go2goplay.golang.org/p/1MJynmFhq6B -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/2bc1a657-c4e0-42a1-82b0-8bc605c429a2n%40googlegroups.com.
Re: [go-nuts] Can generics help me make my library safer?
Hi Ian , I've modified the example towards a more specific use case. The main idea in the example below is to make code related to database operations(i.e SELECT queries) safer and easier to read. A kind of json.Unmarshal/Marshal for databases, with validation (type checking, param numbers etc) to avoid a class of bugs/errors such invalid param types/numbers passed, invalid queries, invalid resource type to scan into etc. Currently the function returned by *Select* throws the validation errors at runtime (i.e. invalid param type passed etc). It would be great to have that class of errors checked at compile type. The only way I could achieve that kind of type checking was through code generation. I already built a tool to generate functions with the proper param types but it seems that code generation introduces a lot of friction to the point that I stopped using it. My hope is that one day a Go feature (i.e. a version of Generics) could help the function returned by *func* *Select* be type checked at compile time. https://play.golang.org/p/-Th7aHDGORL package main import ( "database/sql" "errors" "fmt" "reflect" ) var sqlDB *sql.DB type Flower struct { Color string Size int Weight int } type T struct{} var FlowerByColor = Select(" * FROM tablex WHERE Color=$ LIMIT 1", reflect.TypeOf(Flower{})) func main() { // Select a flower based on its color from database // invalid resource type; resource of type Flower is expected // I would like this to not compile as I pass an invalid resource type T instead of type Flower var color string if err := FlowerByColor(T{}, &color); err != nil { fmt.Printf("err %v\n", err) } // invalid param; type string (Folower.Color) type is expected var colorInvalid int // I would like this to not compile as I pass an invalid color type if err := FlowerByColor(Flower{}, colorInvalid); err != nil { fmt.Printf("err %v\n", err) } // correct query color = "red" if err := FlowerByColor(Flower{}, color); err != nil { fmt.Printf("err %v\n", err) } // Note: a proper SelectFunc would actually accept only pointers to Flower so that // it can unmarshal data into the resource as below. For brevity I omitted handling // pointers in SelectFunc resource := new(Flower) if err := FlowerByColor(resource, color); err != nil { fmt.Printf("err %v\n", err) } // so something with the data from realVal fmt.Printf("our flower of color %v has a size of %v", color, realVal.Size) } // SelectFunc receives a resource and the query params type SelectFunc func(resource interface{}, params ...interface{}) error // select receives a sql query and the type that represents //the sql table from database. // Returns a function that executes the sql query with the matching params from tv. func Select(q string, tv reflect.Type) SelectFunc { paramTypes, err := parseQuery(q, tv) if err != nil { panic("invalid query") } return func(resource interface{}, param ...interface{}) error { // validate input // resource must match the resource type if reflect.TypeOf(resource) != tv { return errors.New("invalid resource type") } if len(param) != len(paramTypes) { return errors.New("invalid number of params passed") } for k, v := range param { if reflect.TypeOf(v) != paramTypes[k] { return errors.New("invalid argv type passed") } } // do a select database query resourceFields := fieldsFromResource(reflect.ValueOf(resource)) if err := sqlDB.QueryRow("SELECT "+q, param...).Scan(resourceFields...); err != nil { return err } return nil } } // parseQuery parses query and the resource t. // returns the types selected in the query func parseQuery(query string, t reflect.Type) ([]reflect.Type, error) { // skip parsing for brevity return []reflect.Type{t.Field(0).Type}, nil } func fieldsFromResource(v reflect.Value) []interface{} { // skip type fields looping for brevity return []interface{}{ v.Field(0).Addr().Interface(), v.Field(1).Addr().Interface(), v.Field(2).Addr().Interface(), } } On Wednesday, October 6, 2021 at 2:14:20 AM UTC+3 Ian Lance Taylor wrote: > On Sun, Oct 3, 2021 at 8:41 AM mi...@ubo.ro wrote: > > > > I have developed a library that depends very much on reflect package. It > caches a specific type and return a function that encodes(does something > with) with that kind /type of data. Think of defining database schema using > types and generating functions to validate/update/insert data. > > > > I reduced it to a basic example below. Can the generics feature from Go > help me make it any safer? Ideally I would like to make the program below > not to compile due the errors of invalid params passed to PrintT instead to > throw dynamically at runtime. > > I don't understand what your program is trying to do, but the curre
Re: [go-nuts] Can generics help me make my library safer?
I'm using the library with a nosql. I provided the sql example b/c it's part of the std library. I like rog's solution(especially on the generic Resource type) but to reduce friction and make it more idiomatic I would need to be able to define the select arguments in the return func (after the query string is parsed) instead to define them manually in a struct. Defining the filters/params in a struct requires extra effort and makes the code more verbose instead to rely on the query parser. Also a query has just 1-2 filters and I don't find structs with 1-2 fields(i.e like `ByColor` struct) very idiomatic/nice to use instead of function parameters. I assume that as Ian said that kind of "variadic generic types" is not possible in Go/2 generics. On Wednesday, October 6, 2021 at 4:17:43 PM UTC+3 ren...@ix.netcom.com wrote: > I think you can only do that if you make sql parsing a first class > language feature - or you need to construct the query using a syntax tree > of clauses which is a PITA. Sometimes a hybrid approach - not a full orm - > but an sql helper works best so > > sql.Query(table name, field list, where clauses, order by clauses) can > work but for complex sql like joins it becomes unwieldy. > > After years of doing both, I’ve settled on that using DAOs creates the > simplest and highest performing code. > > On Oct 6, 2021, at 8:07 AM, Brian Candler wrote: > > FWIW, I like the idea of being able to write direct SQL and still have > some static type checking. ORMs are OK for simple "get" and "put", but I > have been bitten so many times where I *know* the exact SQL I want for a > particular query, but the ORM makes it so damned hard to construct it their > way. > > > On Wednesday, 6 October 2021 at 12:45:01 UTC+1 ren...@ix.netcom.com wrote: > >> Personally, I think this is overkill (the entire concept not the rog >> solution) >> >> Even with static checking there is no way to ensure that tablex has the >> needed fields. Even if you could check this at compile time - it might be >> different at runtime. >> >> I don’t think the juice is worth the squeeze. >> >> Either use an ORM that generates the sql, or create DAOs and rely on test >> cases to ensure the code is correct. >> >> Far simpler and more robust in my opinion. >> >> On Oct 6, 2021, at 6:35 AM, roger peppe wrote: >> >> >> >> On Wed, 6 Oct 2021 at 09:21, mi...@ubo.ro wrote: >> >>> Hi Ian , >>> >>> I've modified the example towards a more specific use case. The main >>> idea in the example below is to make code related to database >>> operations(i.e SELECT queries) safer and easier to read. A kind of >>> json.Unmarshal/Marshal for databases, with validation (type checking, param >>> numbers etc) to avoid a class of bugs/errors such invalid param >>> types/numbers passed, invalid queries, invalid resource type to scan into >>> etc. >>> >>> Currently the function returned by *Select* throws the validation >>> errors at runtime (i.e. invalid param type passed etc). It would be great >>> to have that class of errors checked at compile type. >>> >>> The only way I could achieve that kind of type checking was through >>> code generation. I already built a tool to generate functions with the >>> proper param types but it seems that code generation introduces a lot of >>> friction to the point that I stopped using it. >>> >>> My hope is that one day a Go feature (i.e. a version of Generics) could >>> help the function returned by *func* *Select* be type checked at >>> compile time. >>> >>> >> If you change your API slightly to use a single selector argument of >> struct type, you could do something like this: >> >> https://go2goplay.golang.org/p/QH1VCQFxrZS >> >> In summary: >> >> type Flower struct { >> Color string >> Size int >> Weight int >> } >> >> type ByColor struct { >> Color string >> } >> >> var FlowerByColor = Select[Flower, ByColor]("* FROM tablex WHERE Color=$ >> LIMIT 1") >> >> type SelectFunc[Resource, Args any] func(args Args) (Resource, error) >> >> // Select returns a function that executes the given SQL query, >> // expecting results to contain fields matching Resource and >> // using fields in Args to select rows. >> // >> // Both Resource and Args must be struct types; All the fields >> // in Args must have matchi
[go-nuts] How to negotiate authentication over HTTP ? (the Go way)
I find myself in need to handle various authentication schemes such "proprietary" oauth2 schemes and NTLM. The easy and clean way to do it (API wise) would be using a http.RoundTripper but looks like it forbids you from even reading the response headers. In the end I just made a ``func DoHTTPRequestWithAuthenticationHandling(cl *http.Client, req *http.Request)(*http.Response, error)`` function that just wraps net/http.Client.Do(), clones the request and response if it's necessary and negotiates the authentication scheme. It's basically what I wanted to do within http.RoundTripper except now the user of the http.Client needs to always remember to execute the requests the right way (i.e. using a different function) instead of http.Do. Is there a better way to do it? Would it be a good idea in Go 2.0 make http.Client an interface to prevent this kind of limitation/workarounds? -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/f6ecfd34-a5b9-4b83-a56d-bcbf6b47086fn%40googlegroups.com.
[go-nuts] Re: How to negotiate authentication over HTTP ? (the Go way)
Thanks for your help. I'm aware I can set the headers that way but the authentication transport also needs to inspect the response headers and optionally re-submit the request based on the headers received. That's part of the "negotiation" mechanism. That's because we don;t know what authentication scheme is supported by the remote party until we receive the actual response/headers from a blind request. Below is a simple use case: - the http client executes a reques with no authentication. - we receive a status code 403 along with a www-header indicating that oauth2 authentication is supported. - we resend the request with appropriate oauth2 headers. Note that we had to read the response code and response headers so that we can re-send the request. This is the part that I wish I could automate using the http.RoundTripper. The alternative is to either handle the response manually after each request or create a custom endpoint < i.e. func ExecuteHTTP(*http.Client, *http.Request)(*http.Response, error) > that handles the response headers and retries the requests using the proper authentication headers. In both cases you loose the flexibility to use a http client that does all this in the background. Some packages (i.e ElasticSearch package) support configuration using your own http client. However you cannot pass a custom `ExecuteHTTP` function to ElasticSearch so it becomes quite hard to hack the authentication/negotiation. - Mihai. On Wednesday, October 27, 2021 at 1:32:24 AM UTC+3 ben...@gmail.com wrote: > I'm not sure what these proprietary auth schemes look like (and don't know > much about oauth2 or NTLM), but for many kinds of auth you'd just set > headers in the request and read them in the body. For example: > > request, err := http.NewRequest("GET", "https://httpbin.org/get";, nil) > // handle err > request.Header.Set("X-My-Auth", "abcd1234") > response, err := client.Do(request) > // handle err > // handle response > > Runnable code example here: > https://play.golang.org/p/cocv1avzNCo > > And of course you can roll your own function to create a new request with > auth headers already applied. Would this kind of thing work for you? > > -Ben > > On Tuesday, October 26, 2021 at 2:05:15 PM UTC+13 mi...@ubo.ro wrote: > >> I find myself in need to handle various authentication schemes such >> "proprietary" oauth2 schemes and NTLM. The easy and clean way to do it (API >> wise) would be using a http.RoundTripper but looks like it forbids you from >> even reading the response headers. >> >> In the end I just made a ``func >> DoHTTPRequestWithAuthenticationHandling(cl *http.Client, req >> *http.Request)(*http.Response, error)`` function that just wraps net/ >> http.Client.Do(), clones the request and response if it's necessary and >> negotiates the authentication scheme. It's basically what I wanted to do >> within http.RoundTripper except now the user of the http.Client needs to >> always remember to execute the requests the right way (i.e. using a >> different function) instead of http.Do. >> >> Is there a better way to do it? Would it be a good idea in Go 2.0 make >> http.Client an interface to prevent this kind of limitation/workarounds? >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/add71229-4714-4826-afee-c964c4c706f2n%40googlegroups.com.
[go-nuts] Re: How to negotiate authentication over HTTP ? (the Go way)
That would work great but the documentation on RoundTriper specifically forbids it[0]. Unless I get it wrong(do I?) you are not allowed to read the response within RoundTrip. The request needs to be .Clone-ed as well but that's not an issue. [0] https://pkg.go.dev/net/http#RoundTripper // RoundTrip should not attempt to interpret the response. In // particular, RoundTrip must return err == nil if it obtained // a response, regardless of the response's HTTP status code. // A non-nil err should be reserved for failure to obtain a // response. Similarly, RoundTrip should not attempt to // handle higher-level protocol details such as redirects, // authentication, or cookies. // RoundTrip should not modify the request *On 27/10/2021 02:09, Ben Hoyt wrote:* *Oh, I see. An "ExecuteHTTP" function seems reasonable, but yeah, if you need an *http.Client, I think you have to customize the Transport/Roundtripper. You mention in your initial email that RoundTripper "forbids you from even reading the response headers", but I don't think that's the case, right?* *For example, here's a custom "AuthTransport":* *type AuthTransport struct { AuthHeader string}func (t *AuthTransport) RoundTrip(request *http.Request) (*http.Response, error) { request.Header.Set("X-My-Auth", t.AuthHeader) response, err := http.DefaultTransport.RoundTrip(request) if err != nil { return nil, err } if response.StatusCode == 403 || response.Header.Get("WWW-Authenticate") != "" { fmt.Println("Would retry here") }return response, nil}* *and then you'd instantiate your http.Client like so:* *client := &http.Client{Timeout: 5 * time.Second,Transport: &AuthTransport{AuthHeader: "abcd1234"},}* *Did that not work for you?* *-Ben* On Wednesday, October 27, 2021 at 1:52:18 AM UTC+3 mi...@ubo.ro wrote: > Thanks for your help. I'm aware I can set the headers that way but the > authentication transport also needs to inspect the response headers and > optionally re-submit the request based on the headers received. That's part > of the "negotiation" mechanism. That's because we don;t know what > authentication scheme is supported by the remote party until we receive the > actual response/headers from a blind request. > > Below is a simple use case: > > - the http client executes a reques with no authentication. > > - we receive a status code 403 along with a www-header indicating that > oauth2 authentication is supported. > >- we resend the request with appropriate oauth2 headers. > > Note that we had to read the response code and response headers so that we > can re-send the request. This is the part that I wish I could automate > using the http.RoundTripper. > > The alternative is to either handle the response manually after each > request or create a custom endpoint < i.e. func ExecuteHTTP(*http.Client, > *http.Request)(*http.Response, error) > that handles the response headers > and retries the requests using the proper authentication headers. > > In both cases you loose the flexibility to use a http client that does all > this in the background. > > Some packages (i.e ElasticSearch package) support configuration using your > own http client. However you cannot pass a custom `ExecuteHTTP` function to > ElasticSearch so it becomes quite hard to hack the > authentication/negotiation. > > - Mihai. > > On Wednesday, October 27, 2021 at 1:32:24 AM UTC+3 ben...@gmail.com wrote: > >> I'm not sure what these proprietary auth schemes look like (and don't >> know much about oauth2 or NTLM), but for many kinds of auth you'd just set >> headers in the request and read them in the body. For example: >> >> request, err := http.NewRequest("GET", "https://httpbin.org/get";, >> nil) >> // handle err >> request.Header.Set("X-My-Auth", "abcd1234") >> response, err := client.Do(request) >> // handle err >> // handle response >> >> Runnable code example here: >> https://play.golang.org/p/cocv1avzNCo >> >> And of course you can roll your own function to create a new request with >> auth headers already applied. Would this kind of thing work for you? >> >> -Ben >> >> On Tuesday, October 26, 2021 at 2:05:15 PM UTC+13 mi...@ubo.ro wrote: >> >>> I find myself in need to handle various authentication schemes such >>> "proprietary" oauth2 schemes and NTLM. The easy and clean way to do it (API >>> wise) would be using a http.RoundTripper but l
[go-nuts] go/wasm how to test browser APIs
I'm trying to unitest a browser wasm/js application but it seems that the browser objects are missing. I assume that the testing package is using a "serverside" node API so I wonder how can I make go test use a real browser to run the tests. The test below fails func TestTransform(t *testing.T) { doc := js.Global().Get("document") if !doc.Truthy() { t.Fatalf("window.document not available?? wrong environment") } } // $ GOOS=js GOARCH=wasm go test -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/dbca55ec-d0eb-4585-8bc9-43b92e7691a9n%40googlegroups.com.
[go-nuts] Is it possible to define a generic/inherited method?
I have a function that encodes Go data structure in a certain way. Below is a an example // Data structure type DataX struct{ X string `query:"x"` } func MarshalJSON(v interface)([]byte, error){ s := query.Encode(v) return s.Bytes(), nil } Currently in order to force the json encoder use this function on `type DataX` I have to implement json.Marshaler on it and call the same MarshalJSON function. e.g. func(x *DataX)MarshalJSON()([]byte, error){ // use our "generic"/common function return MarshalJSON(v) } Is there is generic way to reduce this boilerplate ? Why do I have to write a method if I know that all it does it to call the same function? I thinking about something like this: // Define DataX as a struct that also implements QueryMarshaler?? // unicorn code. type DataX struct [QueryMarshaler]{ X string `query:"name"` } // DataY implments QueryMarshaler's methods as well type DataY [QueryMarshaler]{ Y string `query:"y"` } ///... // QueryMarshaler method type QueryMarshaler interface{} func [K QueryMarshaler](v K)MarshalJSON()([]byte, error){ s := query.Encode(v) return s.Bytes(), nil } -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/4cf4f8da-5866-4a32-8624-6ffb9740ed18n%40googlegroups.com.
[go-nuts] oauth2/jws creates "invalid" signatures
Does anyone know why the jws signatures created by the golang.org/x/oauth2/jws are displayed as "invalid signature" on jwt.io ? As far as I'm concerned it seems compliant with the JWS creation specs[0] but it looks like jwt.io is expecting a public key or "jwk string" as well ? Below is an example of signatures that appears as "invalid" on jwt.io [1] and the code[2] [0] https://openid.net/specs/draft-jones-json-web-signature-04.html#anchor5 [1] eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJodHRwOi8vZ29vZ2xlLmNvbS8iLCJhdWQiOiIiLCJleHAiOjM2MTAsImlhdCI6MTB9.iIT1HnaZbpbN80TUunM_FAPgerBD4LilNZIX-M55tzRqgE8nDC57inkQF0KcVyLk4Y55WOtBlSj045u35twKkHokEGjSpSSQT31Rcf6ugxqYMKnqIvw9quzwaPJA_RmiudJVuCe_zyVka008M7fZfblwcaTWr1AXZ3iUrwOZnnP9Hli0merjPicVhNIG7SbZTyGFh6P9NUiX0y54iqsV_3yXQZep_UGJYuLR7v1hRRr1tphEiNUt4lBtcp_7nraLnUDTyMraZ8WpTwvn57GAQ4ShzxotEkR3z_5zDxsHRirJcLSBWZ-SNHl3XYXhGV48ePiMJlZ-PR6OQfJ35f-WiQ [2] https://go.dev/play/p/7fr-CxOIVvd // You can edit this code! // Click here and start typing. package main import ( "crypto/rand" "crypto/rsa" "fmt" jws "golang.org/x/oauth2/jws" ) func main() { header := &jws.Header{ Algorithm: "RS256", Typ: "JWT", } payload := &jws.ClaimSet{ Iss: "http://google.com/";, Aud: "", Exp: 3610, Iat: 10, } privateKey, err := rsa.GenerateKey(rand.Reader, 2048) if err != nil { panic(err) } token, err := jws.Encode(header, payload, privateKey) if err != nil { panic(err) } fmt.Println(token) } -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/4fc2b94e-009a-4b3b-81c3-740a10e4255en%40googlegroups.com.