[go-nuts] Go X/Tools SSA init$guard

[Xie Yuchen]

I tried to write the SSA information out and found the *init$guard *which 
is likely to decide the execution of the sub-packages *init *functions.

For example, give the following go code, I will get the SSA output.

package main

import "fmt"
import _ "unsafe"

const message = "Hello, World!"

func init(){}

func main() {
fmt.Println(message)
}

func empty(){}

The output is shown below. Based on the code, I have a rough idea  that the 
variable *init$guard *might be set by someone to control the execution of 
*init*. However, i failed to find the documentation about it when I search 
"init$guard", "package initializer" and so on. Could someone help to 
explain how it works and which component sets up the variable. 















*# Name: hello.init# Package: hello# Synthetic: package initializerfunc 
init():0:   
 entry P:0 S:2t0 = *init$guard 
  boolif t0 goto 2 else 11: 
  init.start P:1 S:1*init$guard = 
true:boolt1 = fmt.init()   
   ()t2 = unsafe.init() 
  ()t3 = init#1()   
 ()jump 22: 
   init.done P:2 S:0return*

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b851ef80-838c-4bbb-b575-508ae1988f39n%40googlegroups.com.

Re: [go-nuts] net/http: no Client API to close server connection based on Response #21978

[Robert Engels]

That probably wasn’t clear. Why not create a Transport per host. Then when the 500 is encountered stop using that transport completely and create a new instance. Probably want to cancel any requests currently in flight. The connection pool is per transport. On Apr 2, 2024, at 11:05 PM, Eli Lindsey  wrote:There isn’t a great way to handle this currently - we maintain out of tree patches to do something similar, though ours are h2 specific. The crux of the problem is that net currently lacks a usable connection pool API (there is some slightly newer discussion here, but it’s similar to the issue you linked https://github.com/golang/go/discussions/60746).If you want to stay in tree, one option may be using httptrace GotConnInfo and calling Close on the underlying connection (in direct violation of GotConnInfo’s doc). I would expect this to error out anything inflight, but otherwise be benign (though I have not checked :) ).
-eli


On Apr 2, 2024, at 3:29 PM, Jim Minter  wrote:Hello,I was wondering if anyone had any ideas about https://github.com/golang/go/issues/21978 ("net/http: no Client API to close server connection based on Response") -- it's an old issue, but it's something that's biting me currently and I can't see a neat way to solve it.As an HTTP client, I'm hitting a case where some HTTP server instance behind a load balancer breaks and starts returning 500s (FWIW with no body) and without the "Connection: close" header.  I retry, but I end up reusing the same TCP connection to the same broken HTTP instance, so I never hit a different backend server and my retry policy is basically useless.Obviously I need to get the server owner to fix its behavior, but it would be great if, as a client, there were a way to get net/http not to reuse the connection further, in order to be less beholden to the server's behavior.This happens with both HTTP/1.1 and HTTP/2.If appropriate, I could live with the request to close the connection racing with other new requests to the same endpoint.  Getting to the point where 2 or 3 requests fail and then the connection is closed is way better than having requests fail ad infinitum.http.Transport.CloseIdleConnections() doesn't solve the problem well (a) because it's a big hammer, and (b) because there's no guarantee that the connection is idle when CloseIdleConnections() is called.FWIW I can see in `func (pc *persistConn) readLoop()` there's the following test:```goif resp.Close || rc.req.Close || resp.StatusCode <= 199 || bodyWritable {	// Don't do keep-alive on error if either party requested a close	// or we get an unexpected informational (1xx) response.	// StatusCode 100 is already handled above.	alive = false}```I imagine that extending that to `if resp.Close || rc.req.Close || resp.StatusCode <= 199 || bodyWritable || resp.StatusCode >= 500 {` might probably help this specific case, but I imagine that's an unacceptably large behavior change for the rest of the world.I'm not sure how else this could be done.  Does anyone have any thoughts?Many thanks for the help,Jim

-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/34d597cf-a84c-48eb-b555-537a8768f468n%40googlegroups.com.




-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/080B6923-51DA-4DDB-9400-B1054C1DFCE4%40siliconsprawl.com.




-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/6AA93658-082F-4580-A3BD-6603D1D83394%40ix.netcom.com.

Re: [go-nuts] user process instruction pointer symbol lookup

['TheDiveO' via golang-nuts]

Thank you Ian!

Not sure yet if I found something useful related to the real process 
addresses/offsets, but at least the "bcc" sources have one 
"bcc_elf_get_text_scn_info()" that returns the address and offset of the 
".text" section.

On Tuesday, April 2, 2024 at 8:01:11 PM UTC+2 Ian Lance Taylor wrote:

> On Tue, Apr 2, 2024 at 2:35 AM 'TheDiveO' via golang-nuts
>  wrote:
> >
> > On Linux, given an arbitrary binary executable with symbol information 
> in the executable, how can I lookup an instruction pointer address to get 
> the corresponding symbol name?
> >
> > The binary (and its process) isn't a Go binary, but any arbitrary 
> executable. The stack unwinding has already been done, so I'm presented 
> with a list of instruction pointer addresses (return addresses) which I 
> need to convert to more useful symbol names.
> >
> > I've seen the stdlib's debug/elf package, but I lack the ELF knowledge 
> to press the elf package's knobs in the right order. Any examples of how to 
> use debug/elf, and is it even the right package to use in this case?
>
> debug/elf is the package to use. You'll want to call the Symbols
> method and look through the Symbols for one whose Value is <= the PC
> you want while Value+Size is > the PC you want.
>
> If you are looking at runtime PC's from a stack trace, be aware that
> on most systems these days programs are position-independent, so there
> will be an offset between the addresses in the binary and the
> addresses from the stack trace. I don't know offhand of a standard
> way to figure out that offset.
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/7076c963-9cad-4c24-b00f-dcf9d88ede8fn%40googlegroups.com.

[go-nuts] Re: CMSSignedData in Golang

[Shivli Srivastava]

I tried with pkcs7, but for the same canonical input the digest is matching 
but the signature format is not similar to the one generated by Java Bouncy 
Castle

On Tuesday, April 2, 2024 at 8:17:52 PM UTC+5:30 Tamás Gulácsi wrote:

> Earlier questions bubbled up 
> https://pkg.go.dev/github.com/fullsailor/pkcs7?utm_source=godoc#SignedData
> (https://github.com/smallstep/pkcs7)
>
> For the XML canonicalization, you may try 
> https://pkg.go.dev/github.com/lafriks/go-xmldsig/v2#Canonicalizer
>
>
>
> Shivli Srivastava a következőt írta (2024. április 2., kedd, 15:43:54 
> UTC+2):
>
>> I have to replicate the Java code for signing xml in Go. The java code 
>> uses org.bouncycastle.cms.CMSSignedData from BouncyCastle for signing and 
>> org.apache.xml.serialize.XMLSerializer for Serializing the input xml . 
>>
>> The signing process should be exactly same as the signature otherwise 
>> would be different and I cannot afford that . Any help will be appreciated
>>
>>
>> Regards
>> Shivli
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/3412b81f-7c84-446e-a11a-0d9344f03bc1n%40googlegroups.com.

[go-nuts] Re: CMSSignedData in Golang

[Shivli Srivastava]

If I have to call my java code from golang application , how can I 
integrate it ? 

On Wednesday, April 3, 2024 at 5:21:28 PM UTC+5:30 Shivli Srivastava wrote:

> I tried with pkcs7, but for the same canonical input the digest is 
> matching but the signature format is not similar to the one generated by 
> Java Bouncy Castle
>
> On Tuesday, April 2, 2024 at 8:17:52 PM UTC+5:30 Tamás Gulácsi wrote:
>
>> Earlier questions bubbled up 
>> https://pkg.go.dev/github.com/fullsailor/pkcs7?utm_source=godoc#SignedData
>> (https://github.com/smallstep/pkcs7)
>>
>> For the XML canonicalization, you may try 
>> https://pkg.go.dev/github.com/lafriks/go-xmldsig/v2#Canonicalizer
>>
>>
>>
>> Shivli Srivastava a következőt írta (2024. április 2., kedd, 15:43:54 
>> UTC+2):
>>
>>> I have to replicate the Java code for signing xml in Go. The java code 
>>> uses org.bouncycastle.cms.CMSSignedData from BouncyCastle for signing and 
>>> org.apache.xml.serialize.XMLSerializer for Serializing the input xml . 
>>>
>>> The signing process should be exactly same as the signature otherwise 
>>> would be different and I cannot afford that . Any help will be appreciated
>>>
>>>
>>> Regards
>>> Shivli
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/e6f93770-d0a5-4054-aa46-ce0c0f32715bn%40googlegroups.com.

Re: [go-nuts] net/http: no Client API to close server connection based on Response #21978

[Eli Lindsey]

It would work, but has potentially high cost since it also causes any healthy 
conns in the pool to be torn down. How useful it is in practice depends on 
request rate, number of backends behind the lb, and ratio of healthy to 
unhealthy (500’ing) connections. It’s hard to tell from the description if it 
would work here - retrying and reusing the same busted connection could mean 
that the request rate is very low and there’s only one idle conn (in which case 
cycling the transport is a good solution), or it could mean that the unhealthy 
conn is quicker to respond than the pooled healthy conns and gobbles up a 
disproportionate share of requests.

Tangential question, when the backend servers land in this state does the lb 
not detect and remove them?

-eli

> On Apr 3, 2024, at 6:41 AM, Robert Engels  wrote:
> 
> That probably wasn’t clear. Why not create a Transport per host. Then when 
> the 500 is encountered stop using that transport completely and create a new 
> instance. Probably want to cancel any requests currently in flight. 
> 
> The connection pool is per transport. 
> 
>> On Apr 2, 2024, at 11:05 PM, Eli Lindsey  wrote:
>> 
>> There isn’t a great way to handle this currently - we maintain out of tree 
>> patches to do something similar, though ours are h2 specific. The crux of 
>> the problem is that net currently lacks a usable connection pool API (there 
>> is some slightly newer discussion here, but it’s similar to the issue you 
>> linked https://github.com/golang/go/discussions/60746).
>> 
>> If you want to stay in tree, one option may be using httptrace GotConnInfo 
>> and calling Close on the underlying connection (in direct violation of 
>> GotConnInfo’s doc). I would expect this to error out anything inflight, but 
>> otherwise be benign (though I have not checked :) ).
>> 
>> -eli
>> 
>>> On Apr 2, 2024, at 3:29 PM, Jim Minter  wrote:
>>> 
>>> Hello,
>>> 
>>> I was wondering if anyone had any ideas about 
>>> https://github.com/golang/go/issues/21978 ("net/http: no Client API to 
>>> close server connection based on Response") -- it's an old issue, but it's 
>>> something that's biting me currently and I can't see a neat way to solve it.
>>> 
>>> As an HTTP client, I'm hitting a case where some HTTP server instance 
>>> behind a load balancer breaks and starts returning 500s (FWIW with no body) 
>>> and without the "Connection: close" header.  I retry, but I end up reusing 
>>> the same TCP connection to the same broken HTTP instance, so I never hit a 
>>> different backend server and my retry policy is basically useless.
>>> 
>>> Obviously I need to get the server owner to fix its behavior, but it would 
>>> be great if, as a client, there were a way to get net/http not to reuse the 
>>> connection further, in order to be less beholden to the server's behavior.
>>> 
>>> This happens with both HTTP/1.1 and HTTP/2.
>>> 
>>> If appropriate, I could live with the request to close the connection 
>>> racing with other new requests to the same endpoint.  Getting to the point 
>>> where 2 or 3 requests fail and then the connection is closed is way better 
>>> than having requests fail ad infinitum.
>>> 
>>> http.Transport.CloseIdleConnections() doesn't solve the problem well (a) 
>>> because it's a big hammer, and (b) because there's no guarantee that the 
>>> connection is idle when CloseIdleConnections() is called.
>>> 
>>> FWIW I can see in `func (pc *persistConn) readLoop()` there's the following 
>>> test:
>>> 
>>> ```go
>>> if resp.Close || rc.req.Close || resp.StatusCode <= 199 || bodyWritable {
>>> // Don't do keep-alive on error if either party requested a close
>>> // or we get an unexpected informational (1xx) response.
>>> // StatusCode 100 is already handled above.
>>> alive = false
>>> }
>>> ```
>>> 
>>> I imagine that extending that to `if resp.Close || rc.req.Close || 
>>> resp.StatusCode <= 199 || bodyWritable || resp.StatusCode >= 500 {` might 
>>> probably help this specific case, but I imagine that's an unacceptably 
>>> large behavior change for the rest of the world.
>>> 
>>> I'm not sure how else this could be done.  Does anyone have any thoughts?
>>> 
>>> Many thanks for the help,
>>> 
>>> Jim
>>> 
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golang-nuts+unsubscr...@googlegroups.com 
>>> .
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/34d597cf-a84c-48eb-b555-537a8768f468n%40googlegroups.com
>>>  
>>> .
>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and st

[go-nuts] [security] Go 1.22.2 and Go 1.21.9 are released

[announce]

Hello gophers,

We have just released Go versions 1.22.2 and 1.21.9, minor point releases.

These minor releases include 1 security fixes following the security policy 
:

-   http2: close connections when receiving too many headers

Maintaining HPACK state requires that we parse and process all HEADERS 
and CONTINUATION frames on a connection. When a request's headers exceed 
MaxHeaderBytes, we don't allocate memory to store the excess headers but we do 
parse them. This permits an attacker to cause an HTTP/2 endpoint to read 
arbitrary amounts of header data, all associated with a request which is going 
to be rejected. These headers can include Huffman-encoded data which is 
significantly more expensive for the receiver to decode than for an attacker to 
send.

Set a limit on the amount of excess header frames we will process 
before closing a connection.

Thanks to Bartek Nowotarski (https://nowotarski.info/) for reporting 
this issue.

This is CVE-2023-45288 and Go issue https://go.dev/issue/65051.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.22.2

You can download binary and source distributions from the Go website:
https://go.dev/dl/

To compile from source using a Git clone, update to the release with
git checkout go1.22.2 and build as usual.

Thanks to everyone who contributed to the releases.

Cheers,
Than and Dmitri for the Go team

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/rfAzX27jQnC-bZEsfyVhQg%40geopod-ismtpd-21.

Re: [go-nuts] Re: CMSSignedData in Golang

[Karlovsky Alexey]

You can wrap your java BC code in http server and access it with golang
http client, also, you can wrap your BC code in CLI tool and exec java
process with os.exec.

But, as I can understand you need CMS signature for xml content, with
golang you can do CMS signatures with https://github.com/fullsailor/pkcs7
or some of its forks (for example https://github.com/mozilla-services/pkcs7
has some improvements, but marked as archived recently).
What really matters - the normalization of your XML content and you can
look at
https://github.com/russellhaering/goxmldsig/blob/main/etreeutils/canonicalize.go#L11
.

On Wed, Apr 3, 2024 at 1:54 PM Shivli Srivastava 
wrote:

> If I have to call my java code from golang application , how can I
> integrate it ?
>
> On Wednesday, April 3, 2024 at 5:21:28 PM UTC+5:30 Shivli Srivastava wrote:
>
>> I tried with pkcs7, but for the same canonical input the digest is
>> matching but the signature format is not similar to the one generated by
>> Java Bouncy Castle
>>
>> On Tuesday, April 2, 2024 at 8:17:52 PM UTC+5:30 Tamás Gulácsi wrote:
>>
>>> Earlier questions bubbled up
>>> https://pkg.go.dev/github.com/fullsailor/pkcs7?utm_source=godoc#SignedData
>>> (https://github.com/smallstep/pkcs7)
>>>
>>> For the XML canonicalization, you may try
>>> https://pkg.go.dev/github.com/lafriks/go-xmldsig/v2#Canonicalizer
>>>
>>>
>>>
>>> Shivli Srivastava a következőt írta (2024. április 2., kedd, 15:43:54
>>> UTC+2):
>>>
 I have to replicate the Java code for signing xml in Go. The java code
 uses org.bouncycastle.cms.CMSSignedData from BouncyCastle for signing and
 org.apache.xml.serialize.XMLSerializer for Serializing the input xml .

 The signing process should be exactly same as the signature otherwise
 would be different and I cannot afford that . Any help will be appreciated


 Regards
 Shivli

>>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/e6f93770-d0a5-4054-aa46-ce0c0f32715bn%40googlegroups.com
> 
> .
>


-- 
Kind regards,
Alexey

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAMDzThhRD3FKL1tNykgQQAOW8YDa3Oq2w1MbCO8w%3DD3Nr-ZD2A%40mail.gmail.com.

Re: [go-nuts] net/http: no Client API to close server connection based on Response #21978

[Robert Engels]

Just create a recyclable transport for the bad server and put all of the rest on a single shared transport. If one connection is returning 500 for all requests I can’t see how a different connection would solve that - unless the backend is completely broken. On Apr 3, 2024, at 7:48 AM, Eli Lindsey  wrote:It would work, but has potentially high cost since it also causes any healthy conns in the pool to be torn down. How useful it is in practice depends on request rate, number of backends behind the lb, and ratio of healthy to unhealthy (500’ing) connections. It’s hard to tell from the description if it would work here - retrying and reusing the same busted connection could mean that the request rate is very low and there’s only one idle conn (in which case cycling the transport is a good solution), or it could mean that the unhealthy conn is quicker to respond than the pooled healthy conns and gobbles up a disproportionate share of requests.Tangential question, when the backend servers land in this state does the lb not detect and remove them?
-eli


On Apr 3, 2024, at 6:41 AM, Robert Engels  wrote:That probably wasn’t clear. Why not create a Transport per host. Then when the 500 is encountered stop using that transport completely and create a new instance. Probably want to cancel any requests currently in flight. The connection pool is per transport. On Apr 2, 2024, at 11:05 PM, Eli Lindsey  wrote:There isn’t a great way to handle this currently - we maintain out of tree patches to do something similar, though ours are h2 specific. The crux of the problem is that net currently lacks a usable connection pool API (there is some slightly newer discussion here, but it’s similar to the issue you linked https://github.com/golang/go/discussions/60746).If you want to stay in tree, one option may be using httptrace GotConnInfo and calling Close on the underlying connection (in direct violation of GotConnInfo’s doc). I would expect this to error out anything inflight, but otherwise be benign (though I have not checked :) ).
-eli


On Apr 2, 2024, at 3:29 PM, Jim Minter  wrote:Hello,I was wondering if anyone had any ideas about https://github.com/golang/go/issues/21978 ("net/http: no Client API to close server connection based on Response") -- it's an old issue, but it's something that's biting me currently and I can't see a neat way to solve it.As an HTTP client, I'm hitting a case where some HTTP server instance behind a load balancer breaks and starts returning 500s (FWIW with no body) and without the "Connection: close" header.  I retry, but I end up reusing the same TCP connection to the same broken HTTP instance, so I never hit a different backend server and my retry policy is basically useless.Obviously I need to get the server owner to fix its behavior, but it would be great if, as a client, there were a way to get net/http not to reuse the connection further, in order to be less beholden to the server's behavior.This happens with both HTTP/1.1 and HTTP/2.If appropriate, I could live with the request to close the connection racing with other new requests to the same endpoint.  Getting to the point where 2 or 3 requests fail and then the connection is closed is way better than having requests fail ad infinitum.http.Transport.CloseIdleConnections() doesn't solve the problem well (a) because it's a big hammer, and (b) because there's no guarantee that the connection is idle when CloseIdleConnections() is called.FWIW I can see in `func (pc *persistConn) readLoop()` there's the following test:```goif resp.Close || rc.req.Close || resp.StatusCode <= 199 || bodyWritable {	// Don't do keep-alive on error if either party requested a close	// or we get an unexpected informational (1xx) response.	// StatusCode 100 is already handled above.	alive = false}```I imagine that extending that to `if resp.Close || rc.req.Close || resp.StatusCode <= 199 || bodyWritable || resp.StatusCode >= 500 {` might probably help this specific case, but I imagine that's an unacceptably large behavior change for the rest of the world.I'm not sure how else this could be done.  Does anyone have any thoughts?Many thanks for the help,Jim

-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/34d597cf-a84c-48eb-b555-537a8768f468n%40googlegroups.com.


-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/080B6923-51DA-4DDB-9400-B1054C1DFCE4%40siliconsprawl.com.




-- 
You received this message because you are subscribed to the Google Groups

Re: [go-nuts] net/http: no Client API to close server connection based on Response #21978

[Jim Minter]

Yes, I agree, I think this approach makes sense (and should have been
obvious to me as something to try...).  It could be implementable as a
wrapper transport too.  I'll try it out and reply back here if it doesn't
work.

Thank-you!

Jim


On Wed, 3 Apr 2024 at 12:46, Robert Engels  wrote:

> Just create a recyclable transport for the bad server and put all of the
> rest on a single shared transport. If one connection is returning 500 for
> all requests I can’t see how a different connection would solve that -
> unless the backend is completely broken.
>
> On Apr 3, 2024, at 7:48 AM, Eli Lindsey  wrote:
>
> It would work, but has potentially high cost since it also causes any
> healthy conns in the pool to be torn down. How useful it is in practice
> depends on request rate, number of backends behind the lb, and ratio of
> healthy to unhealthy (500’ing) connections. It’s hard to tell from the
> description if it would work here - retrying and reusing the same busted
> connection could mean that the request rate is very low and there’s only
> one idle conn (in which case cycling the transport is a good solution), or
> it could mean that the unhealthy conn is quicker to respond than the pooled
> healthy conns and gobbles up a disproportionate share of requests.
>
> Tangential question, when the backend servers land in this state does the
> lb not detect and remove them?
>
> -eli
>
> On Apr 3, 2024, at 6:41 AM, Robert Engels  wrote:
>
> That probably wasn’t clear. Why not create a Transport per host. Then when
> the 500 is encountered stop using that transport completely and create a
> new instance. Probably want to cancel any requests currently in flight.
>
> The connection pool is per transport.
>
> On Apr 2, 2024, at 11:05 PM, Eli Lindsey  wrote:
>
> There isn’t a great way to handle this currently - we maintain out of
> tree patches to do something similar, though ours are h2 specific. The crux
> of the problem is that net currently lacks a usable connection pool API
> (there is some slightly newer discussion here, but it’s similar to the
> issue you linked https://github.com/golang/go/discussions/60746).
>
> If you want to stay in tree, one option may be using httptrace GotConnInfo
> and calling Close on the underlying connection (in direct violation of
> GotConnInfo’s doc). I would expect this to error out anything inflight, but
> otherwise be benign (though I have not checked :) ).
>
> -eli
>
> On Apr 2, 2024, at 3:29 PM, Jim Minter  wrote:
>
> Hello,
>
> I was wondering if anyone had any ideas about
> https://github.com/golang/go/issues/21978 ("net/http: no Client API to
> close server connection based on Response") -- it's an old issue, but it's
> something that's biting me currently and I can't see a neat way to solve it.
>
> As an HTTP client, I'm hitting a case where some HTTP server instance
> behind a load balancer breaks and starts returning 500s (FWIW with no body)
> and without the "Connection: close" header.  I retry, but I end up reusing
> the same TCP connection to the same broken HTTP instance, so I never hit a
> different backend server and my retry policy is basically useless.
>
> Obviously I need to get the server owner to fix its behavior, but it would
> be great if, as a client, there were a way to get net/http not to reuse the
> connection further, in order to be less beholden to the server's behavior.
>
> This happens with both HTTP/1.1 and HTTP/2.
>
> If appropriate, I could live with the request to close the connection
> racing with other new requests to the same endpoint.  Getting to the point
> where 2 or 3 requests fail and then the connection is closed is way better
> than having requests fail ad infinitum.
>
> http.Transport.CloseIdleConnections() doesn't solve the problem well (a)
> because it's a big hammer, and (b) because there's no guarantee that the
> connection is idle when CloseIdleConnections() is called.
>
> FWIW I can see in `func (pc *persistConn) readLoop()` there's the
> following test:
>
> ```go
> if resp.Close || rc.req.Close || resp.StatusCode <= 199 || bodyWritable {
> // Don't do keep-alive on error if either party requested a close
> // or we get an unexpected informational (1xx) response.
> // StatusCode 100 is already handled above.
> alive = false
> }
> ```
>
> I imagine that extending that to `if resp.Close || rc.req.Close ||
> resp.StatusCode <= 199 || bodyWritable || resp.StatusCode >= 500 {` might
> probably help this specific case, but I imagine that's an unacceptably
> large behavior change for the rest of the world.
>
> I'm not sure how else this could be done.  Does anyone have any thoughts?
>
> Many thanks for the help,
>
> Jim
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golan

Re: [go-nuts] [generics] type constraint for structs

[Adam Manwaring]

While this would make some things much easier for me, it seems this would 
be a pretty fundamental change. Constraints are essentially interfaces and 
interfaces in Go are defined by behaviors. Structs, on the other hand, are 
defined by properties. There is no behavior that all structs have that 
every other type couldn't also have. Thus having a struct constraint would 
be a one-off exception which for the most part seems anathema to Go. In a 
similar vein, there are many times I'd like an interface to require certain 
exported properties in addition to behaviors, but this isn't going to 
happen.

On Wednesday, March 27, 2024 at 6:28:19 PM UTC-6 Makis Maropoulos wrote:

> Same here @Abraham, 
>
> ResponseType interface {
> ~struct{}
> }
>
> Obviously this doesn't work, I would love to see it working though.
> On Wednesday 14 September 2022 at 17:48:19 UTC+3 Abraham wrote:
>
>> I am glad I found this thread because I was just now breaking my head 
>> figuring out why my  was not working
>>
>> On Wednesday, May 18, 2022 at 10:41:29 PM UTC-4 Ian Lance Taylor wrote:
>>
>>> On Wed, May 18, 2022 at 7:36 PM Jeremy Kassis  wrote: 
>>> > 
>>> > Where exactly did this land? Seems like an important conversation... 
>>>
>>> To date there is no way to write a constraint that requires that a 
>>> type argument be a struct type. 
>>>
>>>
>>> > ``` 
>>> > // RPCHandler passes RPCReq and RPCRes as fn args 
>>> > func RPCHandler[T RPCReq, S RPCRes](fn func(T, S)) http.HandlerFunc { 
>>> > return func(w http.ResponseWriter, r *http.Request) { 
>>> > req := T{} 
>>> > if err := reqBodyReadAll(w, r, &req); err != nil { 
>>> > resWriteErr(w, err) 
>>> > return 
>>> > } 
>>> > res := S{} 
>>> > fn(req, res) 
>>> > resWriteAll(w, r, res) 
>>> > } 
>>> > } 
>>> > ``` 
>>>
>>> I would write simply "var req T" and "var res S". 
>>>
>>> Ian 
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b23017ff-d591-47ba-baea-3f8e140192ban%40googlegroups.com.

Re: [go-nuts] [generics] type constraint for structs

[atd...@gmail.com]

Might have come across this today as I was trying to simplify some code.

Basically, I have a base type called *Element that has a method AsElement() 
*Element that returns itself.

And this base element is used as a base for many others, for isntance:

type Div struct{ *Element}
type Span struct{*Element}
type Anchor {*Element}
...

Somehow, I need a function Ref that can take a pointer to any of these 
elements and modify them (basically modifying the inner *Element pointer).

Currently, I don't think I can abstract over these types which share in 
common the *Element field and the AsEleemnt method promoted from it.
I don't want to have to implement a specific method to do that mutation 
operation for each and every one of these types either.

Basically, trying to make generic this function:

func Ref(vref **Element) func(*Element) *Element{
return func(e *Element)*Element{
*vref = e
return e
}
}


Or does anyone see something that I missed?

On Thursday, April 4, 2024 at 2:52:16 AM UTC+2 Adam Manwaring wrote:

> While this would make some things much easier for me, it seems this would 
> be a pretty fundamental change. Constraints are essentially interfaces and 
> interfaces in Go are defined by behaviors. Structs, on the other hand, are 
> defined by properties. There is no behavior that all structs have that 
> every other type couldn't also have. Thus having a struct constraint would 
> be a one-off exception which for the most part seems anathema to Go. In a 
> similar vein, there are many times I'd like an interface to require certain 
> exported properties in addition to behaviors, but this isn't going to 
> happen.
>
> On Wednesday, March 27, 2024 at 6:28:19 PM UTC-6 Makis Maropoulos wrote:
>
>> Same here @Abraham, 
>>
>> ResponseType interface {
>> ~struct{}
>> }
>>
>> Obviously this doesn't work, I would love to see it working though.
>> On Wednesday 14 September 2022 at 17:48:19 UTC+3 Abraham wrote:
>>
>>> I am glad I found this thread because I was just now breaking my head 
>>> figuring out why my  was not working
>>>
>>> On Wednesday, May 18, 2022 at 10:41:29 PM UTC-4 Ian Lance Taylor wrote:
>>>
 On Wed, May 18, 2022 at 7:36 PM Jeremy Kassis  
 wrote: 
 > 
 > Where exactly did this land? Seems like an important conversation... 

 To date there is no way to write a constraint that requires that a 
 type argument be a struct type. 


 > ``` 
 > // RPCHandler passes RPCReq and RPCRes as fn args 
 > func RPCHandler[T RPCReq, S RPCRes](fn func(T, S)) http.HandlerFunc { 
 > return func(w http.ResponseWriter, r *http.Request) { 
 > req := T{} 
 > if err := reqBodyReadAll(w, r, &req); err != nil { 
 > resWriteErr(w, err) 
 > return 
 > } 
 > res := S{} 
 > fn(req, res) 
 > resWriteAll(w, r, res) 
 > } 
 > } 
 > ``` 

 I would write simply "var req T" and "var res S". 

 Ian 

>>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/f2150cf5-75d9-4cb3-9a29-d5a8c8e655a5n%40googlegroups.com.

Re: [go-nuts] [generics] type constraint for structs

[atd...@gmail.com]

Ah, I forgot to use reflection for instance... That might be doable.

On Thursday, April 4, 2024 at 4:13:35 AM UTC+2 atd...@gmail.com wrote:

> Might have come across this today as I was trying to simplify some code.
>
> Basically, I have a base type called *Element that has a method 
> AsElement() *Element that returns itself.
>
> And this base element is used as a base for many others, for isntance:
>
> type Div struct{ *Element}
> type Span struct{*Element}
> type Anchor {*Element}
> ...
>
> Somehow, I need a function Ref that can take a pointer to any of these 
> elements and modify them (basically modifying the inner *Element pointer).
>
> Currently, I don't think I can abstract over these types which share in 
> common the *Element field and the AsEleemnt method promoted from it.
> I don't want to have to implement a specific method to do that mutation 
> operation for each and every one of these types either.
>
> Basically, trying to make generic this function:
>
> func Ref(vref **Element) func(*Element) *Element{
> return func(e *Element)*Element{
> *vref = e
> return e
> }
> }
>
>
> Or does anyone see something that I missed?
>
> On Thursday, April 4, 2024 at 2:52:16 AM UTC+2 Adam Manwaring wrote:
>
>> While this would make some things much easier for me, it seems this would 
>> be a pretty fundamental change. Constraints are essentially interfaces and 
>> interfaces in Go are defined by behaviors. Structs, on the other hand, are 
>> defined by properties. There is no behavior that all structs have that 
>> every other type couldn't also have. Thus having a struct constraint would 
>> be a one-off exception which for the most part seems anathema to Go. In a 
>> similar vein, there are many times I'd like an interface to require certain 
>> exported properties in addition to behaviors, but this isn't going to 
>> happen.
>>
>> On Wednesday, March 27, 2024 at 6:28:19 PM UTC-6 Makis Maropoulos wrote:
>>
>>> Same here @Abraham, 
>>>
>>> ResponseType interface {
>>> ~struct{}
>>> }
>>>
>>> Obviously this doesn't work, I would love to see it working though.
>>> On Wednesday 14 September 2022 at 17:48:19 UTC+3 Abraham wrote:
>>>
 I am glad I found this thread because I was just now breaking my head 
 figuring out why my  was not working

 On Wednesday, May 18, 2022 at 10:41:29 PM UTC-4 Ian Lance Taylor wrote:

> On Wed, May 18, 2022 at 7:36 PM Jeremy Kassis  
> wrote: 
> > 
> > Where exactly did this land? Seems like an important conversation... 
>
> To date there is no way to write a constraint that requires that a 
> type argument be a struct type. 
>
>
> > ``` 
> > // RPCHandler passes RPCReq and RPCRes as fn args 
> > func RPCHandler[T RPCReq, S RPCRes](fn func(T, S)) http.HandlerFunc 
> { 
> > return func(w http.ResponseWriter, r *http.Request) { 
> > req := T{} 
> > if err := reqBodyReadAll(w, r, &req); err != nil { 
> > resWriteErr(w, err) 
> > return 
> > } 
> > res := S{} 
> > fn(req, res) 
> > resWriteAll(w, r, res) 
> > } 
> > } 
> > ``` 
>
> I would write simply "var req T" and "var res S". 
>
> Ian 
>


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/c34ebc0f-c2bf-4b59-a59d-ee3b3047bca6n%40googlegroups.com.

Re: [go-nuts] net/http: no Client API to close server connection based on Response #21978

[Robert Engels]

Happy that it sparked an idea. I also don’t think Eli’s concern is valid. if there are other requests in flight (on different connections I assume) - let those continue - just put any new requests on a new transport for that host (after a 500 error is encountered) - then tear down the bad when it has no more requests being processed. On Apr 3, 2024, at 1:50 PM, Jim Minter  wrote:Yes, I agree, I think this approach makes sense (and should have been obvious to me as something to try...).  It could be implementable as a wrapper transport too.  I'll try it out and reply back here if it doesn't work.Thank-you!JimOn Wed, 3 Apr 2024 at 12:46, Robert Engels  wrote:Just create a recyclable transport for the bad server and put all of the rest on a single shared transport. If one connection is returning 500 for all requests I can’t see how a different connection would solve that - unless the backend is completely broken. On Apr 3, 2024, at 7:48 AM, Eli Lindsey  wrote:It would work, but has potentially high cost since it also causes any healthy conns in the pool to be torn down. How useful it is in practice depends on request rate, number of backends behind the lb, and ratio of healthy to unhealthy (500’ing) connections. It’s hard to tell from the description if it would work here - retrying and reusing the same busted connection could mean that the request rate is very low and there’s only one idle conn (in which case cycling the transport is a good solution), or it could mean that the unhealthy conn is quicker to respond than the pooled healthy conns and gobbles up a disproportionate share of requests.Tangential question, when the backend servers land in this state does the lb not detect and remove them?
-eli


On Apr 3, 2024, at 6:41 AM, Robert Engels  wrote:That probably wasn’t clear. Why not create a Transport per host. Then when the 500 is encountered stop using that transport completely and create a new instance. Probably want to cancel any requests currently in flight. The connection pool is per transport. On Apr 2, 2024, at 11:05 PM, Eli Lindsey  wrote:There isn’t a great way to handle this currently - we maintain out of tree patches to do something similar, though ours are h2 specific. The crux of the problem is that net currently lacks a usable connection pool API (there is some slightly newer discussion here, but it’s similar to the issue you linked https://github.com/golang/go/discussions/60746).If you want to stay in tree, one option may be using httptrace GotConnInfo and calling Close on the underlying connection (in direct violation of GotConnInfo’s doc). I would expect this to error out anything inflight, but otherwise be benign (though I have not checked :) ).
-eli


On Apr 2, 2024, at 3:29 PM, Jim Minter  wrote:Hello,I was wondering if anyone had any ideas about https://github.com/golang/go/issues/21978 ("net/http: no Client API to close server connection based on Response") -- it's an old issue, but it's something that's biting me currently and I can't see a neat way to solve it.As an HTTP client, I'm hitting a case where some HTTP server instance behind a load balancer breaks and starts returning 500s (FWIW with no body) and without the "Connection: close" header.  I retry, but I end up reusing the same TCP connection to the same broken HTTP instance, so I never hit a different backend server and my retry policy is basically useless.Obviously I need to get the server owner to fix its behavior, but it would be great if, as a client, there were a way to get net/http not to reuse the connection further, in order to be less beholden to the server's behavior.This happens with both HTTP/1.1 and HTTP/2.If appropriate, I could live with the request to close the connection racing with other new requests to the same endpoint.  Getting to the point where 2 or 3 requests fail and then the connection is closed is way better than having requests fail ad infinitum.http.Transport.CloseIdleConnections() doesn't solve the problem well (a) because it's a big hammer, and (b) because there's no guarantee that the connection is idle when CloseIdleConnections() is called.FWIW I can see in `func (pc *persistConn) readLoop()` there's the following test:```goif resp.Close || rc.req.Close || resp.StatusCode <= 199 || bodyWritable {	// Don't do keep-alive on error if either party requested a close	// or we get an unexpected informational (1xx) response.	// StatusCode 100 is already handled above.	alive = false}```I imagine that extending that to `if resp.Close || rc.req.Close || resp.StatusCode <= 199 || bodyWritable || resp.StatusCode >= 500 {` might probably help this specific case, but I imagine that's an unacceptably large behavior change for the rest of the world.I'm not sure how else this could be done.  Does anyone have any thoughts?Many thanks for the help,Jim

-- 
You received this message because you are

Re: [go-nuts] [generics] type constraint for structs

['Axel Wagner' via golang-nuts]

How does `interface{ AsElement() *Element }` not do exactly what you want?

On Thu, Apr 4, 2024 at 4:14 AM atd...@gmail.com  wrote:

> Might have come across this today as I was trying to simplify some code.
>
> Basically, I have a base type called *Element that has a method
> AsElement() *Element that returns itself.
>
> And this base element is used as a base for many others, for isntance:
>
> type Div struct{ *Element}
> type Span struct{*Element}
> type Anchor {*Element}
> ...
>
> Somehow, I need a function Ref that can take a pointer to any of these
> elements and modify them (basically modifying the inner *Element pointer).
>
> Currently, I don't think I can abstract over these types which share in
> common the *Element field and the AsEleemnt method promoted from it.
> I don't want to have to implement a specific method to do that mutation
> operation for each and every one of these types either.
>
> Basically, trying to make generic this function:
>
> func Ref(vref **Element) func(*Element) *Element{
> return func(e *Element)*Element{
> *vref = e
> return e
> }
> }
>
>
> Or does anyone see something that I missed?
>
> On Thursday, April 4, 2024 at 2:52:16 AM UTC+2 Adam Manwaring wrote:
>
>> While this would make some things much easier for me, it seems this would
>> be a pretty fundamental change. Constraints are essentially interfaces and
>> interfaces in Go are defined by behaviors. Structs, on the other hand, are
>> defined by properties. There is no behavior that all structs have that
>> every other type couldn't also have. Thus having a struct constraint would
>> be a one-off exception which for the most part seems anathema to Go. In a
>> similar vein, there are many times I'd like an interface to require certain
>> exported properties in addition to behaviors, but this isn't going to
>> happen.
>>
>> On Wednesday, March 27, 2024 at 6:28:19 PM UTC-6 Makis Maropoulos wrote:
>>
>>> Same here @Abraham,
>>>
>>> ResponseType interface {
>>> ~struct{}
>>> }
>>>
>>> Obviously this doesn't work, I would love to see it working though.
>>> On Wednesday 14 September 2022 at 17:48:19 UTC+3 Abraham wrote:
>>>
 I am glad I found this thread because I was just now breaking my head
 figuring out why my  was not working

 On Wednesday, May 18, 2022 at 10:41:29 PM UTC-4 Ian Lance Taylor wrote:

> On Wed, May 18, 2022 at 7:36 PM Jeremy Kassis 
> wrote:
> >
> > Where exactly did this land? Seems like an important conversation...
>
> To date there is no way to write a constraint that requires that a
> type argument be a struct type.
>
>
> > ```
> > // RPCHandler passes RPCReq and RPCRes as fn args
> > func RPCHandler[T RPCReq, S RPCRes](fn func(T, S)) http.HandlerFunc
> {
> > return func(w http.ResponseWriter, r *http.Request) {
> > req := T{}
> > if err := reqBodyReadAll(w, r, &req); err != nil {
> > resWriteErr(w, err)
> > return
> > }
> > res := S{}
> > fn(req, res)
> > resWriteAll(w, r, res)
> > }
> > }
> > ```
>
> I would write simply "var req T" and "var res S".
>
> Ian
>
 --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/f2150cf5-75d9-4cb3-9a29-d5a8c8e655a5n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAEkBMfHmvPxbTqh0BhV9Ubd8ZM3XhaABHD9TXmWOP-wYKwO4rw%40mail.gmail.com.