Use the replace directive to point the net package to your local copy. Much easier to test X changes than stdlib changes.
> On Nov 19, 2021, at 10:58 AM, Kirth Gersen <kirthal...@gmail.com> wrote: > > Your CL works well with the POC. > > Side question not specific to this issue: how to test changes to > golang.org/x/net with net/http ? > The 'h2_bundle' trick with go generate & bundle requires to fork the std lib > too ? > I have a hard time figuring how to do this. I tried with gotip but I get an > error with "gotip generate" (bundle: internal error: package "strings" > without types was imported from "golang.org/x/net/http2") > Any doc/tutorial on how to deal with this 'bundle' trick ? > > thx >> Le lundi 15 novembre 2021 à 17:32:48 UTC+1, ren...@ix.netcom.com a écrit : >> Since http2 multiplexes streams it will delicately affect latency on other >> streams. This is why I suggested using multiple transports - one for high >> throughput transfers and another for lower latency “interactive” sessions. >> >>>> On Nov 15, 2021, at 9:23 AM, Kevin Chowski <ke...@chowski.com> wrote: >>>> >>> These are interesting results, thanks for investigating and sharing >>> results! >> >>> >>> I see that you have mostly been focusing on throughput in your posts, have >>> you done testing for latency differences too? >>> >>>> On Saturday, November 13, 2021 at 6:11:40 PM UTC-7 ren...@ix.netcom.com >>>> wrote: >>>> As another data point, I decided to test a few implementations of http2 >>>> downloads on OSX. >>>> >>>> Using a Go server with default frame size (16k): >>>> >>>> Go client: 900 MB/s >>>> Java client: 1300 MB/s >>>> curl: 1500 MB/s >>>> >>>> Using a Java server with default frame size (16k): >>>> >>>> Go client: 670 MB/s >>>> Java client: 720 MB/s >>>> curl: 800 M/s >>>> >>>> Using Go server using 256k client max frame size: >>>> >>>> Go client: 2350 MB/s >>>> Java client: 2800 MB/s >>>> h2load: 4300 MB/s >>>> >>>> Using Java server using 256k client max frame size: >>>> >>>> Go client: 2900 MB/s >>>> Java client: 2800 MB/s >>>> h2load: 3750 MB/s >>>> >>>> For h2load, I needed to create a PR to allow the frame size to be set, see >>>> https://github.com/nghttp2/nghttp2/pull/1640 >>>> >>>> >>>>> On Nov 10, 2021, at 7:04 PM, robert engels <ren...@ix.netcom.com> wrote: >>>>> >>>>> No worries. I updated the issue and the CL. I will comment in the CL with >>>>> a few more details. >>>>> >>>>>> On Nov 10, 2021, at 2:30 PM, Andrey T. <xnow4f...@sneakemail.com> wrote: >>>>>> >>>>>> Thank you Robert, >>>>>> I somehow missed the reference to the ticket in the first message, sorry >>>>>> about that. >>>>>> >>>>>> As for the CL - I think adding link to the github issue, and add a bit >>>>>> of explanation in a commit message would help. >>>>>> I added link to your CL to the github issue's discussion, hopefully it >>>>>> will bring more attention to it. >>>>>> >>>>>> A. >>>>>> >>>>>>> On Wednesday, November 10, 2021 at 1:22:42 PM UTC-7 >>>>>>> ren...@ix.netcom.com wrote: >>>>>>> As reported in the OP, the issue was filed long ago >>>>>>> https://github.com/golang/go/issues/47840 >>>>>>> >>>>>>> My CL https://go-review.googlesource.com/c/net/+/362834 is a viable fix >>>>>>> (and should of been supported originally). >>>>>>> >>>>>>>>> On Nov 10, 2021, at 12:59 PM, Andrey T. <xnow4f...@sneakemail.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>> Fellas, >>>>>>>> I would say the 5x throughput difference is a serious problem.Would >>>>>>>> you be kind and open an issue on github about it? >>>>>>>> Also, the PR that you have might benefit from explanation about what >>>>>>>> you are trying to solve (and probably link to an issue on github), so >>>>>>>> it would get more attention. >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>> Andrey >>>>>>>> >>>>>>>> >>>>>>>>> On Tuesday, November 9, 2021 at 4:50:34 PM UTC-7 ren...@ix.netcom.com >>>>>>>>> wrote: >>>>>>>>> Well, I figured out a way to do it simply. The CL is here >>>>>>>>> https://go-review.googlesource.com/c/net/+/362834 >>>>>>>>> >>>>>>>>> The frame size will be used for all connections using that transport, >>>>>>>>> so it is probably better to create a transport specifically for the >>>>>>>>> high-throughput transfers. >>>>>>>>> >>>>>>>>> You can also create perform single shot requests like: >>>>>>>>> >>>>>>>>> if useH2C { >>>>>>>>> rt = &http2.Transport{ >>>>>>>>> AllowHTTP: true, >>>>>>>>> DialTLS: func(network, addr string, cfg *tls.Config) (net.Conn, >>>>>>>>> error) { >>>>>>>>> return dialer.Dial(network, addr) >>>>>>>>> }, >>>>>>>>> MaxFrameSize: 1024*256, >>>>>>>>> } >>>>>>>>> } >>>>>>>>> >>>>>>>>> var body io.ReadCloser = http.NoBody >>>>>>>>> >>>>>>>>> req, err := http.NewRequestWithContext(ctx, "GET", url, body) >>>>>>>>> if err != nil { >>>>>>>>> return err >>>>>>>>> } >>>>>>>>> >>>>>>>>> resp, err := rt.RoundTrip(req) >>>>>>>>> >>>>>>>>> >>>>>>>>>> On Nov 9, 2021, at 3:31 PM, Robert Engels <ren...@ix.netcom.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> To be clear, I have no plans to submit a Cl to improve this at this >>>>>>>>>> time. >>>>>>>>>> >>>>>>>>>> It would require some api changes to implement properly. >>>>>>>>>> >>>>>>>>>>>> On Nov 9, 2021, at 12:19 PM, Kirth Gersen <kirth...@gmail.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>> Great ! >>>>>>>>>>> >>>>>>>>>>> > I made some local mods to the net library, increasing the frame >>>>>>>>>>> > size to 256k, and the http2 performance went from 8Gbps to 38Gbps. >>>>>>>>>>> That is already enormous for us. thx for finding this. >>>>>>>>>>> >>>>>>>>>>> 4 -> Indeed a lot of WINDOW_UPDATE messages are visible when using >>>>>>>>>>> GODEBUG=http2debug=1 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> On Tuesday, November 9, 2021 at 6:28:16 PM UTC+1 >>>>>>>>>>>> ren...@ix.netcom.com wrote: >>>>>>>>>>>> I did a review of the codebase. >>>>>>>>>>>> >>>>>>>>>>>> Http2 is a multiplexed protocol with independent streams. The Go >>>>>>>>>>>> implementation uses a common reader thread/routine to read all of >>>>>>>>>>>> the connection content, and then demuxes the streams and passes >>>>>>>>>>>> the data via pipes to the stream readers. This multithreaded >>>>>>>>>>>> nature requires the use of locks to coordinate. By managing the >>>>>>>>>>>> window size, the connection reader should never block writing to a >>>>>>>>>>>> steam buffer - but a stream reader may stall waiting for data to >>>>>>>>>>>> arrive - get descheduled - only to be quickly rescheduled when >>>>>>>>>>>> reader places more data in the buffer - which is inefficient. >>>>>>>>>>>> >>>>>>>>>>>> Out of the box on my machine, http1 is about 37 Gbps, and http2 is >>>>>>>>>>>> about 7 Gbps on my system. >>>>>>>>>>>> >>>>>>>>>>>> Some things that jump out: >>>>>>>>>>>> >>>>>>>>>>>> 1. The chunk size is too small. Using 1MB pushed http1 from 37 Gbs >>>>>>>>>>>> to 50 Gbps, and http2 to 8 Gbps. >>>>>>>>>>>> >>>>>>>>>>>> 2. The default buffer in io.Copy() is too small. Use >>>>>>>>>>>> io.CopyBuffer() with a larger buffer - I changed to 4MB. This >>>>>>>>>>>> pushed http1 to 55 Gbs, and http2 to 8.2. Not a big difference but >>>>>>>>>>>> needed for later. >>>>>>>>>>>> >>>>>>>>>>>> 3. The http2 receiver frame size of 16k is way too small. There is >>>>>>>>>>>> overhead on every frame - the most costly is updating the window. >>>>>>>>>>>> >>>>>>>>>>>> I made some local mods to the net library, increasing the frame >>>>>>>>>>>> size to 256k, and the http2 performance went from 8Gbps to 38Gbps. >>>>>>>>>>>> >>>>>>>>>>>> 4. I haven’t tracked it down yet, but I don’t think the window >>>>>>>>>>>> size update code is not working as intended - it seems to be >>>>>>>>>>>> sending window updates (which are expensive due to locks) far too >>>>>>>>>>>> frequently. I think this is the area that could use the most >>>>>>>>>>>> improvement - using some heuristics there is the possibility to >>>>>>>>>>>> detect the sender rate, and adjust the refresh rate (using >>>>>>>>>>>> high/low water marks). >>>>>>>>>>>> >>>>>>>>>>>> 5. The implementation might need improvements using lock-free >>>>>>>>>>>> structures, atomic counters, and busy-waits in order to achieve >>>>>>>>>>>> maximum performance. >>>>>>>>>>>> >>>>>>>>>>>> So 38Gbps for http2 vs 55 Gbps for http1. Better but still not >>>>>>>>>>>> great. Still, with some minor changes, the net package could allow >>>>>>>>>>>> setting of a large frame size on a per stream basis - which would >>>>>>>>>>>> enable much higher throughput. The gRPC library allows this. >>>>>>>>>>>> >>>>>>>>>>>>>> On Nov 8, 2021, at 10:58 AM, Kirth Gersen <kirth...@gmail.com> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>> http/2 implementation seems ~5x slower in bytes per seconds (when >>>>>>>>>>>>> transfer is cpu capped). >>>>>>>>>>>>> >>>>>>>>>>>>> POC: https://github.com/nspeed-app/http2issue >>>>>>>>>>>>> >>>>>>>>>>>>> I submitted an issue about this 3 months ago in the Go Github ( >>>>>>>>>>>>> https://github.com/golang/go/issues/47840 ) but first commenter >>>>>>>>>>>>> misunderstood it and it got buried (they're probably just swamped >>>>>>>>>>>>> with too many open issues (5k+...)). >>>>>>>>>>>>> >>>>>>>>>>>>> Everything using Golang net/http is impacted, the Caddy web >>>>>>>>>>>>> server for instance. >>>>>>>>>>>>> >>>>>>>>>>>>> I know it probably doesn't matter for most use cases because it's >>>>>>>>>>>>> only noticeable with high throughput transfers (>1 Gbps). >>>>>>>>>>>>> Most http benchmarks focus on "requests per second" and not "bits >>>>>>>>>>>>> per seconds" but this performance matters too sometimes. >>>>>>>>>>>>> >>>>>>>>>>>>> If anyone with expertise in profiling Go code and good knowledge >>>>>>>>>>>>> of the net/http lib internal could take a look. It would be nice >>>>>>>>>>>>> to optimize it or at least have an explanation. >>>>>>>>>>>>> >>>>>>>>>>>>> thx (sorry if wrong group to post this). >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>> Google Groups "golang-nuts" group. >>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>>>>> send an email to golang-nuts...@googlegroups.com. >>>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>> https://groups.google.com/d/msgid/golang-nuts/89926c2f-ec73-43ad-be49-a8bc76a18345n%40googlegroups.com. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>>>> Groups "golang-nuts" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>>> send an email to golang-nuts...@googlegroups.com. >>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/d/msgid/golang-nuts/7332f727-6716-4c4d-85c5-a86cacd0c89fn%40googlegroups.com. >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "golang-nuts" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>>>> an email to golang-nuts...@googlegroups.com. >>>>>>> >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/golang-nuts/1bfe6aec-abd2-4f63-bf77-bbfa6fd213ban%40googlegroups.com. >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "golang-nuts" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>> an email to golang-nuts...@googlegroups.com. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/golang-nuts/1b63863b-45af-45d0-a885-8716acc65ac7n%40googlegroups.com. >>>>> >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "golang-nuts" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to golang-nuts...@googlegroups.com. >> >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/golang-nuts/3138434b-d480-4473-8b20-2598412a0eden%40googlegroups.com. > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/03b1493a-bf8a-4cd2-b10d-20e2cc57cad9n%40googlegroups.com. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/8CDA24E0-7BA4-4FB8-9527-27817953C7F4%40ix.netcom.com.
