Hi all! Thanks for the inputs, I'll do some profiling here and update about my findings.
I don't want to change anything on Nginx because the comparison I'm doing between different stacks is on the same basis. If I try to tune Nginx or anything like that I'll be comparing apples to oranges, so that's not the point of my experiment. Cheers! On Saturday, December 3, 2022 at 6:07:42 AM UTC-3 Brian Candler wrote: > Have you tried a tcpdump of the packets between the Go program and nginx? > Is it using HTTP/1.1 or HTTP/2? If it's HTTP/1.1, does tcpdump show that > it actually starts all 50 HTTP client requests simultaneously? > > You are making all these concurrent requests to the same host. The default > of MaxConnsPerHost I believe is 0 (unlimited), but > DefaultMaxIdleConnsPerHost is 2. It could be worth cranking that up. I > wonder if it's being forced to close down 48 of those connections > immediately because it can't return them to the pool. > > I also wonder whether there's tuning required at the Nginx side, e.g. for > the backlog queue. > > On Saturday, 3 December 2022 at 05:08:40 UTC harr...@spu.edu wrote: > >> I wonder a bit about io.ReadAll versus constructing a JSON Decoder. In >> general, though, using pprof is the best way to start to break down a >> question like this. Would the actual workload involve more structured JSON, >> or more computation with decoded values? >> >> On Friday, December 2, 2022 at 7:31:50 PM UTC-8 bse...@computer.org >> wrote: >> >>> On Fri, Dec 2, 2022 at 8:13 PM Diogo Baeder <diogo...@gmail.com> wrote: >>> >>>> Hi guys, >>>> >>>> I've been working on some experiments with different web application >>>> stacks to check their performances under a specific scenario: one in which >>>> I have to make several concurrent requests and then gather the results >>>> together (in order) and throw them out as JSON in the response body. (This >>>> project is only an experiment, but it's informing me for decisions that >>>> have to be made for a real-world project where we have a similar scenario.) >>>> >>>> However, probably due to my ignorance in Go, I cannot make it perform >>>> as well as I expected - actually the best I'm getting are results that are >>>> even slower than Python, which was a surprise to me. Here they are: >>>> https://github.com/yougov/concurrency-tests#edit-13-added-golang-with-gin >>>> >>>> So, looking at the code here: >>>> https://github.com/yougov/concurrency-tests/blob/master/stacks/goapp/main.go >>>> >>>> - does anybody see any problem in the implementation that could be hurting >>>> performance? I tried using a WaitGroup, tried sharing memory (nasty, I >>>> know, but just for the sake of experimentation), tried multiple JSON >>>> codecs, different web frameworks, and nothing worked so far. I have a >>>> feeling that I'm doing something fundamentally wrong and stupid, and that >>>> somehow I can make a small change to make the experiment much faster. >>>> >>> >>> Have you measured how much time is spent on the http.Get calls? It is >>> likely that the 50 concurrent http.Get calls is the bottleneck. >>> >>> Also note that you don't need a channel there. You can simply use a >>> waitgroup and set the results from inside the goroutine, because each >>> goroutine knows the index. But that is unlikely to change anything >>> measurable when compared to the Get calls. >>> >>> >>> >>>> >>>> Thanks in advance, I'm sure this will help me learning more about the >>>> language! :-) >>>> >>>> Cheers! >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "golang-nuts" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to golang-nuts...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/golang-nuts/42d7d04f-f6d8-4d96-bcdf-bcf32b99a73cn%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/golang-nuts/42d7d04f-f6d8-4d96-bcdf-bcf32b99a73cn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/66d94fac-a7f2-4fd3-a809-b1be1f668181n%40googlegroups.com.
