Thanks a lot for all your help Marcin! Your expertise makes a total
difference here!
Double-thumbs up!
On Mon, Jun 10, 2019 at 7:46 PM Marcin Romaszewicz <marc...@gmail.com> wrote:
>
> In my example in the previous email, I accidentally used a very old version
> of echo. If you use the latest ("github.com/labstack/echo/v4"), then it's a
> lot faster than using simple string splitting, resulting in about 22,000
> requests per second.
>
> Concurrency Level: 100
> Time taken for tests: 0.443 seconds
> Complete requests: 10000
> Failed requests: 0
> Total transferred: 2160000 bytes
> HTML transferred: 430000 bytes
> Requests per second: 22565.77 [#/sec] (mean)
> Time per request: 4.431 [ms] (mean)
> Time per request: 0.044 [ms] (mean, across all concurrent requests)
> Transfer rate: 4759.97 [Kbytes/sec] received
>
> Connection Times (ms)
> min mean[+/-sd] median max
> Connect: 0 2 0.5 2 9
> Processing: 1 2 0.6 2 9
> Waiting: 0 2 0.6 2 9
> Total: 3 4 0.8 4 11
>
> Percentage of the requests served within a certain time (ms)
> 50% 4
> 66% 4
> 75% 5
> 80% 5
> 90% 5
> 95% 5
> 98% 6
> 99% 6
> 100% 11 (longest request)
>
>
> On Mon, Jun 10, 2019 at 4:12 PM Marcin Romaszewicz <marc...@gmail.com> wrote:
>>
>> One more followup.
>>
>> Here's an example using an HTTP router named Echo, which I use in
>> production. With proper HTTP parsing and validation, and no regular
>> expressions involved in routing, it still does about 14,000 requests per
>> second. I stubbed some of your stuff which doesn't affect the result. This
>> is a much better implementation than speaking HTTP yourself over sockets and
>> it's stupendously fast.
>>
>> Concurrency Level: 100
>> Time taken for tests: 0.717 seconds
>> Complete requests: 10000
>> Failed requests: 0
>> Total transferred: 2160000 bytes
>> HTML transferred: 430000 bytes
>> Requests per second: 13940.29 [#/sec] (mean)
>> Time per request: 7.173 [ms] (mean)
>> Time per request: 0.072 [ms] (mean, across all concurrent requests)
>> Transfer rate: 2940.53 [Kbytes/sec] received
>>
>> Connection Times (ms)
>> min mean[+/-sd] median max
>> Connect: 0 3 1.2 3 8
>> Processing: 1 4 1.2 4 9
>> Waiting: 0 3 1.2 3 8
>> Total: 2 7 1.3 7 13
>>
>> Percentage of the requests served within a certain time (ms)
>> 50% 7
>> 66% 7
>> 75% 8
>> 80% 8
>> 90% 9
>> 95% 10
>> 98% 11
>> 99% 11
>> 100% 13 (longest request)
>>
>>
>> package main
>>
>> import (
>> "flag"
>> "fmt"
>> "net/http"
>> "strconv"
>>
>> "github.com/labstack/echo"
>> )
>>
>> const pixel =
>> "\x47\x49\x46\x38\x39\x61\x01\x00\x01\x00\x80\x00\x00\xFF\xFF\xFF\x00\x00\x00\x21\xF9\x04\x01\x00\x00\x00\x00\x2C\x00\x00\x00\x00\x01\x00\x01\x00\x00\x02\x02\x44\x01\x00\x3B"
>>
>> func main() {
>> var port = flag.Int("port", 8080, "service port")
>> flag.Parse()
>>
>> autoProxy := fmt.Sprintf(
>> "function FindProxyForURL(url, host) { return \"PROXY %s:3128;
>> DIRECT\"; }",
>> "stubbed.host")
>> autoProxyBuf := []byte(autoProxy)
>>
>> e := echo.New()
>>
>> serveAutoProxy := func(c echo.Context) error {
>> response := c.Response()
>> response.Header().Add("Connection", "close")
>> return c.Blob(http.StatusOK, "application/octet-stream", autoProxyBuf)
>> }
>> e.GET("/proxy.pac", serveAutoProxy)
>> e.GET("/wpad.dat", serveAutoProxy)
>>
>> pixelBuf := []byte(pixel)
>> servePixel := func(c echo.Context) error {
>> response := c.Response()
>> response.Header().Add("Connection", "close")
>> response.Header().Add("ETag", "dbab")
>> response.Header().Add("Cache-Control", "public, max-age=31536000")
>> response.Header().Add("Content-length", strconv.Itoa(len(pixelBuf)))
>> return c.Blob(http.StatusOK, "image/gif", pixelBuf)
>> }
>> e.GET("*", servePixel)
>>
>> // Start server
>> e.Logger.Fatal(e.Start(fmt.Sprintf("0.0.0.0:%d", *port)))
>> }
>>
>>
>>
>> On Mon, Jun 10, 2019 at 4:07 PM Marcin Romaszewicz <marc...@gmail.com> wrote:
>>>
>>> I think the others were correct in pointing the finger at the RegEx engine
>>> in Go. It is quite slow. I hacked your inside loop which checks the request
>>> not to use regular expressions, and it's tons faster. You can't say that
>>> something can't be responsible for too much slowdown because it "1 line",
>>> since that one line has a lot of weight behind it. Using regular
>>> expressions, a benchmark showed 1782 requests per second. Using my simple
>>> hack, it's 18827 per second. Let's call it 10x faster.
>>>
>>> if strings.Contains(line, "HTTP") {
>>> parts := strings.Split(line, " ")
>>> req.Method = strings.ToUpper(strings.TrimSpace(parts[0]))
>>> req.URL = strings.ToUpper(strings.TrimSpace(parts[1]))
>>> req.Version = strings.ToUpper(strings.TrimSpace(parts[2]))
>>> continue
>>> }
>>>
>>>
>>> For this benchmark it behaves correctly, I realize this is fragile. I ran
>>> `ab -n 10000 -c 100...` to run the tests.
>>>
>>> Benchmarks using regular expressions:
>>> Concurrency Level: 100
>>> Time taken for tests: 5.610 seconds
>>> Complete requests: 10000
>>> Failed requests: 0
>>> Total transferred: 1790000 bytes
>>> HTML transferred: 430000 bytes
>>> Requests per second: 1782.41 [#/sec] (mean)
>>> Time per request: 56.104 [ms] (mean)
>>> Time per request: 0.561 [ms] (mean, across all concurrent requests)
>>> Transfer rate: 311.57 [Kbytes/sec] received
>>>
>>> Connection Times (ms)
>>> min mean[+/-sd] median max
>>> Connect: 0 46 352.7 8 3557
>>> Processing: 0 10 10.6 8 158
>>> Waiting: 0 10 10.6 8 158
>>> Total: 0 56 352.5 21 3565
>>>
>>> Percentage of the requests served within a certain time (ms)
>>> 50% 21
>>> 66% 24
>>> 75% 24
>>> 80% 24
>>> 90% 25
>>> 95% 26
>>> 98% 164
>>> 99% 3549
>>> 100% 3565 (longest request)
>>>
>>> Benchmarks using my hack:
>>>
>>> Concurrency Level: 100
>>> Time taken for tests: 0.531 seconds
>>> Complete requests: 10000
>>> Failed requests: 0
>>> Total transferred: 1790000 bytes
>>> HTML transferred: 430000 bytes
>>> Requests per second: 18827.39 [#/sec] (mean)
>>> Time per request: 5.311 [ms] (mean)
>>> Time per request: 0.053 [ms] (mean, across all concurrent requests)
>>> Transfer rate: 3291.12 [Kbytes/sec] received
>>>
>>> Connection Times (ms)
>>> min mean[+/-sd] median max
>>> Connect: 0 3 0.3 3 4
>>> Processing: 1 3 0.4 3 5
>>> Waiting: 0 3 0.4 3 5
>>> Total: 3 5 0.5 5 8
>>>
>>> Percentage of the requests served within a certain time (ms)
>>> 50% 5
>>> 66% 5
>>> 75% 5
>>> 80% 5
>>> 90% 6
>>> 95% 6
>>> 98% 7
>>> 99% 7
>>> 100% 8 (longest request)
>>>
>>>
>>> On Mon, Jun 10, 2019 at 2:28 PM Tong Sun <suntong...@gmail.com> wrote:
>>>>
>>>> Just to clarify some facts.
>>>>
>>>> On Sun, Jun 9, 2019 at 11:09 AM 'Axel Wagner' wrote:
>>>> >
>>>> > As I've also mentioned: I don't think this test is meaningful.
>>>> >
>>>> > First, as it has been pointed out, your Perl program isn't actually a
>>>> > web server. It only understands ridiculously simple requests and as such
>>>> > violates the spec left and right. It's also super naive in how it treats
>>>> > malformed input or actively malicious clients - all of which are handled
>>>> > by the Go http package, so of course it's going to have some overhead.
>>>>
>>>> There is a second test well before your this post, which is a direct
>>>> translation of Perl code, that is now reading and writing directly to
>>>> a socket. Hanging on to the first test method and not referring to the
>>>> second test is not a very constructive way of discussion, let alone
>>>> using words like "ridiculously ...".
>>>>
>>>> > In its most generous form, you translate the programs faithfully to do
>>>> > the same syscalls and implement the same logic - and at that point, you
>>>> > are essentially benchmarking the Perl regular expression engine against
>>>> > the Go regular expression engine, as that's the only thing the program
>>>> > really is doing: Branching on a regexp-match. And the Perl RE-engine is
>>>> > famously optimized and the Go RE-engine famously is not. In fact, you
>>>> > aren't even benchmarking Perl against Go, you are benchmarking *C*
>>>> > against Go, as Perls RE-engine is written in C, AFAIK.
>>>>
>>>> Over 90% of the code are NOT doing regular expression matching.
>>>> Focusing *only* on regular expression, not >90% of the rest, is not
>>>> very convincing but miss the elephant in the room, at least seems to
>>>> me. Many people have given valid inputs as where things might get
>>>> improved, including the one you are quoting.
>>>>
>>>> > Lastly, as I read the results I get with either net/http *or* the naive
>>>> > regexp-parsing, I just… disagree with your conclusions. All the numbers
>>>> > I've seen seem to imply that Go responded *on average* a lot faster and
>>>> > *might* have higher variance and a slightly higher tail latency (though,
>>>> > FTR, the measurements I got also suggest that the data is mostly noise).
>>>> > And I'm struggling to find an application, where that would matter a
>>>> > lot. Like, yeah, tail latency matters, but so does average latency. In
>>>> > basically all applications I can think of, you a) want tail latencies to
>>>> > not be catastrophic, especially when considering fan-out, but b) lower
>>>> > averages are just as important anyway. So, I think you should, for a
>>>> > decent test, formulate your criteria beforehand. Currently, there seems
>>>> > to be a lot of reading in tea-leafs involved.
>>>>
>>>> Please don't get personal and emotional over technical discussions,
>>>> and please refrain from using terms like "ridiculously simple" or
>>>> "reading in tea-leafs" in future discussions. It is inappropriately
>>>> condescending, thus not the correct attitude of communication. It
>>>> does not align with the code of conduct of this mlist, and neither of
>>>> google's, I believe.
>>>>
>>>> If you have test results that contradict with mine, please show me
>>>> yours -- Let's talk the data, and not let the emotion get in the way
>>>> of technical discussion, and fact the fact, whatever it is.
>>>>
>>>> > Anyway, all of that being said: While I don't think the test you devised
>>>> > allows the broad generalizations you are making, ultimately I don't have
>>>> > any stakes in this (which is why I haven't responded further on the blog
>>>> > post). If you like Perl and think it performs good enough or better for
>>>> > your use case - then go ahead and use Perl. No one here will begrudge
>>>> > you for it.
>>>>
>>>> As I've comment in the blog many times, I'm not trying to prove Perl
>>>> performs better than Go. On the contrary, I was trying to improve Perl
>>>> performs with Go, that's why the whole thing get started, as I have
>>>> replied to your comment in the blog:"that's why I was rewriting the
>>>> Perl code to Go".
>>>>
>>>> It is still the case, and I want to try out everyone's suggestion to
>>>> see how things can improve.
>>>>
>>>> Moreover, if you have taken a look at my second test, you will
>>>> understand my other goal is to make it clear "who and what to trust".
>>>> Because if you have taken a look at the httperf test result I posted
>>>> to this mlist before you reply, you may have realized that, of all the
>>>> performance testing tools I've used so far, all are suggesting Perl
>>>> performs better than Go, except for httperf. So maybe the performance
>>>> testing tools are biased toward Go, and I made it clear in my second
>>>> blog that I want to get to the bottom of it.
>>>>
>>>> > All of that being said
>>>> >
>>>> > IMO this is just not a meaningful test. Or its results are totally
>>>> > unsurprising.
>>>>
>>>> Again, I was trying to improve Perl performs with Go. That
>>>> "ridiculously simple" Perl code is the foundation of the Debian dbab
>>>> package, and I was trying to improve it.
>>>>
>>>> It might be meaningless to you but it is perfectly meaningful to me.
>>>> Please don't be so judgmental.
>>>>
>>>> > On Sun, Jun 9, 2019 at 4:54 AM Justin Israel <justinisr...@gmail.com>
>>>> > wrote:
>>>> >>
>>>> >> I'm wondering about a couple factors in this comparison that seem to
>>>> >> make a difference in my local test:
>>>> >>
>>>> >> I think perl sockets are write buffered. So would the equivalent be to
>>>> >> wrap the net.Conn in bufio.NewWriter(c) and flush before the Close?
>>>> >> Since this is a straigh-line test where both servers are not using
>>>> >> concurrent handling of connections (uncommon for a Go server even on 1
>>>> >> core), would it not make sense to run the Go server with GOMAXPROCS=1?
>>>> >>
>>>> >> - Justin
>>>> >>
>>>> >> On Saturday, June 8, 2019 at 1:36:49 AM UTC+12, Tong Sun wrote:
>>>> >>>
>>>> >>> I had always believed that the web projects build with Go should be
>>>> >>> much faster than Perl, since Go is a compiled language.
>>>> >>>
>>>> >>> However that belief was crushed brutally last night, when I did a
>>>> >>> comparison -- the Go implementation is 8 times worse than the Perl! --
>>>> >>> the mean response time jumped from 6ms to 48ms.
>>>> >>>
>>>> >>> I know this is the simplest possible web server, but still, when it
>>>> >>> comes to simple web servers like this, I have to say that Perl
>>>> >>> performs much better than Go.
>>>> >>>
>>>> >>> I don't think there is much I can twist on the Go side, since it can't
>>>> >>> be more simpler than that. However, I also believe it won't hurt to
>>>> >>> ask and confirm. So,
>>>> >>>
>>>> >>> Have I missed anything? Is it possible for me to make my Go
>>>> >>> implementation anywhere near the Perl's performance?
>>>> >>>
>>>> >>> Thanks
>>>> >>>
>>>> >>>
>>>> >> --
>>>> >> You received this message because you are subscribed to the Google
>>>> >> Groups "golang-nuts" group.
>>>> >> To unsubscribe from this group and stop receiving emails from it, send
>>>> >> an email to golang-nuts+unsubscr...@googlegroups.com.
>>>> >> To view this discussion on the web visit
>>>> >> https://groups.google.com/d/msgid/golang-nuts/37a1ac7e-85ee-4775-b348-5673c41a162c%40googlegroups.com.
>>>> >> For more options, visit https://groups.google.com/d/optout.
>>>> >
>>>> > --
>>>> > You received this message because you are subscribed to a topic in the
>>>> > Google Groups "golang-nuts" group.
>>>> > To unsubscribe from this topic, visit
>>>> > https://groups.google.com/d/topic/golang-nuts/iH2Ck_hpCpI/unsubscribe.
>>>> > To unsubscribe from this group and all its topics, send an email to
>>>> > golang-nuts+unsubscr...@googlegroups.com.
>>>> > To view this discussion on the web visit
>>>> > https://groups.google.com/d/msgid/golang-nuts/CAEkBMfF62D5v%2BRiGtAtzuH0wAHzCLqcwNUidC1Oe2KN9DfRv6Q%40mail.gmail.com.
>>>> > For more options, visit https://groups.google.com/d/optout.
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups
>>>> "golang-nuts" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an
>>>> email to golang-nuts+unsubscr...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/golang-nuts/CAMmz1OcefkrZN3_5R7%2BbjuwHjRx%2BLzttaiXmer0ytwmXUjxN_A%40mail.gmail.com.
>>>> For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/CAMmz1Oeed76vLx8jC7cqkq00EbHH8LsGwNbnGbqnM5Pko--www%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
