Re: [go-nuts] Performance issue with os.File.Write

2020-12-23 Thread Matt Harden
On Tue, Dec 22, 2020 at 11:21 AM komuW wrote: > The bufio package also uses a max buffer of 64KB: > https://go.googlesource.com/go/+/go1.15.6/src/bufio/scan.go#80 > That limit is for bufio.Scanner. It doesn't have anything to do with bufio.Reader and bufio.Writer, which don't have any particular

Re: [go-nuts] Performance issue with os.File.Write

2020-12-22 Thread komuW
The bufio package also uses a max buffer of 64KB: https://go.googlesource.com/go/+/go1.15.6/src/bufio/scan.go#80 io.copybuffer on the other hand uses 32KB; https://go.googlesource.com/go/+/go1.15.6/src/io/io.go#398 On Monday, 21 December 2020 at 14:46:21 UTC+3 arn...@gmail.com wrote: > @Ben i

Re: [go-nuts] Performance issue with os.File.Write

2020-12-21 Thread Amnon
This is really more of an OS question than a Go language question. Writing single characters is going to be expensive in any language because you pay the considerable overhead of a system call/user-mode to kernel mode context switch overhead for each character. That is why the C stdio library buf

Re: [go-nuts] Performance issue with os.File.Write

2020-12-21 Thread Ben Hoyt
> @Ben interesting, I did something similar and also ended up with a 64KB buffer (seemed like the default of 4KB didn't work very well in my context). How did you decide of the buffer size? I forget how I ended up with 64KB. I think I was doing performance testing and a buffer larger than the 4KB

Re: [go-nuts] Performance issue with os.File.Write

2020-12-21 Thread Arnaud Delobelle
I agree that it's very sensible this way! My confusion stemmed for a combination of blind spots in both Go and C (in fact it was somewhere in my head that the File API provided unbuffered access, just not accessible at that time!). Thanks all for clarifying. Arnaud On Monday, 21 December 202

Re: [go-nuts] Performance issue with os.File.Write

2020-12-21 Thread Arnaud Delobelle
@Ben interesting, I did something similar and also ended up with a 64KB buffer (seemed like the default of 4KB didn't work very well in my context). How did you decide of the buffer size? Also, there is something that I don't understand. The default buffer size works really well for a toy exa

Re: [go-nuts] Performance issue with os.File.Write

2020-12-20 Thread Diego Joss
As Jan said "apples and oranges", in this case comparing *os.Stdout with C File *stdout is not fair. The equivalent of *os.Stdout in C is the filedescriptor 1 (STDOUT macro), and the equivalent of *os.Stdout.Write is write(2) (the syscall), not fwrite or fputs. If you retry your microbenchmark usin

Re: [go-nuts] Performance issue with os.File.Write

2020-12-20 Thread ben...@gmail.com
And os.Stdout (and friends) are all regular *os.File objects (which as Jan said, don't buffer). It was non-intuitive to me that stdout didn't buffer by default, because it's such a bad thing for efficiently writing lots of output, but I guess it makes sense when you want terminal output to appea

Re: [go-nuts] Performance issue with os.File.Write

2020-12-20 Thread Arnaud Delobelle
Ah, that is it, thank you! On Sunday, 20 December 2020 at 11:06:05 UTC Jan Mercl wrote: > On Sun, Dec 20, 2020 at 11:53 AM Arnaud Delobelle > wrote: > > > TLDR; a simple test program appears to show that Go's (*os.File).Write > is 10x slower than C's fputs (on MacOS). > > Apples and oranges. f

Re: [go-nuts] Performance issue with os.File.Write

2020-12-20 Thread Jan Mercl
On Sun, Dec 20, 2020 at 11:53 AM Arnaud Delobelle wrote: > TLDR; a simple test program appears to show that Go's (*os.File).Write is 10x > slower than C's fputs (on MacOS). Apples and oranges. fputs buffers, os.File does not. Rewrite the benchmark using bufio. -- You received this message bec

[go-nuts] Performance issue with os.File.Write

2020-12-20 Thread Arnaud Delobelle
Hi there! TLDR; a simple test program appears to show that Go's (*os.File).Write is 10x slower than C's fputs (on MacOS). While doing some benchmarking for my lua implementation in Go [1], I found very big differences between C Lua and and golua for benchmarks that do a lot of output to stdout