The bufio package also uses a max buffer of 64KB: https://go.googlesource.com/go/+/go1.15.6/src/bufio/scan.go#80 io.copybuffer on the other hand uses 32KB; https://go.googlesource.com/go/+/go1.15.6/src/io/io.go#398
On Monday, 21 December 2020 at 14:46:21 UTC+3 arn...@gmail.com wrote: > @Ben interesting, I did something similar and also ended up with a 64KB > buffer (seemed like the default of 4KB didn't work very well in my > context). How did you decide of the buffer size? > > Also, there is something that I don't understand. The default buffer size > works really well for a toy example (like the one I posted in my original > question), but the improvements are much less dramatic in my program, for > the same pattern of writing to stdout. I can't work out a reason for this > yet. > > > On Sunday, 20 December 2020 at 20:31:52 UTC ben...@gmail.com wrote: > >> And os.Stdout (and friends) are all regular *os.File objects (which as >> Jan said, don't buffer). It was non-intuitive to me that stdout didn't >> buffer by default, because it's such a bad thing for efficiently writing >> lots of output, but I guess it makes sense when you want terminal output to >> appear right away. So I realized it made sense, and gives you more control. >> And it's so easy to wrap it in a bufio.NewWriter() ... Flush() if you need >> buffering. >> >> I ran into this exact same issue when implementing GoAWK ... a 10-line >> fix gave me a 10x speedup. >> https://github.com/benhoyt/goawk/commit/60745c3503ba3d99297816f5c7b5364a08ec47ab >> >> -Ben >> >> On Monday, December 21, 2020 at 12:27:43 AM UTC+13 arn...@gmail.com >> wrote: >> >>> Ah, that is it, thank you! >>> >>> On Sunday, 20 December 2020 at 11:06:05 UTC Jan Mercl wrote: >>> >>>> On Sun, Dec 20, 2020 at 11:53 AM Arnaud Delobelle <arn...@gmail.com> >>>> wrote: >>>> >>>> > TLDR; a simple test program appears to show that Go's >>>> (*os.File).Write is 10x slower than C's fputs (on MacOS). >>>> >>>> Apples and oranges. fputs buffers, os.File does not. Rewrite the >>>> benchmark using bufio. >>>> >>> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/f98810f0-9227-4002-a272-d458e21dcf2bn%40googlegroups.com.
