Sorry I was wrong. I posted a correction, but it as deleted for some reason.
Channels do delete the element from the circular buffer on receive <https://github.com/golang/go/blob/4fc9565ffce91c4299903f7c17a275f0786734a1/src/runtime/chan.go#L522>. So my conclusion was wrong. The reason for the heap growth is that the producer go routine can send chunks faster than the consumer go routines can process them. So by keeping the channel's buffer size smaller we essentially rate limit the producer and keep the heap bounded. Additionally, I learned that an unbuffered or 'full' channel has some slight performance benefit. A buffered channel a send requires copying the chunk to the channels queue, then a receive will copy it from the channel's queue to the consumers stack. However, if the buffer is full (or unbuffered), then send will copy the item directly to the receivers' stack (if there is a receiver waiting). Which also reduces heap usage. Thanks, Brendan On Wednesday, July 24, 2024 at 5:55:04 PM UTC-5 Ian Lance Taylor wrote: > On Wed, Jul 24, 2024 at 2:58 PM Brendan Goldacker <brend...@gmail.com> > wrote: > >> >> I took a look over the channels code >> <https://github.com/golang/go/blob/4fc9565ffce91c4299903f7c17a275f0786734a1/src/runtime/chan.go>. >> >> I can't say I follow it 100%, but it seems like there is a circular buffer >> which would be of size 128 in the larger buffer case. Then there are two >> offset pointers for the heads and tails of the queue. It seems like the >> heap grows unbounded since when receiving from the channel, it doesn't >> actually delete the item form the circular queue. Instead it just >> increments our head pointer. Therefore, a channel holds a reference to a >> chunk buffer (up to 128 of them) even if they've already been read/received >> by a worker. Thus, these 'old' buffers cannot be GC'ed. The reference is >> only removed from the circular queue once the channel gets enough sends to >> overwrite the slot. >> > > That's interesting. Would you mind filing an issue at > https://go.dev/issue so that the runtime team can take a look? If you > are describing the issue correctly--I have no reason to think you > aren't--perhaps it would be relatively cheap to zero out the entry in the > channel buffer after the value is delivered to a goroutine. Thanks. > > Ian > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/d3a787e0-0f09-411b-a754-4aba6d9104f4n%40googlegroups.com.
