Re: [go-nuts] Go 1BRC Challenge and GC Overhead

Sorry I was wrong. I posted a correction, but it as deleted for some reason. Channels do delete the element from the circular buffer on receive . So my conclusion was wrong. The reason for t

Re: [go-nuts] Go 1BRC Challenge and GC Overhead

On Wed, Jul 24, 2024 at 2:58 PM Brendan Goldacker wrote: > > I took a look over the channels code > . > I can't say I follow it 100%, but it seems like there is a circular buffer > which would be of si

Re: [go-nuts] Go 1BRC Challenge and GC Overhead

Thanks everyone for the leads and resources. After a few more days hacking away and experimenting with changes I was able to get a solution that runs under 20s (see here ). Turns out GC wasn't a problem. Its was mostly issue

Re: [go-nuts] Go 1BRC Challenge and GC Overhead

Hi Brendan, take a look of this amazing event of Ardan Labs where they explore in depth how to analyze the GC. I think you will find your answer how the GC works and the best way to know how much goroutines vs GC memory use for your best performance. You need to sign in into Ardan Labs but it

Re: [go-nuts] Go 1BRC Challenge and GC Overhead

Hi Brendan, this is a fun problem. I'm looking at https://github.com/Goldabj/1brc-go/blob/main/cmd/brc/log_processor.go, and I suspect the main thing is that you're converting []byte to string, which almost certainly allocates. You're also scanning through the bytes several times (first to find

Re: [go-nuts] Go 1BRC Challenge and GC Overhead

On Mon, Jul 22, 2024 at 1:05 PM Brendan Goldacker wrote: > I've been using the 1 billion row challenge > as a way to learn > go over the past few days. With my limited go experience, I've seem to hit > a wall. > > I have my submission here