Re: [go-nuts] Re: Low memory utilization when using soft memory limit with GOGC=off

2024-01-10 Thread Zhihui Jiang
On Monday, January 8, 2024 at 7:24:27 AM UTC-8 Michael Knyszek wrote: On Sun, Jan 7, 2024 at 9:06 PM 'Zhihui Jiang' via golang-nuts < golan...@googlegroups.com> wrote: Hi Micheal, Sorry about delayed response! Please see my replies inline below. On Wednesday, December 6, 20

[go-nuts] Re: Low memory utilization when using soft memory limit with GOGC=off

2024-01-07 Thread &#x27;Zhihui Jiang' via golang-nuts
Hi Micheal, Sorry about delayed response! Please see my replies inline below. On Wednesday, December 6, 2023 at 8:22:34 PM UTC-8 Michael Knyszek wrote: On Tuesday, December 5, 2023 at 1:06:19 AM UTC-5 Zhihui Jiang wrote: Hi there, We are running a large scale recommendation system using

Re: [go-nuts] Low memory utilization when using soft memory limit with GOGC=off

2023-12-05 Thread Zhihui Jiang
Hi Andrew, Can you be more specific? We use GOMEMLIMIT to set the soft memory limit. On Tuesday, December 5, 2023 at 4:30:49 AM UTC-8 Harris, Andrew wrote: > Might be worth looking into GOMEMLIMIT. > -- > *From:* golan...@googlegroups.com on behalf &g

[go-nuts] Low memory utilization when using soft memory limit with GOGC=off

2023-12-04 Thread Zhihui Jiang
Hi there, We are running a large scale recommendation system using Golang and we are working on some GC related improvement recently. One of the changes we are trying to apply is to use soft memory limit with GOGC=off as suggested here: https://github.com/golang/go/issues/48409. But during ou

Re: [go-nuts] Suggestions on optimizing Go GC

2023-10-30 Thread Zhihui Jiang
le it. Can you explain a little bit more about spawn and destroy request handler? On Oct 30, 2023, at 11:56 PM, Zhihui Jiang wrote: Hi Michael, Jason and Robert, thanks a lot for the replies and suggestions! I did some profiling today, here are some specific findings: 1, CPUs used for GC is a

Re: [go-nuts] Suggestions on optimizing Go GC

2023-10-30 Thread Zhihui Jiang
Hi Michael, Jason and Robert, thanks a lot for the replies and suggestions! I did some profiling today, here are some specific findings: 1, CPUs used for GC is around 35% after we enabled soft memory limit, and it was 45%+ before. I don't have too much experience here on how much CPU we should s

[go-nuts] Suggestions on optimizing Go GC

2023-10-29 Thread Zhihui Jiang
Hi there, We have a large-scale recommendation system serving millions of users which is built using Golang. It has worked well until recently when we are trying to enlarge our index or candidate pool by 10X in which case the number of candidate objects created to serve each user request can al