On Friday, 21 April 2017 11:44:45 UTC+3, Lee Armstrong wrote: > > Thanks all. > > Jesper: I will try those things and may look to force a GC when I do my > redcuing of the map. Your explanation was very useful! > The application is processing a fast stream from RabbitMQ so yeah there is > a lot of throughput onto this map. > > Egon: Can you explain what to and why this would help? >
There are several ways of implementing a map and there are also offheap implementations (https://godoc.org/github.com/glycerine/offheap) that avoid GC completely -- at the cost of a more complicated API. For example ideas to try: 1. sharding 2. using map[string]struct 3. using map[string]int and [][1<<10]struct (and manage collection/reuse yourself) 4. if the keys don't change often you could use an atom table e.g. map[string]Atom and map[Atom]struct If you can show the minimized real-world case (or at least similar in struct sizes/types and read-write distribution) then it would be possible to suggest something more concrete. > 刘桂祥: I do currently have it as map[string]*struct, how will changing it > (in what I can see two places) make a difference. The key to a byte and > then a value in the map. > [20]byte is a value []byte contains a pointer to a value string contains a pointer to a value The grossly oversimplified version: * "Pointers make GC slower, the less pointers you have the less work GC has to do."* > Thanks again, > > Lee > > > > On Friday, April 21, 2017 at 3:39:07 AM UTC+1, 刘桂祥 wrote: >> >> try use value type >> example: >> map[string]*struct => map[[20]byte]struct >> >> 在 2017年4月20日星期四 UTC+8下午9:49:49，Lee Armstrong写道： >>> >>> See attached graph which shows the GC pauses of an application we have. >>> >>> I am frequently seeing pauses of 1-1.5 seconds. This is using Go 1.8.1 >>> and have a large map that is frequently accessed and items are removed and >>> added to it. These can be of some size. >>> >>> Is there a way to get these pauses down at all? Would forcing a GC() >>> after removing a batch of elements help at all? >>> >>> Alongside the pauses I see some substantial CPU usage showing up in >>> traces for the GC scan. >>> >>> Thanks in advance! >>> >> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
