[go-nuts] Re: Async process state snapshotting in Golang

2018-01-30 Thread jul . semaan
As a follow-up, I managed to get everything to work as I wanted with forking but I got a major issue.. The golang runtime mutexes are invalidated by the forking, that means that anything in the runtime that needs a mutex is broken so garbage collection had to be disabled and additional memory a

[go-nuts] Re: Async process state snapshotting in Golang

2018-01-30 Thread jul . semaan
I believe copying the map containing structs and not pointers would make it consume twice the amount of memory which isn't desirable in my case. - Julien On Tuesday, 30 January 2018 00:09:21 UTC-5, Tamás Gulácsi wrote: > > Do you really need pointers tonyour structs in the map? Because that > c

[go-nuts] Re: Async process state snapshotting in Golang

2018-01-29 Thread Tamás Gulácsi
Do you really need pointers tonyour structs in the map? Because that causes the headache here: if no pointers, then simply rlock-ing the map, copying / serializing would be enough. Could even have two maps, each gets every modification, except when copying, when the secondary is serialized, thn

[go-nuts] Re: Async process state snapshotting in Golang

2018-01-29 Thread jul . semaan
My main goal is to avoid complex granular locking inside the data structure since writes are very common in the map (whether its adding items or modifying the actual item). My idea was to implement MarshalJSON for my map and then while creating the JSON I'd be holding locks at the appropriate ti

[go-nuts] Re: Async process state snapshotting in Golang

2018-01-29 Thread matthewjuran
Can you describe your performance needs in more detail? I missed the need for the struct values in the copy (obviously needed to encode to JSON). Going back to locks for a moment, my thought now is that a sync.RWMutex on this map would have an RLock taken by every struct read/write and then the

[go-nuts] Re: Async process state snapshotting in Golang

2018-01-29 Thread jul . semaan
Hi, Thanks for all the input. Unfortunately, as Tamás and Jake pointed out, I can't simply grab a copy of the map that has the same pointers since I'll need to be locking the structure everytime I'm copying it. So that means Matt's idea wouldn't really work for my use-case I was really hoping

[go-nuts] Re: Async process state snapshotting in Golang

2018-01-27 Thread jake6502
A couple of comments. This is probably obvious, but I wanted to point out that matthe...'s code would require *all *access to the map to be synchronized using the mutex. It's a simple and effective solution. One downside to his code is that it holds the lock for the duration of the Copy(). Thi

[go-nuts] Re: Async process state snapshotting in Golang

2018-01-26 Thread matthewjuran
Why not this? type StructMap map[string]*SomeStruct type SyncStructMap struct { *sync.Mutex // maybe change to *sync.RWMutex if there are mixed read/write synced operations StructMap } func (a SyncStructMap) Copy() StructMap { out := make(StructMap) a.Lock() for key, value :