Re: Copy-on-write when forking a python process

2011-04-12 Thread jac
Heiko, Thank you for pointing out POSH. I have used some of python's other shared memory facilities, but was completely unaware of POSH, it seems nice. Also, I agree that shared memory would solve the use-case I outlined above, but it is not hard to imagine a slightly different case where the chil

Re: Copy-on-write when forking a python process

2011-04-08 Thread Heiko Wundram
Am 08.04.2011 20:34, schrieb jac: > I disagree with your statement that COW is an optimization for a > complete clone, it is an optimization that works at the memory page > level, not at the memory image level. In other words, if I write to a > copy-on-write page, only that page is copied into my

Re: Copy-on-write when forking a python process

2011-04-08 Thread jac
Hi Heiko, I just realized I should probably have put a clearer use-case in my previous message. A example use-case would be if you have a parent process which creates a large dictionary (say several gigabytes). The process then forks several worker processes which access this dictionary. The wor

Re: Copy-on-write when forking a python process

2011-04-08 Thread Heiko Wundram
Am 08.04.2011 18:14, schrieb John Connor: > Has anyone else looked into the COW problem? Are there workarounds > and/or other plans to fix it? Does the solution I am proposing sound > reasonable, or does it seem like overkill? Does anyone foresee any > problems with it? Why'd you need a "fix" l