On Nov 15, 1:03 am, de...@web.de (Diez B. Roggisch) wrote: > You don't say what data you share, and if all of it is needed for each > child. So it's hard to suggest optimizations.
Here is an example of such a problem I'm dealing with now: I'm building large index of words in memory, it takes 50% of RAM. After building it I want to enable querying it using multiple processors enabling fast handling of multiple clients at once (communication is done with sockets). So I create the index in the parent process and than fork children that handle client queries. But after a short time only traversing the index by children makes them use much memory, resulting shortly in Out Of Memory Errors. > And AFAIK there is no > built-in way of doing what you want. It's complex and error-prone. Having read-only objects is complex and error-prone? > Maybe mmap + (struct|pickle) help, if what you need can be formulated in a way > that traversing the whole data piecewise by explicitly > marshaling-demarshaling data? In case of this word index it is impossible... And probably would be too slow. Using mmap and mapping the object structure into structs should probably work... -- http://mail.python.org/mailman/listinfo/python-list
