Hi Antoine On Dec 11, 3:00 pm, Antoine Pitrou <solip...@pitrou.net> wrote: > I was going to suggest memcached but it probably serializes non-atomic > types. It doesn't mean it will be slow, though. Serialization implemented > in C may well be faster than any "smart" non-serializing scheme > implemented in Python.
No serializing could be faster than NO serializing at all :) If child process could directly read the parent RAM -- what could be better? > What do you call "problems because of the GIL"? It is quite a vague > statement, and an answer would depend on your OS, the number of threads > you're willing to run, and whether you want to extract throughput from > multiple threads or are just concerned about latency. it seems to be a known fact, that only one CPython iterpreter will be running at a time, because a thread is aquiring the GIL during the execution and other threads within same process are then just waiting for GIL to be released. > In any case, you have to do some homework and compare the various > approaches on your own data, and decide whether the numbers are > satisfying to you. well, I the least evil is to pack-unpack things into array.array and/ or similarly NumPy. I do hope that Klauss' patch will be accepted, because it will let me to forget a lot of those unneeded packing-unpacking. > > I am a big fan of parallel map() approach > > I don't see what map() has to do with accessing data. map() is for > *processing* of data. In other words, whether or not you use a map()-like > primitive does not say anything about how the underlying data should be > accessed. right. However, saying "a big fan" has had another focus here: if you write your code based on maps then you have a tiny effort to convert your code into a MULTIprocessing one :) just that. Kind regards. Valery -- http://mail.python.org/mailman/listinfo/python-list
