robert <[EMAIL PROTECTED]> wrote: ... > What? When I add/del an item to a dict or list, this is not an atomic > thread-safe operation?
Exactly: there is no such guarantee in the Python language. > E.g.: > One thread does things like d['x']='y' > Another thread reads d['z'] or sets d['z']='w' or dels something. > > If those operations are not atomic, then you'd have to use locks all the > time to not get RuntimeErrors and worse !? If you want to be writing correct Python, yes. A preferred approach is to simply avoid sharing objects among threads, except for objects designed to be thread-safe (chiefly Queue.Queue). > Infact I rely on that all the time and standard python modules also do > so AFAIK You're relying on an accident of a specific, particular implementation; if any Python-coded standard library module does likewise, and I'm not aware of any, that's somewhat different (since that module is PART of the implementation, it may rely on all kinds of implementation details, correctly if maybe not wisely). The situation is quite different for C-coded modules in the CPython implementation, Java-coded ones in the Jython one, C#-coded one in the IronPython one; each of these is subject to specific constraints that it's perfectly wise to rely on (since each implementation, as the language specification fully allows it to do, adopts a different locking strategy at these low levels). > I think cPickle has not necessarily to iterate free over native dicts. It's not forced to by language specification, but neither is it forbidden. It would be an absurd implementation strategy to waste time and space to extract a dict's keys() first, as it would NOT buy "atomicity" anyway -- what if some other thread deletes keys while you're looping, or calls any nonatomic method on the very value you're in the process of serializing?! In some Python implementations, a C-coded module may count on some atomicity as long as it doesn't explicitly allow other threads nor ever call back into ANY python-coded part, but obviously cpickle cannot avoid doing that, so even in those implementations it will never be atomic. > Whats does copy/deepcopy/[:] ? Roughly the same situation. If as you indicate you want to stick with a Python-like language but do not want to change your style to make it correct Python, you could perhaps fork the implementation into an "AtomicPython" in which you somehow fix all nonatomicities (not sure how that would even be possible if pickling, deepcopying or anything else ever needs to fork into Python coded parts, but perhaps you might make the GIL into a reentrant lock and somehow hack it to work, with some constraints). Or perhaps you might be able to write an extension containing atomicset, atomicget, atomicpickle, and other operations you feel you need to be atomic (again barring the difficulties due to possible callbacks into Python) and use those instead of bare Python primitives. Alex -- http://mail.python.org/mailman/listinfo/python-list
