Feature Requests item #1453341, was opened at 2006-03-18 20:52
Message generated for change (Comment added) made by gfe
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1453341&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
Status: Closed
Resolution: Rejected
Priority: 5
Submitted By: kxroberto (kxroberto)
Assigned to: Nobody/Anonymous (nobody)
Summary: sys.setatomicexecution - for critical sections
Initial Comment:
In order to maintain threaded code uncomplicated (VHL)
and in order to avoid massive use of locks, when
introducing few critical non-atomic sections, it should
be possible to put out that practical hammer ..
try:
last_ci=sys.setcheckinterval(sys.maxint)
critical_function() # now runs atomically
finally:
sys.setcheckinterval(last_ci)
#(sys.setcheckinterval assumed to return the last
value)
..by an official function/class (in sys or thread):
Maybe:
======
atomic = sys.setatomicexecution(mode)
try:
print "Executing critical section"
finally:
sys.setatomicexecution(atomic)
there should be different modes/levels for blocking :
* 0=None
* 1=block only python execution in other threads
* 2=block signals
* 4/5/7/15=block threads at OS level (if OS supports)
* 8=(reserved: block future sub-/stackless switching
inside current thread..)
see:
http://groups.google.de/group/comp.lang.python/browse_frm/thread/bf5461507803975e/3bd7dfa9422a1200
compare:
http://www.stackless.com/wiki/Tasklets#critical-sections
---
Also Python could define officially its time atoms
beyond CPU atoms in the docs (That also defines the VHL
usability of the language).
Thus thinks like single-element changes ...
obj.var=x , d[key]=x , l.append(x) .pop()
should be guaranteed to work atomic/correct FOR EVER.
l[slice], d.keys() .items(), .. questionable?
If not guaranteed for the future, offer duplicates for
speed critical key building blocks like:
l.slice_atomic(slice), d.keys_atomic() ,... in order
to make code compatible for the future.
---
Extra fun for blowing up python libs for poeple who
don't want to learn that try..finally all the time:
copy/deepcopy/dump maybe could be enriched by
copy_atomic , deepcopy_atomic, dump_atomic - or just
RuntimeError tolerant versions, deepcopy_save (no
use of .iterxxx)
----------------------------------------------------------------------
Comment By: Martin Gfeller (gfe)
Date: 2006-05-29 15:06
Message:
Logged In: YES
user_id=884167
Should I enter a separate feature request for this,
separating it from this closed item?
Martin
----------------------------------------------------------------------
Comment By: Martin Gfeller (gfe)
Date: 2006-05-29 15:03
Message:
Logged In: YES
user_id=884167
Nick Coghlan wrote:
> Python level access to the Global Interpreter Lock is
neither necessary nor desirable.
I agree. But what I think would be very useful (as I
already mentioned below) is C level API access that tells
Python not to relinquish the GIL during a callback.
I don't know of any other way to achieve this, apart from
introducing a separate C-level locking mechanism.
I think Tim Peters thinks thought this is feasible when he
wrote the following (private conversation, June 2005):
> It would be more feasible to patch Python, introducing
> a way to tell the interpreter not to allow other
> _Python_ threads to run. It has such a mechanism
> already (the GIL -- the global interpreter lock), but
> as soon as C code calls back into Python, the interpreter
> may release the GIL and give ownership of it to a
> different thread. I think it would be pretty easy to
> introduce a Python C API to say "don't allow other Python
> threads to run until I tell you otherwise".
Best regards, Martin Gfeller
----------------------------------------------------------------------
Comment By: Nick Coghlan (ncoghlan)
Date: 2006-04-08 20:34
Message:
Logged In: YES
user_id=1038590
On the other changes you suggest (which Raymond didn't bring
up on python-dev):
Python can't formally define as atomic any operations that
may execute arbitrary Python code, as the interpreter cannot
control what that code may do. All of the examples you give
are in that category.
Slowing down the common cases (unthreaded code and
Queue-based threaded code) by adding internal locking to
every data structure is also considered highly undesirable.
----------------------------------------------------------------------
Comment By: Nick Coghlan (ncoghlan)
Date: 2006-04-08 20:17
Message:
Logged In: YES
user_id=1038590
Raymond brought this idea up on python-dev at the time of
the c.l.p discussion - it was rejected on the basis that
thread synchronisation tools (Queues and Locks) are provided
for a reason. Python level access to the Global Interpreter
Lock is neither necessary nor desirable.
Avoiding the tools provided to permit threading to work
correctly and then finding that threaded code doesn't work
as desired really shouldn't be surprising.
FWIW, Python 2.5 aims to make normal locking easier to use
by permitting:
from __future__ import with_statement
from threading import Lock
sync_lock = Lock()
def my_func(*args, **kwds):
with sync_lock:
# Only one thread at a time can enter this section
# regardless of IO or anything else
# This section, on the other hand, is a free-for-all
If you genuinely have to avoid normal thread synchronisation
primitives, you can abuse (and I really do mean abuse) the
interpreter's import lock for this purpose:
imp.acquire_lock()
try:
print 'critical section'
finally:
imp.release_lock()
Or even:
@contextlib.contextmanager
def CriticalSection()
imp.acquire_lock()
try:
yield
finally:
imp.release_lock()
with CriticalSection():
print 'critical section'
----------------------------------------------------------------------
Comment By: kxroberto (kxroberto)
Date: 2006-03-20 16:28
Message:
Logged In: YES
user_id=972995
... only PyEval_RestoreThread with the harder execution
level in its tstate
----------------------------------------------------------------------
Comment By: kxroberto (kxroberto)
Date: 2006-03-20 16:24
Message:
Logged In: YES
user_id=972995
thus the GIL could simply have a harder state 2 : "locked
hard for PyEval_AcquireThread/PyEval_AcquireLock/.." ?
Only PyEval_RestoreThread gets the lock again after
PyEval_SaveThread.
Robert
----------------------------------------------------------------------
Comment By: Martin Gfeller (gfe)
Date: 2006-03-20 13:17
Message:
Logged In: YES
user_id=884167
- sys.setcheckinterval(sys.maxint) does not prevent
thread switching when doing IO, does it? There is
no way that I know of to prevent thread switching
in this situation.
- When calling back into Python from C Code, there is
no way to tell Python not to relinquish the GIL; such
an option could be useful for some C code, because
the GIL provides a "global critical section".
Martin
----------------------------------------------------------------------
Comment By: kxroberto (kxroberto)
Date: 2006-03-18 22:54
Message:
Logged In: YES
user_id=972995
or:
>>> atomic = sys.acquireatomicexecution(mode)
>>> try:
... print 'critical section'
... finally:
... atomic.release()
>>> with sys.acquireatomicexecution(mode):
... print 'critical section'
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1453341&group_id=5470
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
