Re: [Python-Dev] New relative import issue
Hi Fabio, On Sun, Sep 17, 2006 at 03:38:42PM -0300, Fabio Zadrozny wrote: > I've been playing with the new features and there's one thing about > the new relative import that I find a little strange and I'm not sure > this was intended... My (limited) understanding of the motivation for relative imports is that they are only here as a transitional feature. Fully-absolute imports are the official future. Neither relative nor fully-absolute imports address the fact that in any multi-package project I've been involved with, there is some kind of sys.path hackery required (or even custom import hooks). Indeed, there is no clean way from a test module 'foo.bar.test.test_hello' to import 'foo.bar.hello': the top-level directory must first be inserted into sys.path magically. > /foo/bar/imp1.py <-- has a "from . import imp2" > /foo/bar/imp2.py > > if I now put a test-case (or any other module I'd like as the main module) at: > /foo/bar/mytest.py > > if it imports imp1, it will always fail. Indeed: foo/bar/mytest.py must do 'import foo.bar.imp1' or 'from foo.bar import imp1', and then it works (if sys.path was properly hacked first, of course). (I'm not sure, but I think that this not so much a language design decision as a consequence of the complexities of import.c, which is the largest C source file of CPython and steadily growing.) A bientot, Armin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New relative import issue
Fabio Zadrozny wrote: > I've been playing with the new features and there's one thing about > the new relative import that I find a little strange and I'm not sure > this was intended... > > When you do a from . import xxx, it will always fail if you're in a > top-level module, and when executing any module, the directory of the > module will automatically go into the pythonpath, thus making all the > relative imports in that structure fail. Correct. Relative imports are based on __name__ and don't work properly if __name__ does not properly reflect the module's position in the package hierarchy (usually because the module is the main module, so name is set to '__main__'). This is noted briefly in PEP 328 [1], with the current workarounds explained in more detail in PEP 338 [2]. Cheers, Nick. [1] http://www.python.org/dev/peps/pep-0328/#relative-imports-and-name [2] http://www.python.org/dev/peps/pep-0338/#import-statements-and-the-main-module -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Before 2.5 - More signed integer overflows
Neal Norwitz schrieb: > I'm getting a crash when running test_builtin and test_calendar (at > least) with gcc 4.1.1 on amd64. It's happening in pymalloc, though I > don't know what the cause is. I thought I tested with gcc 4.1 before, > but probably would have been in debug mode. Can't really check right now, but it might be that this is just the limitation that a debug obmalloc doesn't work on 64-bit systems. There is a header at each block with a fixed size of 4 bytes, even though it should be 8 bytes on 64-bit systems. This header is there only in a debug build. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] deja-vu .. python locking
Hello,
as someone has written in FAQ, sometimes someone starts a thread about
finer grained locking in Python.
Ok here is one.
I don't want to start a flamewar. I only seek suggestions and constructive
critic. I have some ideas whose are new in this context (I believe) and
I only wanted to make them public in case someone finds them interesting.
Comments are welcome.
Martin
Round 1, Greg Stein's patch
The patch removes GIL from version 1.6 and replaces locking of
list, dict and other structures with finer grained locking.
The major slowdown seems to be in list and dict structures, dicts
are used for object attributes and these are accessed quite often.
Because (IIUC) mutex struct is quite heavy, dict and list are
locked via pool of locks. When you lock this pooled lock you
have to lock two locks in reality. One locks pool itself, and other
locks the pooled lock (the second locking can be omited in non
contended case because locks in the pool are in locked state).
One lock take about 25 cycles on UP P4 (mainly pipeline flush
during memory barrier) and can be even more expensive (hundreds
of cycles) due to cacheline move between CPUs on SMP machine.
"Global" pool lock is subject to cacheline pinpong as it will
be often reacquired by competing CPUs.
In mappinglookup there is lookmapping guarded by this locking scheme,
lookmapping itself has about 20 cycles in the best (one hope typical) case
plus compareobj cost (in case of string keys say ... 50..100 cycles?).
Thus locking/unlocking the read takes 50..100 cycles and operation
itself is 70-120 cycles.
One might expect about 50% slowdown in dict read path.
RCU like locking
Solution I have in mind is similar to RCU. In Python we have quiscent
state - when a thread returns to main loop of interpreter.
Let's add "owner_thread" field to locked object. It reflects last thread
(its id) which called any lockable method on the object.
Each LOCK operation looks like:
while (ob->owner_thread != self_thread()) {
unlock_mutex(thread_mutex[self_thread()])
// wait for owning thread to go to quiscent state
lock_mutex(thread_mutex[ob->owner_thread])
ob->owner_thread = self_thread()
unlock_mutex(thread_mutex[ob->owner_thread])
lock_mutex(thread_mutex[self_thread()])
}
Unlock is not done - we own the object now and can use it without locking
(until we return to interpreter loop or we call LOCK on other object).
For non-shared objects there is only penalty of ob->owner_thread !=
self_thread()
condition. Not sure about Windows, but in recent Linuxes one can use %gs
register
as thread id, thus compare is about 3 cycles (and owner_thread should be
in object's cacheline anyway).
In contended case there is some cache pingpong with ob and mutex but it is
as expected.
Deadlocks
Our object ownership is long - from getting it in LOCK to next quiscent state
of the thread. Thus when two threads want to step each on other's object, they
will deadlock. Simple solution is to extend set of quiscent states.
It is when thread releases its thread_mutex in main loop (and immediately
reacquires). Additionaly it can release it just before it is going to wait
on another thread's mutex, like in LOCK (already in code above). If you use
LOCK correctly then when you are LOCKing an object you can't be in vulnerable
part of OTHER object. So that let other threads to get ownership of your own
objects in that time.
One can also want to release his lock when going to lock mutex in threading
package and in other places where GIL is released today.
However I admit that I did no formal proof regarding deadlock, I plan
to do it if nobody can find other flaw in the proposal.
Big reader lock
While above scheme might work well, it'd impose performance penalty
for shared dicts which are almost read only (module.__dict__).
For these similar locking can be used, only writer has to wait until
ALL other threads enter quiscent state (take locks of them), then perform
change and unlock them all. Readers can read without any locking.
Compatibilty with 3rd party modules
I've read this argument on pydev list. Maybe I'm not understanding something,
but is it so complex for Py_InitModule4 to use extra flag in apiver for
example ?
When at least one non-freethreaded module is loaded, locking is done in
old good way...
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Testsuite fails on Windows if a space is in the path
Jean-Paul Calderone schrieb: > You can find the quoting/dequoting rules used by cmd.exe documented on msdn: > > http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vclang/html/_pluslang_Parsing_C.2b2b_.Command.2d.Line_Arguments.asp > > Interpreting them is something of a challenge (my favorite part is how the > examples imply that the final argument is automatically uppercased ;) That doesn't talk about cmd.exe, does it? It rather looks like the procedure used to create argc/argv when calling main() in the C run-time library. If cmd.exe would use these rules, the current Python code should be fine, AFAICT. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Testsuite fails on Windows if a space is in the path
Tim Peters schrieb:
> These are the MS docs for cmd.exe's inscrutable quoting rules after /C:
>
> """
> If /C or /K is specified, then the remainder of the command line after
> the switch is processed as a command line, where the following logic is
> used to process quote (") characters:
>
> 1. If all of the following conditions are met, then quote characters
> on the command line are preserved:
I couldn't make sense of the German translation; reading over the
English version several times, I think I now understand what it does
(not that I truly understand *why* it does that, probably because too
many people complained that it would strip off quotes when the
program name had a space in it).
> I personally wouldn't change anything here for 2.5. It's a minefield,
> and people who care a lot already have their own workarounds in place,
> which we'd risk breaking. It remains a minefield for newbies, but
> we're really just passing on cmd.exe's behaviors.
So what do you suggest for 2.6? "Fix" it (i.e. make sure that the
target process is invoked with the same command line that is
passed to popen)? Or leave it as-is, just documenting the limitations
better. It's non-obvious that popen uses %COMSPEC% /c.
(Another problem is that the error message from cmd.exe gets discarded;
that should get fixed regardless)
> People are well-advised to accept the installer's default directory.
That's very true, but difficult to communicate. Too many people actually
complain about that, and some even bring reasonable arguments (such
as the ACL in c:\ being too permissive for a software installation).
Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] deja-vu .. python locking
Martin Devera schrieb: > RCU like locking > Solution I have in mind is similar to RCU. In Python we have quiscent > state - when a thread returns to main loop of interpreter. There might be a terminology problem here. RCU is read-copy-update, right? I fail to see the copy (copy data structure to be modified) and update (replace original pointer with pointer to copy) part. Do this play a role in that scheme? If so, what specific structure is copied for, say, a list or a dict? This confusion makes it very difficult for me to understand your proposal, so I can't comment much on it. If you think it could work, just go ahead and create an implementation. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] deja-vu .. python locking
Martin v. Löwis wrote: > Martin Devera schrieb: >> RCU like locking >> Solution I have in mind is similar to RCU. In Python we have quiscent >> state - when a thread returns to main loop of interpreter. > > There might be a terminology problem here. RCU is read-copy-update, > right? I fail to see the copy (copy data structure to be modified) > and update (replace original pointer with pointer to copy) part. > Do this play a role in that scheme? If so, what specific structure > is copied for, say, a list or a dict? > > This confusion makes it very difficult for me to understand your > proposal, so I can't comment much on it. If you think it could work, > just go ahead and create an implementation. It is why I used a word "similar". I see the similarity in a way to archieve safe "delete" phase of RCU. Probably I selected bad title for the text. It is because I was reading about RCU implementation in Linux kernel and I discovered that the idea of postponing critical code to some safe point in future might work in Python interpreter. So that you are right. It is not RCU. It only uses similar technique as RCU uses for free-ing old copy of data. It is based on assumption that an object is typicaly used by single thread. You must lock it anyway just for case if another thread steps on it. The idea is that each object is "owned" by a thread. Owner can use its objects without locking. If a thread wants to use foreign object then it has to wait for owning thread to go to some safe place (out of interpreter, into LOCK of other object..). It is done by per-thread lock and it is neccessary because owner does no locking, thus you can be sure that nobody it using the object when former owner is somewhere out of the object. Regarding implementation, I wanted to look for some opinions before starting to implement something as big as this patch. Probably someone can look and say, hey it is stupit, you forgot that FILL_IN ... ;-) I hope I explained it better this time, I know my English not the best. At least worse than my Python :-) thanks for your time, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] deja-vu .. python locking
On Mon, 18 Sep 2006 17:06:47 +0200, Martin Devera <[EMAIL PROTECTED]> wrote: >Martin v. Löwis wrote: >> Martin Devera schrieb: >>> RCU like locking >>> Solution I have in mind is similar to RCU. In Python we have quiscent >>> state - when a thread returns to main loop of interpreter. >> >> There might be a terminology problem here. RCU is read-copy-update, >> right? I fail to see the copy (copy data structure to be modified) >> and update (replace original pointer with pointer to copy) part. >> Do this play a role in that scheme? If so, what specific structure >> is copied for, say, a list or a dict? >> >> This confusion makes it very difficult for me to understand your >> proposal, so I can't comment much on it. If you think it could work, >> just go ahead and create an implementation. > >It is why I used a word "similar". I see the similarity in a way to archieve >safe "delete" phase of RCU. Probably I selected bad title for the text. It >is because I was reading about RCU implementation in Linux kernel and >I discovered that the idea of postponing critical code to some safe point in >future might work in Python interpreter. > >So that you are right. It is not RCU. It only uses similar technique as RCU >uses for free-ing old copy of data. > >It is based on assumption that an object is typicaly used by single thread. Which thread owns builtins? Or module dictionaries? If two threads are running the same function and share no state except their globals, won't they constantly be thrashing on the module dictionary? Likewise, if the same method is running in two different threads, won't they thrash on the class dictionary? Jean-Paul ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Before 2.5 - More signed integer overflows
[Neal Norwitz] >> I'm getting a crash when running test_builtin and test_calendar (at >> least) with gcc 4.1.1 on amd64. It's happening in pymalloc, though I >> don't know what the cause is. I thought I tested with gcc 4.1 before, >> but probably would have been in debug mode. Neil, in context it was unclear whether you were using trapv at the time. Were you? [Martin v. Löwis] > Can't really check right now, but it might be that this is just the > limitation that a debug obmalloc doesn't work on 64-bit systems. > There is a header at each block with a fixed size of 4 bytes, even > though it should be 8 bytes on 64-bit systems. This header is there > only in a debug build. Funny then how all the 64-bit buildbots manage to pass running debug builds ;-) As of revs 46637 + 46638 (3-4 months ago), debug-build obmalloc uses sizeof(size_t) bytes for each of its header and trailer debugging fields. Before then, the debug-build obmalloc was "safe" in this respect: if it /needed/ to store more than 4 bytes in a debug bookkeeping field, it assert-failed in a debug build. That would happen if and only if a call to malloc/realloc requested >= 2**32 bytes, so was never provoked by Python's test suite. As of rev 46638, that limitation should have gone away. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] deja-vu .. python locking
>> So that you are right. It is not RCU. It only uses similar technique as RCU >> uses for free-ing old copy of data. >> >> It is based on assumption that an object is typicaly used by single thread. > > Which thread owns builtins? Or module dictionaries? If two threads are > running the same function and share no state except their globals, won't > they constantly be thrashing on the module dictionary? Likewise, if the > same method is running in two different threads, won't they thrash on the > class dictionary? As I've written in "Big reader lock" paragraph of the original proposal, these objects could be handled by not blocking in read path and wait for all other threads to "come home" before modifying. The selection between locking mode could be selected either by something like __locking__ or by detecting the mode. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Before 2.5 - More signed integer overflows
On 9/18/06, Tim Peters <[EMAIL PROTECTED]> wrote: > [Neal Norwitz] > >> I'm getting a crash when running test_builtin and test_calendar (at > >> least) with gcc 4.1.1 on amd64. It's happening in pymalloc, though I > >> don't know what the cause is. I thought I tested with gcc 4.1 before, > >> but probably would have been in debug mode. > > Neil, in context it was unclear whether you were using trapv at the > time. Were you? No trapv, just ./configure --without-pydebug IIRC. I should have sent a msg last night, but was too tired. I got the same crash (I think) with gcc 3.4.4, so it's almost definitely due to an outstanding change, not python's or gcc's fault. n ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Testsuite fails on Windows if a space is in the path
Martin v. Löwis wrote: >> People are well-advised to accept the installer's default directory. > > That's very true, but difficult to communicate. Too many people > actually > complain about that, and some even bring reasonable arguments (such > as the ACL in c:\ being too permissive for a software installation). Besides, it won't be allowed in Vista with the default user permissions. -- Giovanni Bajo ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] deja-vu .. python locking
At 07:08 PM 9/18/2006 +0200, Martin Devera wrote: > >> So that you are right. It is not RCU. It only uses similar technique > as RCU > >> uses for free-ing old copy of data. > >> > >> It is based on assumption that an object is typicaly used by single > thread. > > > > Which thread owns builtins? Or module dictionaries? If two threads are > > running the same function and share no state except their globals, won't > > they constantly be thrashing on the module dictionary? Likewise, if the > > same method is running in two different threads, won't they thrash on the > > class dictionary? > >As I've written in "Big reader lock" paragraph of the original proposal, these >objects could be handled by not blocking in read path and wait for all other >threads to "come home" before modifying. Changing an object's reference count is modifying it, and most accesses to get the dictionaries themselves involve refcount changes. Your plan, so far, does not appear to have any solution for reducing this overhead. Module globals aren't so bad, in that you'd only have to lock and refcount when frames are created and destroyed. But access to class dictionaries to obtain methods happens a lot more often, and refcounting is involved there as well. So, I think for your plan to work, you would have to eliminate reference counting, in order to bring the lock overhead down to a manageable level. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] deja-vu .. python locking
Martin Devera schrieb:
> It is based on assumption that an object is typicaly used by single
> thread. You must lock it anyway just for case if another thread steps
> on it. The idea is that each object is "owned" by a thread. Owner can
> use its objects without locking. If a thread wants to use foreign
> object then it has to wait for owning thread to go to some safe place
> (out of interpreter, into LOCK of other object..). It is done by
> per-thread lock and it is neccessary because owner does no locking,
> thus you can be sure that nobody it using the object when former
> owner is somewhere out of the object.
Ah, I think I understand now. First the minor critique: I believe
the locking algorithm isn't thread-safe:
while (ob->owner_thread != self_thread()) {
unlock_mutex(thread_mutex[self_thread()])
// wait for owning thread to go to quiscent state
lock_mutex(thread_mutex[ob->owner_thread])
ob->owner_thread = self_thread()
unlock_mutex(thread_mutex[ob->owner_thread])
lock_mutex(thread_mutex[self_thread()])
}
If two threads are competing for the same object held by a third
thread, they may simultaneously enter the while loop, and then
simultaneously try to lock the owner_thread. Now, one will win,
and own the object. Later, the other will gain the lock, and
unconditionally overwrite ownership. This will cause two threads
to own the objects, which is an error.
The more fundamental critique is: Why? It seems you do this
to improve efficiency, (implicitly) claiming that it is
more efficient to keep holding the lock, instead of releasing
and re-acquiring it each time.
I claim that this doesn't really matter: any reasonable
mutex implementation will be "fast" if there is no lock
contention. On locking, it will not invoke any system
call if the lock is currently not held (but just
atomically test-and-set some field of the lock); on
unlocking, it will not invoke any system call if
the wait list is empty. As you also need to test, there
shouldn't be much of a performance difference.
Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New relative import issue
Armin Rigo wrote: > My (limited) understanding of the motivation for relative imports is > that they are only here as a transitional feature. Fully-absolute > imports are the official future. Guido does seem to have a dislike for relative imports, but I don't really understand why. The usefulness of being able to make a package self-contained and movable to another place in the package hierarchy without hacking it seems self-evident to me. What's happening in Py3k? Will relative imports still exist? > there > is no clean way from a test module 'foo.bar.test.test_hello' to import > 'foo.bar.hello': the top-level directory must first be inserted into > sys.path magically. I've felt for a long time that problems like this wouldn't arise so much if there were a closer connection between the package hierarchy and the file system structure. There really shouldn't be any such thing as sys.path -- the view that any given module has of the package namespace should depend only on where it is, not on the history of how it came to be invoked. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New relative import issue
Greg Ewing <[EMAIL PROTECTED]> wrote: > Armin Rigo wrote: > > there > > is no clean way from a test module 'foo.bar.test.test_hello' to import > > 'foo.bar.hello': the top-level directory must first be inserted into > > sys.path magically. > > I've felt for a long time that problems like this > wouldn't arise so much if there were a closer > connection between the package hierarchy and the > file system structure. There really shouldn't be > any such thing as sys.path -- the view that any > given module has of the package namespace should > depend only on where it is, not on the history of > how it came to be invoked. Wait, wait, wait. If I remember correctly, one of the use-cases cited was for sub-packages of a single larger package to be able to import other sub-packages, via 'from ..subpackage2 import module2'. That is to say, given a package structure like... .../__init__.py .../subpackage1/module1.py .../subpackage1/__init__.py .../subpackage2/module2.py .../subpackage2/__init__.py Running module1.py, with an import line that read: from ..subpackage2 import module2 ... would import module2 from subpackage2 Testing this in the beta I have installed tells me: Traceback (most recent call last): File "module1.py", line 1, in from ..subpackage2 import module2 ValueError: Relative importpath too deep While I can understand why this is the case (if one is going to be naming modules relative to __main__ or otherwise, unless one preserves the number of leading '.', giving module2 a __name__ of __main__..subpackage2.module2 or ..subpackage2.module2, naming can be confusing), it does remove a very important feature. Guido suggested I make up a PEP way back in March or so, but I was slowed by actually implementing __main__-relative naming (which is currently incomplete). As it stands, in order to "work around" this particular feature, one would need to write a 'loader' to handle importing and/or main() calling in subpackage1/module1.py . - Josiah ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New relative import issue
On 9/18/06, Greg Ewing <[EMAIL PROTECTED]> wrote: Armin Rigo wrote:> My (limited) understanding of the motivation for relative imports is> that they are only here as a transitional feature. Fully-absolute> imports are the official future. Guido does seem to have a dislike for relative imports,but I don't really understand why. The usefulness ofbeing able to make a package self-contained and movableto another place in the package hierarchy without hacking it seems self-evident to me.It is more of how relative imports used to be inherent and thus have no clear way to delineate that an import was being done using a relative path compared to an absolute one. What's happening in Py3k? Will relative imports stillexist? Using the dot notation, yes they will exist in Py3K. -Brett ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] deja-vu .. python locking
Martin Devera wrote: > Regarding implementation, I wanted to look for some opinions before starting > to > implement something as big as this patch. Probably someone can look and say, > hey > it is stupit, you forgot that FILL_IN ... ;-) If I understand correctly, your suggestion for avoiding deadlock relies on the fact that a given thread can really only have one object locked at a time, i.e. after you LOCK an object you can only assume you own it until you LOCK another object or return to some quiescent state. Is this right? If so, the question is whether it's sufficient to be able to lock just one object at a time. Maybe it is, but some more formal consideration of that might be a good idea. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] deja-vu .. python locking
Martin Devera wrote: > As I've written in "Big reader lock" paragraph of the original proposal, these > objects could be handled by not blocking in read path But as was just pointed out, because of refcounting, there's really no such thing as read-only access to an object. What *looks* like read-only access at the Python level involves refcount updates just from the act of touching the object. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] deja-vu .. python locking
Phillip J. Eby wrote: > So, I think for your plan to work, you would have to eliminate reference > counting, in order to bring the lock overhead down to a manageable level. There's a possibility it wouldn't be atrociously bad. Seems like it would only add the 3 instructions or whatever overhead to most refcount operations. How much this would reduce performance depends on what percentage of time is currently used by refcounting. Are there any figures for that? A quick way of getting an idea of how much effect it would have might be to change Py_INCREF and Py_DECREF to go through the relevant motions, and see what timings are produced for single-threaded code. It wouldn't be a working implementation, but you'd find out pretty quickly if it were going to be a disaster. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
