Re: [Python-Dev] New methods for weakref.Weak*Dictionary types
Hi Tim, On Mon, May 01, 2006 at 04:57:06PM -0400, Tim Peters wrote: > """ > # Return a list of weakrefs to all the objects in the collection. > # Because a weak dict is used internally, iteration is dicey (the > # underlying dict may change size during iteration, due to gc or > # activity from other threads). But then, isn't the real problem the fact that applications cannot safely iterate over weak dicts? This fact could be viewed as a bug, and fixed without API changes. For example, I can imagine returning to the client an iterator that "locks" the dictionary. Upon exhaustion, or via the __del__ of the iterator, or even in the 'finally:' part of the generator if that's how iteration is implemented, the dict is unlocked. Here "locking" means that weakrefs going away during this time are not eagerly removed from the dict; they will be removed only when the dict is unlocked. A bientot, Armin. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New methods for weakref.Weak*Dictionary types
[Tim Peters]
>> """
>> # Return a list of weakrefs to all the objects in the collection.
>> # Because a weak dict is used internally, iteration is dicey (the
>> # underlying dict may change size during iteration, due to gc or
>> # activity from other threads).
[Armin Rigo]
> But then, isn't the real problem the fact that applications cannot
> safely iterate over weak dicts?
Well, in the presence of threads, iterating over any dict (weak or
not) may blow up: while some thread is iterating over the dict, other
threads can change the size of the dict, and then the dict iterator
will blow up the next time it's invoked. In the context from which
that comment was extracted, that's a potential problem (which is why
the comment mentions "activity from other threads" :-)).
> This fact could be viewed as a bug, and fixed without API changes.
> For example, I can imagine returning to the client an iterator that "locks"
> the
> dictionary. Upon exhaustion, or via the __del__ of the iterator, or even in
> the
> 'finally:' part of the generator if that's how iteration is implemented, the
> dict is
> unlocked.
>
> Here "locking" means that weakrefs going away during this time are not
> eagerly removed from the dict; they will be removed only when the dict
> is unlocked.
That could remove one source of potential iteration surprises unique
to weak dicts, due to "magical" removal of dict entries (note that it
would probably need a thread-safe count of outstanding iterators, and
not "unlock the dict" until the count fell to 0). Other threads could
still change the dict's size _seemingly_ "by magic" (from the dict
iterator's POV). I don't know whether fixing "magical" weak-dict
removal without fixing "seemingly magical" weak-dict removal or
addition via other threads would be worth the bother. Anyone burned
by either now has learned to avoid the iter{keys,values,items}()
methods.
Without more support in the dict implementation (and support that
would probably be difficult to add), the only thoroughly safe strategy
is to atomically materialize a hidden collection of the keys, values,
or items to be iterated over, and have the iterator march over those.
In effect, apps do that themselves now by iterating over
.keys()/values()/items() instead of their .iterXYZ() versions. Many
apps can get away without that, though, so there's value in keeping
the current "obvious" dict iterators.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r45925 - in python/trunk: Lib/tempfile.py Lib/test/test_os.py Misc/NEWS Modules/posixmodule.c
M.-A. Lemburg wrote: > I've tried to find documentation on _dosmaperr() but there's > nothing on MSDN. Is this an official API ? No. It always was undocumented, and apparently intentionally so (both the API, and the precise details of the mapping). > FWIW, this is the official system error code table: > > http://msdn.microsoft.com/library/default.asp?url=/library/en-us/debug/base/system_error_codes.asp This is actually the contents of winerror.h (in a different presentation form). > A complete table would be quite large... Actually, no. _dosmaperr maps the majority of the codes to EINVAL (invalid argument). Only a small subset is special-cased. > Right, but the APIs you changed used to raise IOError > using the DOS error codes - this is where there's an > incompatibility, since they now raise WindowsErrors > with completely different error codes. > > The discussion > > http://mail.python.org/pipermail/python-dev/2006-January/060243.html > > suggests that you are aware of this. I fully understand that this is an incompatible change; sure. I'm saying that changing WindowsError to include set errno to DOS error codes would *also* be an incompatible change. > Since code catching IOError will also see any WindowsError > exception, I'd opt for making .errno always return the > DOS/BSD error codes and have WindowsError grow an > additional .winerrno attribute which contains the > more fine-grained Win32 error code. Ok. I'll try to come up with a patch, but let me repeat this: even though this would nearly restore the behavior for the functions that I changed, it would *a different* incompatible change, since it would change the behavior of the places that *currently* (i.e. in 2.4) raise WindowsError. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r45925 - in python/trunk: Lib/tempfile.py Lib/test/test_os.py Misc/NEWS Modules/posixmodule.c
Martin v. Löwis wrote: > I'm saying that changing WindowsError to include set errno > to DOS error codes would *also* be an incompatible change. > >> Since code catching IOError will also see any WindowsError >> exception, I'd opt for making .errno always return the >> DOS/BSD error codes and have WindowsError grow an >> additional .winerrno attribute which contains the >> more fine-grained Win32 error code. > > Ok. I'll try to come up with a patch, but let me repeat > this: even though this would nearly restore the behavior > for the functions that I changed, it would *a different* > incompatible change, since it would change the behavior > of the places that *currently* (i.e. in 2.4) > raise WindowsError. Thanks ! I know that this will be a different incompatible change, but only for the better, I think. The only way around this breakage I see would be to introduce the ErrorCode integer sub-class I posted on the checkins list which then allows .errno to also compare equal to the .winerrno values as well: # Error code objects class ErrorCode(int): values = () def __new__(cls, basevalue, *aliasvalues): return int.__new__(cls, basevalue) def __init__(self, basevalue, *aliasvalues): self.values = (basevalue,) + aliasvalues def __cmp__(self, other): if other in self.values: return 0 if other < self.values: return 1 else: return -1 EEXISTS = ErrorCode(17, 183, 80) I'm not sure whether it's worth the trouble. BTW, and intended as offer for compromise, should we instead add the Win32 codes to the errno module (or a new winerrno module) ?! I can write a parser that takes winerror.h and generates the module code. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 10 2006) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r45925 - in python/trunk: Lib/tempfile.py Lib/test/test_os.py Misc/NEWS Modules/posixmodule.c
M.-A. Lemburg wrote: > BTW, and intended as offer for compromise, should we instead > add the Win32 codes to the errno module (or a new winerrno > module) ?! I can write a parser that takes winerror.h and > generates the module code. Instead won't help: the breakage will still occur. It would be possible to put symbolic constants into the except clauses, instead of using the numeric values, but you still have to add all these "except WindowsError,e" clauses, even if the constants were available. However, in addition would be useful: people will want to check for specific Win32 error codes, just because they are more descriptive. I propose to call the module "winerror" (in parallel to winerror.h, just as the errno module parallels errno.h) Adding them all to the errno would work for most cases, except that you get conflicts for errno.errcode. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 open issues
"Neal Norwitz" <[EMAIL PROTECTED]> writes: > Here's what's left for 2.5 after the most recent go around. > > There's no owner for these items. If no one takes them, they won't > get done and I will move them to deferred within a week: > > * Add @decorator decorator to functional, rename to functools? > What's the benefit of @decorator? Who made functional? It's new > in 2.5, right? If so, move it or it will be functional for all 2.x. > > * Remove the fpectl module? >Does anyone use this? It can probably be removed, but someone > needs to do the work. This was my idea; I'm not really offering to do the work now and don't think that it's terribly urgent, so I propose leaving this until 2.6. Cheers, mwh -- it's like a bicycle but with internet -- from Twisted.Quotes ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] nag, nag -- 2.5 open issues
If you are addressed on this message, it means you have open issues that need to be resolved for 2.5. Some of these issues are documentation, others are code issues. This information comes from PEP 356. The best way to get me to stop bugging you is to close out these tasks. :-) Note the next alpha is coming up in a week or two, so now is a good time to finish these. Who is the owner for getting new icons installed with the new logo? Georg did some work, but it seems like a lot more work needs to be done to get them installed on all platforms. This will be deferred unless someone makes it happen. Code issues: +++ Fred, Fredrik, Martin: xmlplus/xmlcore situation re: ElementTree Jeremy: AST compiler issues Philip, AMK: add wsgiref to stdlib -- is this happening? We should really try to get these resolved soon, especially the XML issue since that could result in some sort of API change. The AST issues should just be bug fixes AFAIK. Documentation missing: +++ Fredrik: ElementTree Gerhard: pysqlite Martin: msilib -- Martin/Andrew is this done? Thomas: ctypes I think we heard from Gerhard and Thomas about their doc plans. Fredrik, are you planning to get ET doc into the python docs? Let me know if there's anything I missed. If you don't expect to have time to finish these tasks, then it's your job to find someone else who will (and it would be good to reply to this message). Update the PEP or tell me and I'll update the PEP. n ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
