[Python-Dev] #1858, looking for a reviewer
Hi, The register and upload commands have a --repository option that allows to use them with any server. You can register and upload a package to any server that has the right set of web services. They also work with a configuration file called .pypirc Unfortunately there is no inline option to define the username/password. This has to be defined in .pypirc. But this file does not allow having different sets of username/password, so in reality it is not possible to play with multiple servers. #1858 provides a patch for this [0] I have submitted it in January, after some discussion in catalog-sig and distutils-sig, that led to a proposal [1] I am looking for a final reviewer, to decide if the new .pypirc is considered suitable, and maybe have it integrated in the 2.6. This new format will be used in the Plone community to interact with plone.org and the cheeseshop [2], (with a third party package at this time, that implements the patch) Regards Tarek [0] http://bugs.python.org/issue1858 [1] http://wiki.python.org/moin/EnhancedPyPI [2] http://tarekziade.wordpress.com/2008/05/06/plone-paris-sprint-wrapup-3-newploneorg-collectivedist-released/ -- Tarek Ziadé | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Warn about mktemp once again?
Martin v. Löwis v.loewis.de> writes: > > Are you (or are you not) aware that this strategy allows for malicious > code to provide you with a fake JPEG file? If so, does it not concern > you? Yes, I'm aware of it and no, it doesn't concern me. When you write a processing script for internal use there's no reason to worry about security issues like that (if someone manages to enter your private machine he can do much more dangereous things anyway). > As others have said: the reason the function is deprecated is that it > is easy to run into security problems while using it, I know, I've read the doc :) I'm sure there are lots of other insecure things in Python's stdlib (e.g. HTTP Basic Authentication, or plaintext authentication in email clients, or algorithms known to be weak in hashlib) and they haven't been deprecated. Of course some of those may be necessary for interoperability, while mktemp() isn't. In any case what I wanted to point out is that the secure replacement for a one-liner mktemp() is not as concise and obvious as the insecure version, and that sometimes there are good reasons to not even care about the "security problems". Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Warn about mktemp once again?
Tristan Seligmann wrote: > The correct way to do this is to create a temporary directory, and then > generate a filename underneath that directory to use. There is a platform difference here. On unix mktemp will usually provide a file name in a world-writeable directory (/tmp/) which is wide open to race condition attacks leading to privilege escalation. On win32 it will usually (but not always) provide a file name in a directory writeable only by the current user. The temporary directory step sometimes seems unnecessary to windows developers. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] sock.close() not closing?
Why does sock.close() not actually close sock? If I run the code import socket sock = socket.socket() ... sock.close() I would expect that a system call is done to actually close the socket and free the file descriptor. But that does not happen. Look at the code in socket.py. It merely replaces the socket instance with a dummy instance so that all subsequent calls on the sock object fail, but it does nothing else! -- Sjoerd Mullender ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] sock.close() not closing?
Hello,
2008/5/7 Sjoerd Mullender <[EMAIL PROTECTED]>:
> Why does sock.close() not actually close sock?
>
> If I run the code
>
> import socket
> sock = socket.socket()
> ...
> sock.close()
>
> I would expect that a system call is done to actually close the socket and
> free the file descriptor. But that does not happen. Look at the code in
> socket.py. It merely replaces the socket instance with a dummy instance so
> that all subsequent calls on the sock object fail, but it does nothing else!
It does close the socket:
In socket.py, when self._sock is replaced, its __del__ method will be called.
This __del__ is implemented in C, in socketmodule.c:
static void
sock_dealloc(PySocketSockObject *s)
{
if (s->sock_fd != -1)
(void) SOCKETCLOSE(s->sock_fd);
Py_TYPE(s)->tp_free((PyObject *)s);
}
Of course, if you call sock.dup() or sock.makefile(),
there is another reference to the underlying _sock, and you must
close() all these objects.
--
Amaury Forgeot d'Arc
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] urllib unicode handling
Hi, Jeroen Ruigrok van der Werven in-nomine.org> writes: > Would people object if such functionality got added to urllib? I would ;-) There are IRIs, just that nobody wrote a useful module for that. There are algorithms in the RFC that can convert URIs to IRIs and the other way round. IMO that's the way to go. Regards, Armin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] urllib unicode handling
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf > Of Jeroen Ruigrok van der Werven > Sent: Wednesday, May 07, 2008 05:20 > To: Tom Pinckney > Cc: [email protected] > Subject: Re: [Python-Dev] urllib unicode handling > > -On [20080507 04:06], Tom Pinckney ([EMAIL PROTECTED]) wrote: > >While in theory UTF-8 is not a standard, sites like Last.fm, Facebook > and > >Wikipedia seem to have embraced it (as have pretty much all other > major web > >sites). As with HTML, there is what the standard says and what the > actual > >browsers have to accept in order to work in the real world. > FYI, here is how we have patched urrlib2 for use in EVE: --- C:\p4\sdk\stackless25\Lib\urllib.py 2008-03-21 14:47:23.0 - +++ C:\p4\eve\KALI\common\stdlib\urllib.py 2007-11-06 11:18:01.0 - @@ -1158,12 +1158,29 @@ except KeyError: res[i] = '%' + item except UnicodeDecodeError: res[i] = unichr(int(item[:2], 16)) + item[2:] return "".join(res) +unquote_inner = unquote +def unquote(s): +"""CCP attempt at making sensible choices in unicode quoteing / unquoting """ +s = unquote_inner(s) +try: +u = s.decode("utf-8") +try: +s2 = s.decode("ascii") +except UnicodeDecodeError: +s = u #yes, s was definitely utf8, which isn't pure ascii +else: +if u != s: +s = u +except UnicodeDecodeError: +pass #can't have been utf8 +return s + def unquote_plus(s): """unquote('%7e/abc+def') -> '~/abc def'""" s = s.replace('+', ' ') return unquote(s) always_safe = ('ABCDEFGHIJKLMNOPQRSTUVWXYZ' @@ -1201,12 +1218,20 @@ for i in range(256): c = chr(i) safe_map[c] = (c in safe) and c or ('%%%02X' % i) _safemaps[cachekey] = safe_map res = map(safe_map.__getitem__, s) return ''.join(res) + +quote_inner = quote +def quote(s, safe = '/'): +"""CCP addition, to try to sensibly support / circumvent issues with unicode in urls""" +try: +return quote_inner(s, safe) +except KeyError: +return quote_inner(s.encode("utf-8", safe)) def quote_plus(s, safe = ''): """Quote the query fragment of a URL; replacing ' ' with '+'""" if ' ' in s: s = quote(s, safe + ' ') return s.replace(' ', '+') ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] urllib unicode handling
I may be missing something, but it seems that RFC 3987 (which is about IRIs) basically says: 1) IRIs are identical to URIs except they may have unicode characters in them 2) IRIs must be converted to URIs before being used in HTTP 3) The way to convert IRIs to URIs is to UTF-8 encode the unicode characters in the IRI and then percent encode the resulting octects that are unsafe to have in a URI 4) There's some ambiguity over what to do with the hostname portion of the URI if it hash one (IDN, replace non-ascii characters with dashes etc) If this is indeed the case, it sounds perfectly legal (according to the RFC) and perfectly practical (as required by numerous popular websites) to have urllib.quote and urllib.quote_plus do an automatic UTF-8 encoding of unicode strings before percent encoding them. It's not entirely clear to me if people should be calling urllib.quote on hostnames and expecting them to be encoded properly if the hostname contains non-ascii characters. Perhaps the docs should be clarified on this matter? Similarly, urllib.unquote should precent-decode characters and then attempt to convert the resulting octects from utf-8 to unicode. If that conversion fails, we can assume the octects should be returned as a byte string rather than a unicode string. On May 7, 2008, at 8:12 AM, Armin Ronacher wrote: Hi, Jeroen Ruigrok van der Werven in-nomine.org> writes: Would people object if such functionality got added to urllib? I would ;-) There are IRIs, just that nobody wrote a useful module for that. There are algorithms in the RFC that can convert URIs to IRIs and the other way round. IMO that's the way to go. Regards, Armin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/thomaspinckney3%40gmail.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Community buildbots
Hi, I just wanted to point out a few things: Community 2.5 bots, 6 out of 8 offline, of the remaining two (which are both red), one is actually using Python 2.6, not Python 2.5: http://python.org/dev/buildbot/community/2.5/ Community 2.6 bots, 6 out of 8 offline, but at least the remaining two (both of which are red) seem to be using the correct Python version: http://python.org/dev/buildbot/community/trunk/ Jean-Paul ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] sock.close() not closing?
On 2008-05-07 13:37, Amaury Forgeot d'Arc wrote:
Hello,
2008/5/7 Sjoerd Mullender <[EMAIL PROTECTED]>:
Why does sock.close() not actually close sock?
If I run the code
import socket
sock = socket.socket()
...
sock.close()
I would expect that a system call is done to actually close the socket and
free the file descriptor. But that does not happen. Look at the code in
socket.py. It merely replaces the socket instance with a dummy instance so
that all subsequent calls on the sock object fail, but it does nothing else!
It does close the socket:
In socket.py, when self._sock is replaced, its __del__ method will be called.
This __del__ is implemented in C, in socketmodule.c:
static void
sock_dealloc(PySocketSockObject *s)
{
if (s->sock_fd != -1)
(void) SOCKETCLOSE(s->sock_fd);
Py_TYPE(s)->tp_free((PyObject *)s);
}
Of course, if you call sock.dup() or sock.makefile(),
there is another reference to the underlying _sock, and you must
close() all these objects.
I have to question the design of this. When I close() an object I
expect it to be closed there and then and not at some indeterminate
later time (well, it is determinate when you're fully aware of all
references, but often you aren't--trust me, I understand reference
counting).
Then there also seems to be a bug in imaplib.IMAP4_SSL since in its
shutdown method it closes the socket but leaves the sslobj untouched. I
assume that that object keeps a reference to the socket.
But as I said, I expect the close() to actually close.
--
Sjoerd Mullender
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?
Jeremy Hylton wrote: On Fri, May 2, 2008 at 1:38 PM, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: Thomas Lee wrote: > Martin v. Löwis wrote: >>> This leaves us with a few options: >>> >> >> 5. Reuse/Abuse Num(object) for arbitrary constants. >>AFAICT, this should work out of the box. >> >> > Eek. It *does* seem like Num would work out of the box, but would this > be a good idea? No. I suggested it just for completeness. > What about *replacing* Num with Const? Might make optimizations > specifically for numeric values slightly hairier, and semantically I > think they might be different enough to warrant separate AST nodes > despite the similarity in implementation at the compiler level. I think they should be separate. Const would be a mere addition; for compatibility with other uses of the AST, that's actually better than simultaneous removal of Num. Adding Const sounds good to me. As Thomas mentions in a later message, making it possible to annotate nodes would permit Functions to be annotated as being a generator at the AST stage (currently it is left to the bytecode compiler's symtable generation pass to make that determination). Although I guess an alternative solution to that would be to have separate AST nodes for Functions and Generators as well... Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Warn about mktemp once again?
> On win32 it will usually (but not always) provide a file name in a directory
> writeable only by the current user.
OS X 10.5 also does this.
>>> import tempfile
>>> tempfile.mktemp()
'/var/folders/Ru/RuapMUan2RWWJ++1YwBnRU++0T6/-Tmp-/tmpjjkKha'
>>> os.system("ls -dl /var/folders/Ru/RuapMUan2RWWJ++1YwBnRU++0T6/-Tmp-")
drwx-- 2 wjanssen 100 68 May 7 08:34
/var/folders/Ru/RuapMUan2RWWJ++1YwBnRU++0T6/-Tmp-
0
>>>
Bill
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?
Nick Coghlan wrote: As Thomas mentions in a later message, making it possible to annotate nodes would permit Functions to be annotated as being a generator at the AST stage (currently it is left to the bytecode compiler's symtable generation pass to make that determination). Although I guess an alternative solution to that would be to have separate AST nodes for Functions and Generators as well... I've actually backtracked a little and gone back down the Const path again. I know this is the third time I've changed my mind, but it's primarily because annotations tend to get a little clunky (or my implementation was, at least). Using Const nodes feels a lot more natural inside the optimizer. I think it's going to stick, at least in the short term. Rather than separate FunctionDef and GeneratorDef nodes, I think a new bool attribute (is_generator?) on FunctionDef should do the job nicely. Further, I'm thinking we can move the "generator detection code" from the symtable into Python/ast.c so the generator/function information is available to the optimizer. This is made a little tricky by the absence of the contextual information available that is normally available when flagging generators in the symtable. When generating AST nodes for a suite, we know nothing about the parent node in which the suite resides. Still, it might be doable. If this winds up being ugly, we might need to fall back to the original plan of a separate pass over function bodies to detect yield expressions. I'll look into all this tomorrow night, along with any other crazy suggestions. For now I need to sleep a few hours. :) Thanks for the feedback, it's much appreciated. Cheers, T ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] urllib unicode handling
"Martin v. Löwis" wrote:
> The proper way to implement this would be IRIs (RFC 3987),
> in particular section 3.1. This is not as simple as just
> encoding it as UTF-8, as you might have to apply IDNA to
> the host part.
>
> Code doing so just hasn't been contributed yet.
But if someone wanted to do so, it's pretty simple:
>>> u'www.\u212bngstr\xf6m.com'.encode("idna")
'www.xn--ngstrm-hua5l.com'
Robert Brewer
[EMAIL PROTECTED]
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] sock.close() not closing?
Sjoerd Mullender wrote: > On 2008-05-07 13:37, Amaury Forgeot d'Arc wrote: > > 2008/5/7 Sjoerd Mullender <[EMAIL PROTECTED]>: > >> I would expect that a system call is done to actually close the > >> socket and free the file descriptor. But that does not happen. > > > > It does close the socket: > > > > In socket.py, when self._sock is replaced, its __del__ method will be > > called. > > I have to question the design of this. When I close() an object I > expect it to be closed there and then and not at some indeterminate > later time (well, it is determinate when you're fully aware of all > references, but often you aren't--trust me, I understand reference > counting). Even if you're fully aware of all references, it's indeterminate in multithreaded apps. I've just taken to doing: self.socket._sock.close() self.socket.close() ...in order to send the FIN I wanted ASAP. Robert Brewer [EMAIL PROTECTED] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] urllib unicode handling
Maybe I didn't understand the RFC quite right, but it seemed like how
to handle hostnames was left as a choice between IDNA encoding the
hostname or replacing the non-ascii characters with dashes? I guess in
practice IDNA is the right decision.
Another part I wasn't clear on is whether urllib.quote() understands
it's working on URIs, arbitrary strings, URLs or what. It seems that
from the documentation it looks like it's expecting to just work on
the path component of URLs. If this is so, then it doesn't need to
understand what to do if the IRI contains a hostname.
Seems like the other somewhat under-specified part of all of this is
how urllib.unquote() should work. If after percent decoding it sees
non-ascii octets, should it try to decode them as utf-8 and if that
fails then leave them as is?
On May 7, 2008, at 11:55 AM, Robert Brewer wrote:
"Martin v. Löwis" wrote:
The proper way to implement this would be IRIs (RFC 3987),
in particular section 3.1. This is not as simple as just
encoding it as UTF-8, as you might have to apply IDNA to
the host part.
Code doing so just hasn't been contributed yet.
But if someone wanted to do so, it's pretty simple:
u'www.\u212bngstr\xf6m.com'.encode("idna")
'www.xn--ngstrm-hua5l.com'
Robert Brewer
[EMAIL PROTECTED]
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] sock.close() not closing?
On Wed, May 7, 2008 at 7:37 AM, Sjoerd Mullender <[EMAIL PROTECTED]> wrote:
> On 2008-05-07 13:37, Amaury Forgeot d'Arc wrote:
>
> > Hello,
> >
> > 2008/5/7 Sjoerd Mullender <[EMAIL PROTECTED]>:
> >
> > > Why does sock.close() not actually close sock?
> > >
> > > If I run the code
> > >
> > > import socket
> > > sock = socket.socket()
> > > ...
> > > sock.close()
> > >
> > > I would expect that a system call is done to actually close the socket
> and
> > > free the file descriptor. But that does not happen. Look at the code
> in
> > > socket.py. It merely replaces the socket instance with a dummy instance
> so
> > > that all subsequent calls on the sock object fail, but it does nothing
> else!
> > >
> >
> > It does close the socket:
> >
> > In socket.py, when self._sock is replaced, its __del__ method will be
> called.
> > This __del__ is implemented in C, in socketmodule.c:
> >
> > static void
> > sock_dealloc(PySocketSockObject *s)
> > {
> >if (s->sock_fd != -1)
> >(void) SOCKETCLOSE(s->sock_fd);
> >Py_TYPE(s)->tp_free((PyObject *)s);
> > }
> >
> >
> > Of course, if you call sock.dup() or sock.makefile(),
> > there is another reference to the underlying _sock, and you must
> > close() all these objects.
> >
> >
>
> I have to question the design of this. When I close() an object I
> expect it to be closed there and then and not at some indeterminate
> later time (well, it is determinate when you're fully aware of all
> references, but often you aren't--trust me, I understand reference
> counting).
>
> Then there also seems to be a bug in imaplib.IMAP4_SSL since in its
> shutdown method it closes the socket but leaves the sslobj untouched. I
> assume that that object keeps a reference to the socket.
>
> But as I said, I expect the close() to actually close.
I agree that the design is incredibly fragile. It came about from a
desire to remain compatible with the semantics enshrined in the first
implementation supporting makefile() -- once you call makefile(), you
have multiple references to the socket, and the choice was made to
dup() the file descriptor for each reference. This meant that, while
.close() on each object would close the file descriptor, the socket
connection would not be closed until the last fd was closed,
essentially letting the Unix kernel do our reference counting for us.
Example:
$ python2.2
Python 2.2.3+ (#1, Sep 6 2005, 04:14:07)
[GCC 4.0.2 20050808 (prerelease) (Debian 4.0.1-4ubuntu6)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> s = socket.socket()
>>> s.connect(('python.org', 80))
>>> f = s.makefile()
>>> s.fileno()
3
>>> f.fileno()
4
>>>
Unfortunately we couldn't use this architecture on Windows, which
doesn't (or at the time didn't) have a dup() system call, and I
believe that's where the code that doesn't close the socket but relies
on refcounting comes from. The SSL architecture also caused additional
problems. In the end a version of this approach was used on all
platforms, so that we now have the following:
python2.6
Python 2.6a1 (trunk:61189, Mar 2 2008, 17:07:23)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> s = socket.socket()
>>> s.connect(('python.org', 80))
>>> f = s.makefile()
>>> s.fileno()
3
>>> f.fileno()
3
>>>
Note that s and f now share a file descriptor! Python 2.4 behaves the
same way. I don't recall if we changed this in 2.3 or in 2.4.
All this was because of the requirement that should be allowed to
close (or simply delete) the original socket even when you continue to
use the stream(s) derived from it using makefile(). This requirement
existed out of the desire to be backwards compatible with the original
Unix implementation, which had this property.
I would be okay with changing the requirements in Py3k so that you are
required to keep the socket object open at least as long as you plan
on using the derived stream(s), but this will require some careful
redesign, especially in the light of SSL support. (Read ssl.py and
_ssl.c to understand the level of intertwinement.) In 2.6 I think we
should leave the design alone (to the point that it hasn't already
changed due to Bill Janssen's providing a new, enhanced SSL
implementation) for best compatibility.
I don't know anything about the IMAP issue, sorry.
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?
On Wed, May 7, 2008 at 11:43 AM, Thomas Lee <[EMAIL PROTECTED]> wrote: > Nick Coghlan wrote: > > > > > As Thomas mentions in a later message, making it possible to annotate > nodes would permit Functions to be annotated as being a generator at the AST > stage (currently it is left to the bytecode compiler's symtable generation > pass to make that determination). > > > > Although I guess an alternative solution to that would be to have separate > AST nodes for Functions and Generators as well... > > > > > I've actually backtracked a little and gone back down the Const path again. > I know this is the third time I've changed my mind, but it's primarily > because annotations tend to get a little clunky (or my implementation was, > at least). Using Const nodes feels a lot more natural inside the optimizer. > I think it's going to stick, at least in the short term. > > Rather than separate FunctionDef and GeneratorDef nodes, I think a new bool > attribute (is_generator?) on FunctionDef should do the job nicely. Further, > I'm thinking we can move the "generator detection code" from the symtable > into Python/ast.c so the generator/function information is available to the > optimizer. Does the optimizer run after the symtable? I'd expect the symbol table information to be essential for almost all analysis, so it should be fine to wait until that pass to mark the generators. Jeremy > This is made a little tricky by the absence of the contextual information > available that is normally available when flagging generators in the > symtable. When generating AST nodes for a suite, we know nothing about the > parent node in which the suite resides. Still, it might be doable. If this > winds up being ugly, we might need to fall back to the original plan of a > separate pass over function bodies to detect yield expressions. > > I'll look into all this tomorrow night, along with any other crazy > suggestions. For now I need to sleep a few hours. :) > > Thanks for the feedback, it's much appreciated. > > Cheers, > T > > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] sock.close() not closing?
> I would be okay with changing the requirements in Py3k so that you are > required to keep the socket object open at least as long as you plan > on using the derived stream(s), but this will require some careful > redesign, especially in the light of SSL support. (Read ssl.py and > _ssl.c to understand the level of intertwinement.) In 2.6 I think we > should leave the design alone (to the point that it hasn't already > changed due to Bill Janssen's providing a new, enhanced SSL > implementation) for best compatibility. I'd be perfectly happy to get rid of this. It's not SSL in particular that requires it, if I'm remembering correctly, it's really the HTTP library -- HTTP over SSL was failing when we tried to change this. The HTTP client-side library calls "makefile" on the socket, then closes it, then passes the file returned from "makefile" on to application code to work with. That code merrily reads and writes from the (closed) socket to read responses from the remote Web server. This particular nasty pattern is deeply entwined in all the code that touches the HTTP library in any way, so it will be a big job to get rid of it -- basically re-writing HTTP support and all the services which use it. I didn't want to get into that :-). And I agree with Guido; it doesn't make sense to try to fix this in 2.6. And plan a few months of work to fix it in 3.x; even then, I think you'll break a lot of user code (that won't close the socket when they're finished with it). On the other hand, maybe it's not so bad; maybe you could just (in httplib) read the whole response into a StringIO, and pass that on to the client code. Bill ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] sock.close() not closing?
See http://bugs.python.org/issue1348. Bill ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] sock.close() not closing?
On Wed, May 07, 2008 at 10:29:02AM -0700, Bill Janssen wrote: > This particular nasty pattern is deeply entwined in all the code that > touches the HTTP library in any way, so it will be a big job to get > rid of it -- basically re-writing HTTP support and all the services > which use it. I didn't want to get into that :-). This would be a good chance for Py3K to dump httplib/urllib/urllib2 and use some more modern library. --amk ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] sock.close() not closing?
2008/5/7, A.M. Kuchling <[EMAIL PROTECTED]>: > This would be a good chance for Py3K to dump httplib/urllib/urllib2 > and use some more modern library. Which modern library do you propose? -- .Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] sock.close() not closing?
On Wed, May 07, 2008 at 03:29:33PM -0300, Facundo Batista wrote: > > This would be a good chance for Py3K to dump httplib/urllib/urllib2 > > and use some more modern library. > > Which modern library do you propose? I have no idea -- presumably we'd need to compare a bunch of them (curl, libwget, and whatever else is out there) and decide upon one. --amk ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] urllib unicode handling
> Maybe I didn't understand the RFC quite right, but it seemed like how to > handle hostnames was left as a choice between IDNA encoding the hostname > or replacing the non-ascii characters with dashes? I guess in practice > IDNA is the right decision. I haven't fully understood it, either, but I think that's the right conclusion. People want to fetch the resource, then, and encoding the host name in UTF-8 won't do much good. > Seems like the other somewhat under-specified part of all of this is how > urllib.unquote() should work. If after percent decoding it sees > non-ascii octets, should it try to decode them as utf-8 and if that > fails then leave them as is? That's why I think that using IRIs should be a separate feature, perhaps a separate module entirely. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] urllib unicode handling
> If this is indeed the case, it sounds perfectly legal (according to the > RFC) and perfectly practical (as required by numerous popular websites) > to have urllib.quote and urllib.quote_plus do an automatic UTF-8 > encoding of unicode strings before percent encoding them. It's probably legal, but I don't understand why you think it's practical. The DNS lookup then will certainly fail, no? Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes documentation
> I just made another attempt to use ctypes to wrap a library, and am > facing the same problem I had the last time: the documentation doesn't > really work. I'm wondering if we have any projects underway to > re-write it? If *we* had one, *we* would know, right ?-) So, no. The author of ctypes is not a native English speaker, so unless you want the details clarified in German, you need to find somebody who actually speaks the language in which you want to read the documentation. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] REMINDER: Python Sprint Weekend This Weekend!
Just a friendly reminder that this weekend is the Python sprint weekend! Look forward to seeing everyone on #python-dev irc.freenode.net over the course of the weekend! Trent. On 16 Apr, 18:52, Trent Nelson wrote: > >Following on from the success of previous sprint/bugfix weekends and >sprinting efforts at PyCon 2008, I'd like to propose the next two >Global Python Sprint Weekends, taking place on the following dates: > >* May 10th-11th (four days after 2.6a3 and 3.0a5 are released) >* June 21st-22nd (~week before 2.6b2 and 3.0b2 are released) > >It seems there are a few of the Python User Groups keen on meeting >up in person and sprinting collaboratively, akin to PyCon, which I >highly recommend. I'd like to nominate Saturday across the board >as the day for PUGs to meet up in person, with Sunday geared more >towards an online collaboration day via IRC, where we can take care >of all the little things that got in our way of coding on Saturday >(like finalising/preparing/reviewing patches, updating tracker and >documentation, writing tests ;-). > >For User Groups that are planning on meeting up to collaborate, >please reply to this thread on [email protected] and let every- >one know your intentions! > >As is commonly the case, #python-dev on irc.freenode.net will be >the place to be over the course of each sprint weekend; a large >proportion of Python developers with commit access will be present, >increasing the amount of eyes available to review and apply patches. > >For those that have an idea on areas they'd like to sprint on and >want to look for other developers to rope in (or just to communicate >plans in advance), please also feel free to jump on this thread via >python-dev@ and indicate your intentions. > >For those that haven't the foggiest on what to work on, but would >like to contribute, the bugs tracker at http://bugs.python.org is >the best place to start. Register an account and start searching >for issues that you'd be able to lend a hand with. > >All contributors that submit code patches or documentation updates >will typically get listed in Misc/ACKS.txt; come September when the >final release of 2.6 and 3.0 come about, you'll be able to point at >the tarball or .msi and exclaim loudly ``I helped build that!'', >and actually back it up with hard evidence ;-) > >Bring on the pizza and Red Bull! > >Trent. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] urllib unicode handling
I was assuming urllib.quote/unquote would only be called on text intended to be used in non-hostname portions of the URIs. I'm not sure if this is the actual intent of urllib.quote and perhaps the documentation should be updated to specify what precisely it does and then peopel can decide what parts of URIs it is appropriate to quote/ unquote. I don't believe quote/unquote does anything sensical with hostnames today that contain non-printable ascii, so this is no loss of existing functionality. Re your suggestion that IRIs should be a separate module: I guess my thought is that urllib out of the box should just work with the way websites on the web today actually work. Thus, we should make urllib do the utf-8 encode / decode rather than make users switch to a different module for certain URLs and another library for other URLs. Re the specific issue of how urllib.unquote should work: Perhaps there could be an optional second argument that specified a content encoding to use when decoding escaped characters? I would propose that this parameter have a default value of utf-8 since that is what most websites seem to do, but if the author knew that the website they were using encoded URLs in iso-8559 then they could unquote using that scheme. On May 7, 2008, at 3:10 PM, Martin v. Löwis wrote: If this is indeed the case, it sounds perfectly legal (according to the RFC) and perfectly practical (as required by numerous popular websites) to have urllib.quote and urllib.quote_plus do an automatic UTF-8 encoding of unicode strings before percent encoding them. It's probably legal, but I don't understand why you think it's practical. The DNS lookup then will certainly fail, no? Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Releasing alphas tonight
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all, Just a reminder that I'm going to be cutting the releases tonight. Because of work, I didn't make the 6pm EDT goal, and now I have to run out for a few hours. I will send another message when I'm ready to start spinning the release, but figure it will be at about 10pm EDT. Please limit your commits between now and then to only things you absolutely know will improve stability and test passing. Thanks, - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCI6MXEjvBPtnXfVAQIfgwP+P7XOTMwWex5+YwiOza0fEeUj5n8OJuxU ISK3p3Tas4tPM65eMCHk5vmIFOBfJDFyWBpNhGr+uKmaWMgiqtPX5fs6nMmkbkrY dWrfG5Mgth9U1hpR4/1y/p2W82DJX9exmnjYL2BxjZ/TGeZdbcpUcs6Cc/fpHKR/ wTQ3dagAPNA= =bDtn -END PGP SIGNATURE- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Warn about mktemp once again?
Antoine Pitrou wrote: When you write a processing script for internal use there's no reason to worry about security issues like that Even without security issues, there's still the potential for two processes creating temp files at the same time to trip over each other. -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] GSoC Student Introduction, again
Hi, I have been accepted as a student for GSoC and I'm already working in my project, "Bringing Ttk to Tkinter". This is my second GSoC, first time in PSF, and it has been awesome to have Fredrik Lundh as mentor, we have talked a lot and he is a very nice person. Regarding the project, I expect it to be integrated into Python's stdlib sometime in the future, I believe it will be in very good shape before binaries compiled against tcl/tk 8.5 start showing up. I've put a site that contains all the information available, which may be of interest, at http://gpolo.ath.cx:81/projects/ttk_to_tkinter/ and will be updating it as necessary. There is also a repo you may take a look: http://svn.python.org/view/sandbox/trunk/ttk-gsoc/ Thank you all, -- -- Guilherme H. Polo Goncalves ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Releasing alphas tonight
Barry Warsaw schrieb: > Hi all, > > Just a reminder that I'm going to be cutting the releases tonight. > Because of work, I didn't make the 6pm EDT goal, and now I have to run > out for a few hours. I will send another message when I'm ready to > start spinning the release, but figure it will be at about 10pm EDT. > Please limit your commits between now and then to only things you > absolutely know will improve stability and test passing. The py3k branch has a major show stopper, It's leaking references to the max. ... test_builtin leaked [14, 14, 14, 14] references, sum=56 test_exceptions beginning 9 repetitions 123456789 . test_exceptions leaked [40, 40, 40, 40] references, sum=160 test_types beginning 9 repetitions 123456789 . test_types leaked [2, 2, 2, 2] references, sum=8 test_unittest beginning 9 repetitions 123456789 . test_unittest leaked [23, 23, 23, 23] references, sum=92 ... I'm trying to find the issue. Christian ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] sock.close() not closing?
Bill Janssen wrote: The HTTP client-side library calls "makefile" on the socket, then closes it, then passes the file returned from "makefile" on to application code to work with. Seems to me we really need two different APIs for doing a makefile()-like operation, depending on whether you're expecting to get a copy of the file descriptor or not. -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] PEP 370 (was Re: Reminder: last alphas next Wednesday 07-May-2008)
Christian Heimes wrote: Can you come up with a path that fits the purpose of a base directory? "~/Library/Application Support/Python" would seem to be appropriate. -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] PEP 370 (was Re: Reminder: last alphas next Wednesday 07-May-2008)
Christian Heimes wrote: Can you come up with a path that fits the purpose of a base directory? Second thoughts, I think there's something better -- there's already a system-supplied directory on my system called /Library/Python/2.3 with a site-packages containing a README that says it's for installing 3rd party packages. So the obvious user-level analogy would be "~/Library/Python/x.y". -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] r62778 - in python/branches/py3k: Lib/io.py Lib/test/test_StringIO.py Lib/test/test_io.py Lib/test/test_largefile.py Lib/test/test_memoryio.py Lib/test/test_mimetools.py Modules/_byte
On Tue, May 6, 2008 at 6:52 PM, Christian Heimes <[EMAIL PROTECTED]> wrote: > alexandre.vassalotti schrieb: > > > Author: alexandre.vassalotti > > Date: Tue May 6 21:48:38 2008 > > New Revision: 62778 > > > > Log: > > Added fast alternate io.BytesIO implementation and its test suite. > > Removed old test suite for StringIO. > > Modified truncate() to imply a seek to given argument value. > > Thanks for your great work! But what about the trunk? :] Can you port > your code to the trunk before the alpha gets out? > I have a backported version of my code for the trunk. Should I commit it or should I post it to issue tracker and wait for proper review? -- Alexandre ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Releasing alphas tonight
Hello. > The py3k branch has a major show stopper, It's leaking references to the > max. Is there any chance this leak also will be fixed? http://bugs.python.org/issue Thank you. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Warn about mktemp once again?
This has been a fairly interesting discussion about tempfile.mktemp, but can we return to the original question? Should we begin warning about its use again? Should we ever warn about it? It appears that NamedTemporaryFile gives you a reasonable (secure) replacement. I'm happy to document mktemp->NamedTemporaryFile with an example in tempfile.rst if that's what it will take. Before knowing about the delete=False parameter I proposed a mkstempf() function, whose tracker issue I will close if that hasn't already happened. Skip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Releasing alphas tonight
Christian Heimes schrieb: > The py3k branch has a major show stopper, It's leaking references to the > max. Fixed ;) Christian ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Releasing alphas tonight
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On May 7, 2008, at 9:25 PM, Christian Heimes wrote: Christian Heimes schrieb: The py3k branch has a major show stopper, It's leaking references to the max. Fixed ;) Thanks! Folks, I apologize. I had some system problems tonight so I fell behind on the release. I just applied Antoine's patch for bug 2507 and I'd like to make sure the buildbots complete. Other than that, the only other release critical is one for the release process. I'll complete the releases tomorrow morning (EDT) so in the meantime, please refrain from committing anything. Thanks, - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCKDMXEjvBPtnXfVAQL1CgP9Heg1XlNjpM3wgT4N8PK090HnaGIJ6MzH Fs3QtngvLB/YPf31VrkYILIIMG/YBs+yqCZFziuSR2alNYNBcvwNVfpIljMuq9AM qtj+cu2vbhkoh+gR8LjM1J8ZWAKhI5G6eAxHuGlTWykdumcSllkB6xW4uLQ2RolZ eOS5Avnc/Qs= =kJdD -END PGP SIGNATURE- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Releasing alphas tonight
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On May 7, 2008, at 9:41 PM, Hirokazu Yamamoto wrote: Hello. The py3k branch has a major show stopper, It's leaking references to the max. Is there any chance this leak also will be fixed? http://bugs.python.org/issue Not for the alphas, sorry. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCKDRnEjvBPtnXfVAQKjGAP8CFKRDMBYJC8dm+tR/nHucrRa/Nqfy977 I8rx/B5QWN+feBk6LhODaEQ2NPOQaF+iSTaDnOlF9f2+Z6m85b94zsLJPY9EoiAC qdNmYBmZWYtuzvLmCh5Ef2aCjtfbn4Ik8i3SR9amQJBhuq7ubbdYVUsbcy6HCUUV K2Xp8LV1HWM= =aYB/ -END PGP SIGNATURE- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] GSoC Student Introduction, again
> Regarding the project, I expect it to be integrated into Python's > stdlib sometime in the future, I believe it will be in very good shape > before binaries compiled against tcl/tk 8.5 start showing up. Actually, I would like to release Python 2.6 and 3.0 on Windows with Tk 8.5 included, preferably starting at the next betas. Would you see a problem with that? Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
