Re: [Python-Dev] Distutils2 scripts

2010-10-22 Thread Paul Moore
On 22 October 2010 04:31, Ron Adam  wrote:
> When it's in the stdlib, the -m option should work just like any other
> script run from the stdlib.
>
> What path hacking are you thinking of?

On Windows, neither the "python" executable nor scripts in
C:\Pythonxx\Scripts are in the PATH by default. On the other hand, .py
files will run automatically via the registered file extension.

Manipulating PATH at install time (to add C:\PythonXX and/or
C:\PythonXX\Scripts) is not done - it is (rightly, in my view)
considered too difficult to get right, particularly when it comes to
uninstalling.

Many Windows users will, I guess, manually add python to PATH (so that
python-m works). Some people also add C:\PythonXX\Scripts. Personally,
I don't - so for me a pysetup script in that location would be no use.

So my personal vote is +1 for a python -m approach, and -0 for a
pysetup executable. I'm -1 on a pysetup.bat batch file - bat files
have other issues which IMO make them effectively useless.

Paul.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] 3.1.3 and 2.7.1 release schedule

2010-10-22 Thread Dirkjan Ochtman
On Fri, Oct 22, 2010 at 00:57, Benjamin Peterson  wrote:
> In the interest of getting 3.1.3 and 2.7.1 out by next year, here's a
> tentative release schedule:
>
> November 13th - RC1
> November 27th - RC2
> December 11th - Final

The last one might clash with the hg migration a bit, do we need to
worry about that? Or did you purposely pick the day before the planned
hg migration?

Cheers,

Dirkjan
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] 3.1.3 and 2.7.1 release schedule

2010-10-22 Thread Georg Brandl
Am 22.10.2010 09:36, schrieb Dirkjan Ochtman:
> On Fri, Oct 22, 2010 at 00:57, Benjamin Peterson  wrote:
>> In the interest of getting 3.1.3 and 2.7.1 out by next year, here's a
>> tentative release schedule:
>>
>> November 13th - RC1
>> November 27th - RC2
>> December 11th - Final
> 
> The last one might clash with the hg migration a bit, do we need to
> worry about that? Or did you purposely pick the day before the planned
> hg migration?

If everything goes as planned, there won't be many commits between RC2 and
final, so it should be fine.  The svn repos won't be removed anyway, so
making a release from them is still possible.

(Side question: do we want to move the svn repos to a slightly different
URL so that people tracking the repo know something's happened?)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] 3.1.3 and 2.7.1 release schedule

2010-10-22 Thread Dirkjan Ochtman
On Fri, Oct 22, 2010 at 11:06, Georg Brandl  wrote:
> If everything goes as planned, there won't be many commits between RC2 and
> final, so it should be fine.  The svn repos won't be removed anyway, so
> making a release from them is still possible.

Okay, but accepting commits in both SVN and hg at the same time is
potentially messy.

> (Side question: do we want to move the svn repos to a slightly different
> URL so that people tracking the repo know something's happened?)

I would be in favor of that, yes.

Cheers,

Dirkjan
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] 3.1.3 and 2.7.1 release schedule

2010-10-22 Thread Georg Brandl
Am 22.10.2010 11:41, schrieb Dirkjan Ochtman:
> On Fri, Oct 22, 2010 at 11:06, Georg Brandl  wrote:
>> If everything goes as planned, there won't be many commits between RC2 and
>> final, so it should be fine.  The svn repos won't be removed anyway, so
>> making a release from them is still possible.
> 
> Okay, but accepting commits in both SVN and hg at the same time is
> potentially messy.

I don't think Benjamin would want anyone but him committing to these
branches between rc2 and final anyway :)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] My work on Python3 and non-ascii paths is done

2010-10-22 Thread Victor Stinner
Le jeudi 21 octobre 2010 21:14:55, Toshio Kuratomi a écrit :
> > That's exactly what I was looking for!  Thanks.  I think you've learned a
> > huge amount of good information that's difficult to find, so writing it
> > up in a more permanent and easy to find location will really help future
> > Python developers!
> 
> One further thing I'd be interested in is if you could document any best
> practices from this experience.  Things like, "surrogateescape is a
> good/bad default in these cases",

I advice to use the PEP 383 (surrogateescape) when the *native* data type is 
bytes. Some examples:
 - filenames on UNIX/BSD
 - environment variables on UNIX/BSD
 - well, most data send/received from the system on UNIX/BSD :-)

For network protocols, I don't know. It looks like the new email modules will 
offer two API levels: low level (native type) using bytes, high level using 
str (unicode). I don't know if the high level API uses the PEP 383 or not.

PEP 383 can be used to avoid UnicodeDecodeError. But sometimes it's better to 
raise an error to warn the user that the encoding is incorrect or the input 
data is invalid (well, at least not correctly according to the encoding).

I don't use strict rules. Each problem is different. Eg. it looks like not 
everybody agrees to use the PEP 383 for the host/domain name (issue #9377, I 
didn't read the whole issue, just few lines).

> When is parallel functions for bytes and str better than a single
> polymorphic function?

If you cannot decide the output type depending on the inputs, it's better to 
have two functions.

Examples:
 - 2 functions; os.getcwd() / os.getcwdb().
 - polymorphic: os.path.*()

But you should never accept mixed types, eg. os.path.join(b'bytes', 'unicode) 
have to raise a TypeError.

-- 
Victor Stinner
http://www.haypocalc.com/
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] SVN rev. 85392 broke module handling in py3k

2010-10-22 Thread Stefan Behnel

Hi,

since SVN rev. 85392, Cython's installation fails on the py3k branch with a 
weird globals error. I think it is related to some sys.modules magic that 
we do in order to support running Cython in Python 3 using lib2to3.


Basically, what we do is, we import some parts of Cython at the beginning 
that are Py3 clean, specifically some distutils build_ext replacement for 
building Cython modules. Then we start up distutils, which first runs 
lib2to3 on Cython's sources to convert them into Py3 code. When it then 
gets to building the binary modules, we remove all Cython modules and 
packages from sys.modules and reimport their 2to3-ed sources so that we can 
run the complete compiler during the installation (to bootstrap parts of 
Cython into binary modules).


Since the above revision, this process bails out with an error when 
accessing "os.path" because "os" is None. The "os" module is imported 
globally in our early-imported build_ext module, more or less like this:


import os

from distutils.command import build_ext as _build_ext

class build_ext(_build_ext.build_ext):

def build_extensions(self):
print(os) # prints None!

I suspect that the fact that we remove the modules from sys.modules somehow 
triggers the cleanup of these modules while there are still objects from 
these modules alive that refer to their globals. So, what I think is 
happening is that the module cleanup sets the module's globals to None 
before the objects from that module that refer to these globals have 
actually gone out of scope.


Could someone (benjamin?) please look into this?

Thanks,

Stefan

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] SVN rev. 85392 broke module handling in py3k

2010-10-22 Thread Benjamin Peterson
2010/10/22 Stefan Behnel :
> Hi,
>
> since SVN rev. 85392, Cython's installation fails on the py3k branch with a
> weird globals error. I think it is related to some sys.modules magic that we
> do in order to support running Cython in Python 3 using lib2to3.
>
> Basically, what we do is, we import some parts of Cython at the beginning
> that are Py3 clean, specifically some distutils build_ext replacement for
> building Cython modules. Then we start up distutils, which first runs
> lib2to3 on Cython's sources to convert them into Py3 code. When it then gets
> to building the binary modules, we remove all Cython modules and packages
> from sys.modules and reimport their 2to3-ed sources so that we can run the
> complete compiler during the installation (to bootstrap parts of Cython into
> binary modules).
>
> Since the above revision, this process bails out with an error when
> accessing "os.path" because "os" is None. The "os" module is imported
> globally in our early-imported build_ext module, more or less like this:
>
>    import os
>
>    from distutils.command import build_ext as _build_ext
>
>    class build_ext(_build_ext.build_ext):
>
>        def build_extensions(self):
>            print(os) # prints None!
>
> I suspect that the fact that we remove the modules from sys.modules somehow
> triggers the cleanup of these modules while there are still objects from
> these modules alive that refer to their globals. So, what I think is
> happening is that the module cleanup sets the module's globals to None
> before the objects from that module that refer to these globals have
> actually gone out of scope.
>
> Could someone (benjamin?) please look into this?

Is this broken before 2.7, ie 2.6 and 2.6?



-- 
Regards,
Benjamin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] 3.1.3 and 2.7.1 release schedule

2010-10-22 Thread Benjamin Peterson
2010/10/22 Dirkjan Ochtman :
> On Fri, Oct 22, 2010 at 00:57, Benjamin Peterson  wrote:
>> In the interest of getting 3.1.3 and 2.7.1 out by next year, here's a
>> tentative release schedule:
>>
>> November 13th - RC1
>> November 27th - RC2
>> December 11th - Final
>
> The last one might clash with the hg migration a bit, do we need to
> worry about that? Or did you purposely pick the day before the planned
> hg migration?

I'm not too worried. Commits should be at a minimum, and changesets
can be tagged post-transition if needed.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] SVN rev. 85392 broke module handling in py3k

2010-10-22 Thread Stefan Behnel

Benjamin Peterson, 22.10.2010 16:03:

2010/10/22 Stefan Behnel:

since SVN rev. 85392, Cython's installation fails on the py3k branch with a
weird globals error. I think it is related to some sys.modules magic that we
do in order to support running Cython in Python 3 using lib2to3.

Basically, what we do is, we import some parts of Cython at the beginning
that are Py3 clean, specifically some distutils build_ext replacement for
building Cython modules. Then we start up distutils, which first runs
lib2to3 on Cython's sources to convert them into Py3 code. When it then gets
to building the binary modules, we remove all Cython modules and packages
from sys.modules and reimport their 2to3-ed sources so that we can run the
complete compiler during the installation (to bootstrap parts of Cython into
binary modules).

Since the above revision, this process bails out with an error when
accessing "os.path" because "os" is None. The "os" module is imported
globally in our early-imported build_ext module, more or less like this:

import os

from distutils.command import build_ext as _build_ext

class build_ext(_build_ext.build_ext):

def build_extensions(self):
print(os) # prints None!

I suspect that the fact that we remove the modules from sys.modules somehow
triggers the cleanup of these modules while there are still objects from
these modules alive that refer to their globals. So, what I think is
happening is that the module cleanup sets the module's globals to None
before the objects from that module that refer to these globals have
actually gone out of scope.

Could someone (benjamin?) please look into this?


Is this broken before 2.7, ie 2.6 and 2.6?


I can't tell. Py2 doesn't need 2to3, so we don't unload the modules there.

Stefan

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] SVN rev. 85392 broke module handling in py3k

2010-10-22 Thread Benjamin Peterson
2010/10/22 Stefan Behnel :
> Benjamin Peterson, 22.10.2010 16:03:
>>
>> 2010/10/22 Stefan Behnel:
>>>
>>> since SVN rev. 85392, Cython's installation fails on the py3k branch with
>>> a
>>> weird globals error. I think it is related to some sys.modules magic that
>>> we
>>> do in order to support running Cython in Python 3 using lib2to3.
>>>
>>> Basically, what we do is, we import some parts of Cython at the beginning
>>> that are Py3 clean, specifically some distutils build_ext replacement for
>>> building Cython modules. Then we start up distutils, which first runs
>>> lib2to3 on Cython's sources to convert them into Py3 code. When it then
>>> gets
>>> to building the binary modules, we remove all Cython modules and packages
>>> from sys.modules and reimport their 2to3-ed sources so that we can run
>>> the
>>> complete compiler during the installation (to bootstrap parts of Cython
>>> into
>>> binary modules).
>>>
>>> Since the above revision, this process bails out with an error when
>>> accessing "os.path" because "os" is None. The "os" module is imported
>>> globally in our early-imported build_ext module, more or less like this:
>>>
>>>    import os
>>>
>>>    from distutils.command import build_ext as _build_ext
>>>
>>>    class build_ext(_build_ext.build_ext):
>>>
>>>        def build_extensions(self):
>>>            print(os) # prints None!
>>>
>>> I suspect that the fact that we remove the modules from sys.modules
>>> somehow
>>> triggers the cleanup of these modules while there are still objects from
>>> these modules alive that refer to their globals. So, what I think is
>>> happening is that the module cleanup sets the module's globals to None
>>> before the objects from that module that refer to these globals have
>>> actually gone out of scope.
>>>
>>> Could someone (benjamin?) please look into this?
>>
>> Is this broken before 2.7, ie 2.6 and 2.6?
>
> I can't tell. Py2 doesn't need 2to3, so we don't unload the modules there.

What about 3.1.0 then?



-- 
Regards,
Benjamin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] SVN rev. 85392 broke module handling in py3k

2010-10-22 Thread exarkun

On 02:13 pm, [email protected] wrote:

Benjamin Peterson, 22.10.2010 16:03:

2010/10/22 Stefan Behnel:
since SVN rev. 85392, Cython's installation fails on the py3k branch 
with a
weird globals error. I think it is related to some sys.modules magic 
that we

do in order to support running Cython in Python 3 using lib2to3.

Basically, what we do is, we import some parts of Cython at the 
beginning
that are Py3 clean, specifically some distutils build_ext replacement 
for

building Cython modules. Then we start up distutils, which first runs
lib2to3 on Cython's sources to convert them into Py3 code. When it 
then gets
to building the binary modules, we remove all Cython modules and 
packages
from sys.modules and reimport their 2to3-ed sources so that we can 
run the
complete compiler during the installation (to bootstrap parts of 
Cython into

binary modules).

Since the above revision, this process bails out with an error when
accessing "os.path" because "os" is None. The "os" module is imported
globally in our early-imported build_ext module, more or less like 
this:


import os

from distutils.command import build_ext as _build_ext

class build_ext(_build_ext.build_ext):

def build_extensions(self):
print(os) # prints None!

I suspect that the fact that we remove the modules from sys.modules 
somehow
triggers the cleanup of these modules while there are still objects 
from

these modules alive that refer to their globals. So, what I think is
happening is that the module cleanup sets the module's globals to 
None

before the objects from that module that refer to these globals have
actually gone out of scope.


Instances of classes don't refer to the module their class is defined 
in.  It seems more likely that the reason the module is garbage 
collected is that there really is nothing which refers to it anymore.


The behavior of setting the attributes of a module being freed to None 
has been in place for a long time, r85392 only restored it after a brief 
absence.


Perhaps Cython itself should keep the modules alive that it wants kept 
alive.  Alternatively, if Cython owns the code that's running into the 
zapped global, you could change it to not use globals.


Jean-Paul
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] SVN rev. 85392 broke module handling in py3k

2010-10-22 Thread Stefan Behnel

Benjamin Peterson, 22.10.2010 16:17:

2010/10/22 Stefan Behnel:

Benjamin Peterson, 22.10.2010 16:03:

2010/10/22 Stefan Behnel:


since SVN rev. 85392, Cython's installation fails on the py3k branch with
a
weird globals error. I think it is related to some sys.modules magic that
we
do in order to support running Cython in Python 3 using lib2to3.

Basically, what we do is, we import some parts of Cython at the beginning
that are Py3 clean, specifically some distutils build_ext replacement for
building Cython modules. Then we start up distutils, which first runs
lib2to3 on Cython's sources to convert them into Py3 code. When it then
gets
to building the binary modules, we remove all Cython modules and packages
from sys.modules and reimport their 2to3-ed sources so that we can run
the
complete compiler during the installation (to bootstrap parts of Cython
into binary modules).

Since the above revision, this process bails out with an error when
accessing "os.path" because "os" is None. The "os" module is imported
globally in our early-imported build_ext module, more or less like this:

import os

from distutils.command import build_ext as _build_ext

class build_ext(_build_ext.build_ext):

def build_extensions(self):
print(os) # prints None!

I suspect that the fact that we remove the modules from sys.modules
somehow
triggers the cleanup of these modules while there are still objects from
these modules alive that refer to their globals. So, what I think is
happening is that the module cleanup sets the module's globals to None
before the objects from that module that refer to these globals have
actually gone out of scope.

Could someone (benjamin?) please look into this?


Is this broken before 2.7, ie 2.6 and 2.6?


I can't tell. Py2 doesn't need 2to3, so we don't unload the modules there.


What about 3.1.0 then?


The 3.1.2 release was fine, but the current 3.1 SVN branch is not. I assume 
you have applied the change there, too?


py3.1 branch:
https://sage.math.washington.edu:8091/hudson/job/cython-devel-build-py31/524/console

py3k branch:
https://sage.math.washington.edu:8091/hudson/job/cython-devel-build-py3k/613/console

We have our CI tests running against both branches, as well as all Py2 
branches starting from 2.3. That's how I spotted it.


Stefan

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] SVN rev. 85392 broke module handling in py3k

2010-10-22 Thread Benjamin Peterson
2010/10/22  :
> Instances of classes don't refer to the module their class is defined in.
>  It seems more likely that the reason the module is garbage collected is
> that there really is nothing which refers to it anymore.

Indeed, this is really a Python bug, but there's no good way to deal
with it unless dictionaries can know they are module globals.


-- 
Regards,
Benjamin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] SVN rev. 85392 broke module handling in py3k

2010-10-22 Thread Benjamin Peterson
2010/10/22 Stefan Behnel :
> Benjamin Peterson, 22.10.2010 16:17:
>> What about 3.1.0 then?
>
> The 3.1.2 release was fine, but the current 3.1 SVN branch is not. I assume
> you have applied the change there, too?

Yes. Unfortunately, this behavior is more "correct" for most cases
because it results in module globals being finalized.

You could run 2to3 in a subprocess or hang onto modules you to keep
the globals of while you're using them.


-- 
Regards,
Benjamin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] SVN rev. 85392 broke module handling in py3k

2010-10-22 Thread Antoine Pitrou
On Fri, 22 Oct 2010 09:41:09 -0500
Benjamin Peterson  wrote:
> 2010/10/22  :
> > Instances of classes don't refer to the module their class is defined in.
> >  It seems more likely that the reason the module is garbage collected is
> > that there really is nothing which refers to it anymore.
> 
> Indeed, this is really a Python bug, but there's no good way to deal
> with it unless dictionaries can know they are module globals.

How about making functions keep a reference to the module they're
defined in? Is there any reason we shouldn't do that?

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] SVN rev. 85392 broke module handling in py3k

2010-10-22 Thread Stefan Behnel

[email protected], 22.10.2010 16:32:

Instances of classes don't refer to the module their class is defined
in. It seems more likely that the reason the module is garbage collected
is that there really is nothing which refers to it anymore.

The behavior of setting the attributes of a module being freed to None
has been in place for a long time, r85392 only restored it after a brief
absence.

Perhaps Cython itself should keep the modules alive that it wants kept
alive.


Given that this only happens during an install process, this works for me.

Thanks,

Stefan

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] SVN rev. 85392 broke module handling in py3k

2010-10-22 Thread Benjamin Peterson
2010/10/22 Antoine Pitrou :
> On Fri, 22 Oct 2010 09:41:09 -0500
> Benjamin Peterson  wrote:
>> 2010/10/22  :
>> > Instances of classes don't refer to the module their class is defined in.
>> >  It seems more likely that the reason the module is garbage collected is
>> > that there really is nothing which refers to it anymore.
>>
>> Indeed, this is really a Python bug, but there's no good way to deal
>> with it unless dictionaries can know they are module globals.
>
> How about making functions keep a reference to the module they're
> defined in? Is there any reason we shouldn't do that?

I thought of that, too. It wouldn't be trivial to implement, though,
and wouldn't solve the problem of clearing globals to avoid references
cycles.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] SVN rev. 85392 broke module handling in py3k

2010-10-22 Thread Stefan Behnel

Antoine Pitrou, 22.10.2010 16:55:

On Fri, 22 Oct 2010 09:41:09 -0500
Benjamin Peterson wrote:

2010/10/22 exarkun:

Instances of classes don't refer to the module their class is defined in.
  It seems more likely that the reason the module is garbage collected is
that there really is nothing which refers to it anymore.


Indeed, this is really a Python bug, but there's no good way to deal
with it unless dictionaries can know they are module globals.


How about making functions keep a reference to the module they're
defined in? Is there any reason we shouldn't do that?


I think the general problem is that "the module" can be a pretty broad 
thing, potentially referring to all sorts of stuff such as (usually) 
several other modules. Think of an extreme case where you build and return 
a local function in a module function. Now all you use the module for is to 
get at the outer function and call it once to get an instance of the inner 
function. But the module must stay alive as long as the inner function 
lives, even if its closure is completely self-sufficient. You could 
optimise this specific case, but as soon as as the function refers to any 
non-local name, you'd have to keep the entire module dict alive in case 
someone decides to change it.


It would also mean that each and every function will end up in a reference 
cycle by default that will require garbage collection for cleanup. This 
isn't currently a common case, as only modules refer to their content and 
rarely the other way round.


Augmenting the function closure to include (indirections to) module globals 
won't work very well either, as it would either mean you still keep the 
module dict alive or it would complicate modifications to the module dict 
that would need to be reflected in the closures in some way. That gets us 
back to the usual space-time tradeoff, out of which we currently prefer 
both at the cost of less safe behaviour in the (rather rare) case of module 
unloading.


Stefan

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Summary of Python tracker Issues

2010-10-22 Thread Python tracker

ACTIVITY SUMMARY (2010-10-15 - 2010-10-22)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues stats:
  open2494 (+32)
  closed 19460 (+110)
  total  21954 (+56)

Open issues with patches: 1039 


Issues opened (32)
==

#3367: Uninitialized value read in parsetok.c
http://bugs.python.org/issue3367  reopened by skrah

#10117: Tools/scripts/reindent.py fails on non-UTF-8 encodings
http://bugs.python.org/issue10117  opened by belopolsky

#10118: Tkinter does not find font
http://bugs.python.org/issue10118  opened by mark.saaltink

#10121: test_multiprocessing stuck in test_make_pool if run in a loop
http://bugs.python.org/issue10121  opened by sandro.tosi

#10126: test_distutils failure with --enable-shared
http://bugs.python.org/issue10126  opened by pitrou

#10128: multiprocessing.Pool throws exception with __main__.py
http://bugs.python.org/issue10128  opened by Michael.Olson

#10130: Create epub format docs and offer them on the download page
http://bugs.python.org/issue10130  opened by georg.brandl

#10131: deepcopying an xml.dom.minidom.Document generates an invalid X
http://bugs.python.org/issue10131  opened by flox

#10133: multiprocessing: conn_recv_string() broken error handling
http://bugs.python.org/issue10133  opened by hfuru

#10134: test_email failures on Windows: end of line issue?
http://bugs.python.org/issue10134  opened by haypo

#10136: kill_python doesn't work with short path
http://bugs.python.org/issue10136  opened by ocean-city

#10138: calendar module does not support years outside [1, ] range
http://bugs.python.org/issue10138  opened by belopolsky

#10141: SocketCan support
http://bugs.python.org/issue10141  opened by instigatorirc

#10142: Support for SEEK_HOLE/SEEK_DATA
http://bugs.python.org/issue10142  opened by jcea

#10143: Update "os.pathconf" values
http://bugs.python.org/issue10143  opened by jcea

#10145: float.is_integer is undocumented
http://bugs.python.org/issue10145  opened by ralph.corderoy

#10148: st_mtime differs after shutil.copy2
http://bugs.python.org/issue10148  opened by shaurz

#10149: Data truncation in expat parser
http://bugs.python.org/issue10149  opened by Maciek.J

#10153: Memory leak in pythonrun.c
http://bugs.python.org/issue10153  opened by skrah

#10154: locale.normalize strips "-" from UTF-8, which fails on Mac
http://bugs.python.org/issue10154  opened by ixokai

#10155: Add fixups for encoding problems to wsgiref
http://bugs.python.org/issue10155  opened by aclover

#10156: Initialization of globals in unicodeobject.c
http://bugs.python.org/issue10156  opened by skrah

#10157: Memory leak (r70152)
http://bugs.python.org/issue10157  opened by skrah

#10158: BadInternalCall running test_multiprocessing
http://bugs.python.org/issue10158  opened by rthalley

#10160: operator.attrgetter slower than lambda after adding dotted nam
http://bugs.python.org/issue10160  opened by tzot

#10161: unittest: use ascii() instead of repr() to display values on e
http://bugs.python.org/issue10161  opened by haypo

#10164: Add an assertBytesEqual to unittest and use it for bytes asser
http://bugs.python.org/issue10164  opened by r.david.murray

#10167: ESET Trojan Alert [python-3.1.2.amd64 ON Win7-64]
http://bugs.python.org/issue10167  opened by Mr.Collins

#10169: socket.sendto raises incorrect exception when passed incorrect
http://bugs.python.org/issue10169  opened by exarkun

#10170: Relationship between turtle speed setting and actual speed is 
http://bugs.python.org/issue10170  opened by belopolsky

#10171: Ugly buttons in some Tkinter objects in Windows
http://bugs.python.org/issue10171  opened by rafe.kettler

#10172: code block has no syntax coloring
http://bugs.python.org/issue10172  opened by wcyang



Most recent 15 issues with no replies (15)
==

#10172: code block has no syntax coloring
http://bugs.python.org/issue10172

#10171: Ugly buttons in some Tkinter objects in Windows
http://bugs.python.org/issue10171

#10170: Relationship between turtle speed setting and actual speed is 
http://bugs.python.org/issue10170

#10169: socket.sendto raises incorrect exception when passed incorrect
http://bugs.python.org/issue10169

#10161: unittest: use ascii() instead of repr() to display values on e
http://bugs.python.org/issue10161

#10138: calendar module does not support years outside [1, ] range
http://bugs.python.org/issue10138

#10136: kill_python doesn't work with short path
http://bugs.python.org/issue10136

#10133: multiprocessing: conn_recv_string() broken error handling
http://bugs.python.org/issue10133

#10130: Create epub format docs and offer them on the download page
http://bugs.python.org/issue10130

#10118: Tkinter does not find font
http://bugs.python.org/issue10118

#10112: Use -Wl,--dynamic-list=x.list, not -Xlinker -export-dynamic
http://bugs.python.org/issue10112

#10090: p

Re: [Python-Dev] SVN rev. 85392 broke module handling in py3k

2010-10-22 Thread Antoine Pitrou
On Fri, 22 Oct 2010 17:37:53 +0200
Stefan Behnel  wrote:
> 
> I think the general problem is that "the module" can be a pretty broad 
> thing, potentially referring to all sorts of stuff such as (usually) 
> several other modules.

I wouldn't think of unloading modules as a general problem. We should
support it for various reasons (including your use case), but if it
takes a further garbage collection pass to get rid of all stale
objects, then so be it.

> It would also mean that each and every function will end up in a reference 
> cycle by default that will require garbage collection for cleanup.

This is already the case, since most functions are in a reference cycle
with their own globals dict (== module dict). It's just that the
reference cycle would then include the module object as well, but I
don't see it as a big drawback. The module object holds little to no
state except its dict, precisely.

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Exposing pkguitl's import emulation (was Re: [Python-checkins] r85538 - python/branches/py3k/Doc/library/pkgutil.rst)

2010-10-22 Thread Georg Brandl
Am 19.10.2010 17:24, schrieb P.J. Eby:
> At 08:03 AM 10/18/2010 +1000, Nick Coghlan wrote:
>>I'm a little dubious about exposing these officially. They're mainly a
>>hack to get some parts of the standard library working (e.g. runpy) in
>>the absence of full PEP 302 support in the imp module, not really
>>something we want to encourage anyone else to use (and yes, they
>>should probably have underscores in their names, but we missed that
>>when the various private implementations scattered around the stdlib
>>were consolidated in pkgutil).
> 
> Well, my intention at least was that they should be documented and 
> released; it's the documenting part I didn't get around to.  ;-)
> 
> Of course, this was also pre-importlib; were we starting the work 
> today, the obvious thing to do would be to expose the Python 
> implementations of the relevant objects.

I don't care much either way; however I don't really like when there are
public APIs (i.e. non-underscore-prefixed globals in a non-underscore-
prefixed module) that aren't documented, because it is confusing to
developers who don't know if they can use it or not.  (See re.scanner.)

The best thing is probably to let Brett (Hello Brett!) determine how
much of it can be replaced by importlib, and add a note to that effect
to the pkgutil docs.

cheers,
Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Distutils2 scripts

2010-10-22 Thread Michael Foord

 On 22/10/2010 08:20, Paul Moore wrote:

On 22 October 2010 04:31, Ron Adam  wrote:

When it's in the stdlib, the -m option should work just like any other
script run from the stdlib.

What path hacking are you thinking of?

On Windows, neither the "python" executable nor scripts in
C:\Pythonxx\Scripts are in the PATH by default. On the other hand, .py
files will run automatically via the registered file extension.

Manipulating PATH at install time (to add C:\PythonXX and/or
C:\PythonXX\Scripts) is not done - it is (rightly, in my view)
considered too difficult to get right, particularly when it comes to
uninstalling.

Many Windows users will, I guess, manually add python to PATH (so that
python-m works). Some people also add C:\PythonXX\Scripts. Personally,
I don't - so for me a pysetup script in that location would be no use.



Well, that is where pip and other scripts installed by Python go, so it 
is the 'right' place for scripts to live.


Any reason not to allow both though? (python -m and an explicit script)

For what its worth I have the same issue with unittest / unittest2. Test 
discovery and test running in Python 2.7 / 3.2 is done with:


python -m unittest args

As unittest2 is a package and supports Python 2.6 (and earlier), python 
-m unittest2 doesn't work so I provide a "unit2" script for accessing 
its functionality. I *much* prefer using "unit2 ..." to "python -m 
unittest ...".


Michael


So my personal vote is +1 for a python -m approach, and -0 for a
pysetup executable. I'm -1 on a pysetup.bat batch file - bat files
have other issues which IMO make them effectively useless.

Paul.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk



--

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (”BOGUS AGREEMENTS”) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Summary of Python tracker Issues

2010-10-22 Thread Antoine Pitrou
On Fri, 22 Oct 2010 18:08:20 +0200 (CEST)
Python tracker  wrote:
> 
> ACTIVITY SUMMARY (2010-10-15 - 2010-10-22)
> Python tracker at http://bugs.python.org/
> 
> To view or respond to any of the issues listed below, click on the issue.
> Do NOT respond to this message.
> 
> Issues stats:
>   open2494 (+32)
>   closed 19460 (+110)
>   total  21954 (+56)

The figures in parentheses look wrong. Last week, the stats said:

> Issues stats:
>   open2557 (+37)
>   closed 19276 (+30)
>   total  21833 (+44)

So, logically, this week's stats should say:

  open2494 (-63)
  closed 19460 (+184)
  total  21954 (+121)

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Exposing pkguitl's import emulation (was Re: [Python-checkins] r85538 - python/branches/py3k/Doc/library/pkgutil.rst)

2010-10-22 Thread Brett Cannon
On Fri, Oct 22, 2010 at 09:26, Georg Brandl  wrote:
> Am 19.10.2010 17:24, schrieb P.J. Eby:
>> At 08:03 AM 10/18/2010 +1000, Nick Coghlan wrote:
>>>I'm a little dubious about exposing these officially. They're mainly a
>>>hack to get some parts of the standard library working (e.g. runpy) in
>>>the absence of full PEP 302 support in the imp module, not really
>>>something we want to encourage anyone else to use (and yes, they
>>>should probably have underscores in their names, but we missed that
>>>when the various private implementations scattered around the stdlib
>>>were consolidated in pkgutil).
>>
>> Well, my intention at least was that they should be documented and
>> released; it's the documenting part I didn't get around to.  ;-)
>>
>> Of course, this was also pre-importlib; were we starting the work
>> today, the obvious thing to do would be to expose the Python
>> implementations of the relevant objects.
>
> I don't care much either way; however I don't really like when there are
> public APIs (i.e. non-underscore-prefixed globals in a non-underscore-
> prefixed module) that aren't documented, because it is confusing to
> developers who don't know if they can use it or not.  (See re.scanner.)
>
> The best thing is probably to let Brett (Hello Brett!) determine how
> much of it can be replaced by importlib, and add a note to that effect
> to the pkgutil docs.

The pkgutil stuff that was exposed cannot be directly replaced with a
public API in Python 3.2, but the plan is that it will be in Python
3.3 when *all* concrete implementations of importers are exposed
(because I will be attempting to bootstrap importlib). So if people
are willing to wait and take me at my word that this will happen in
Python 3.3, then this can come back out. But obviously I cannot make
promises as Real Life will *actually* be starting for me when the
Python 3.3 development cycle begins.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] My work on Python3 and non-ascii paths is done

2010-10-22 Thread Stephen J. Turnbull
First, let me offer congratulations and heartfelt thanks for your hard
work!

Victor Stinner writes:

 > For network protocols, I don't know. It looks like the new email
 > modules will offer two API levels: low level (native type) using
 > bytes, high level using str (unicode). I don't know if the high
 > level API uses the PEP 383 or not.

Give that about 0.3% of the mail in my system uses the UNKNOWN
encoding in MIME-words, "best effort" to decode headers will require
it.  (Half-decoded text is a nightmare, of course.)

 > I don't use strict rules. Each problem is different.

I agree.  The reason is that the application must be designed to
handle PEP 383 strings.  If it isn't, you're just postponing the
exception (perhaps to an unsuspecting second application).  PEP 383
strings are an excellent, consistent way to handle broken text if your
application is like email, where the design specifies passing around
uninterpreted data, but transforming the representation is desirable.
But many applications *should* just raise an exception here.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com