Re: [Python-Dev] [Python-checkins] cpython: Issue #5863: Rewrite BZ2File in pure Python, and allow it to accept

2011-04-04 Thread Georg Brandl 
Am 03.04.2011 23:28, schrieb Nadeem Vawda:
> On Sun, Apr 3, 2011 at 8:02 PM, Antoine Pitrou  wrote:
>> On Sun, 03 Apr 2011 18:55:33 +0200
>> Éric Araujo  wrote:
>>> I think we use Misc/ACKS for code+docs contribution like this one,
>>> Doc/ACKS.txt being used for doc-only changes.  This second file is not
>>> comprehensive nor always used though, so maybe it should be superseded
>>> by the former.
>>
>> Nadeem is already in Misc/ACKS.  I don't know what the policy is for
>> Doc/ACKS.txt, but since he added himself in the patch, I saw no good
>> reason for reverting the change.
> 
> I added myself because I assumed the policy for Doc/ACKS.txt to be the same
> as the policy for Misc/ACKS - if you submit a patch, add your name. Looking
> at the devguide, though, I can't find any mention of Doc/ACKS.txt.
> 
>> +1 for merging these files by the way.
> 
> Sounds good to me. The intro at the top of Misc/ACKS is pretty broad,
> thanking people for all contributions (not just code). Unless there's some
> plan to split the documentation off into a separate repository, I can't
> think of any reason not to merge them.

If we can get Misc/ACKS in a format that is includable in reST, I would be
+1 for a merger.  (That way we can still list acknowledgements in the docs.)

Georg

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Neil Schemenauer 
As a user of the AST, I as well favor just changing the AST and the
version.  IMHO, it is not intended to be stable between Python
releases (similar to bytecode).

  Neil

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Terry Reedy 

On 4/3/2011 10:02 PM, Guido van Rossum wrote:


Sure, but do we have any indication that the warnings for dis make a difference?


I think there had been a few grumbles about bytecode not being stable. 
Without that, it is part of the newish effort to specify in the docs 
what is CPython specific. In http://bugs.python.org/issue11762

I propose a lighter version of the dis notice:

"CPython implementation detail: The ast definition is specific to the 
CPython interpreter! Ast nodes may be added, removed, or changed between 
versions. Use *ast.__version__* to work across versions."


and that ast.__version__ get a normal formal entry

ast.__version__
String constant with version number of the abstract grammar file.
3.1: '67616'; 3.2: '82163'; 3.3: 'x'

--
Terry Jan Reedy

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread [email protected] 
As a re-implementor of ast.py that tries to be node for node
compatible, I'm fine with #1 but would really like to have tests that
will fail in test_ast.py to alert me!

-Frank
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Policy for versions of system python

2011-04-04 Thread Amaury Forgeot d'Arc 
2011/4/4 Eugene Toder :
> Hello,
>
> CPython source code currently contains a number of python scripts (e.g
> Python/makeopcodetargets.py, Objects/typeslots.py, Parser/asdl_c.py)
> that are used during the build of the python interpreter itself. For
> this reason they are run with system installed python. What is the
> policy regarding
> the range of python versions that they should support?
>
> I looked at some of the scripts and they seem to support both 2 and 3,
> starting from at most 2.4. Python/makeopcodetargets.py says at the
> top:
> # This code should stay compatible with Python 2.3, at least while
> # some of the buildbots have Python 2.3 as their system Python.
> Is this the official minimal version or do we have this spelled out
> more explicitly somewhere?

Normally PEP291 lists the packages which should remain compatible
with previous versions of Python:
http://www.python.org/dev/peps/pep-0291/

makeopcodetargets.py is not mentioned there, though.

-- 
Amaury Forgeot d'Arc
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Policy for versions of system python

2011-04-04 Thread Antoine Pitrou 
On Mon, 4 Apr 2011 19:11:52 +0200
"Amaury Forgeot d'Arc"  wrote:
> 2011/4/4 Eugene Toder :
> > Hello,
> >
> > CPython source code currently contains a number of python scripts (e.g
> > Python/makeopcodetargets.py, Objects/typeslots.py, Parser/asdl_c.py)
> > that are used during the build of the python interpreter itself. For
> > this reason they are run with system installed python. What is the
> > policy regarding
> > the range of python versions that they should support?
> >
> > I looked at some of the scripts and they seem to support both 2 and 3,
> > starting from at most 2.4. Python/makeopcodetargets.py says at the
> > top:
> > # This code should stay compatible with Python 2.3, at least while
> > # some of the buildbots have Python 2.3 as their system Python.
> > Is this the official minimal version or do we have this spelled out
> > more explicitly somewhere?
> 
> Normally PEP291 lists the packages which should remain compatible
> with previous versions of Python:
> http://www.python.org/dev/peps/pep-0291/

That's quite orthogonal. PEP 291 is about public stdlib modules, not
build scripts. Furthermore, “this PEP has no bearing on the Python 3
standard library”.

To answer Eugene's question, there's no official policy but the
comment at the top of Python/makeopcodetargets.py can indeed serve as
an useful guideline. I wonder if we still have buildbots with 2.3 as
the system Python, by the way.

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Michael Foord 

On 04/04/2011 18:05, [email protected] wrote:

As a re-implementor of ast.py that tries to be node for node
compatible, I'm fine with #1 but would really like to have tests that
will fail in test_ast.py to alert me!



A lot of tools that work with Python source code use ast - so even 
though other implementations may not use the same ast "under the hood" 
they will probably at least *want* to provide a compatible 
implementation. IronPython is in that boat too (although I don't know if 
we *have* a compatible implementation yet - we certainly feel like we 
*should* have one).


Michael


-Frank
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk



--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Guido van Rossum 
On Mon, Apr 4, 2011 at 10:05 AM, [email protected]
 wrote:
> As a re-implementor of ast.py that tries to be node for node
> compatible, I'm fine with #1 but would really like to have tests that
> will fail in test_ast.py to alert me!

[and]

On Mon, Apr 4, 2011 at 10:38 AM, Michael Foord
 wrote:
> A lot of tools that work with Python source code use ast - so even though
> other implementations may not use the same ast "under the hood" they will
> probably at least *want* to provide a compatible implementation. IronPython
> is in that boat too (although I don't know if we *have* a compatible
> implementation yet - we certainly feel like we *should* have one).

Ok, so it sounds like ast is *not* limited to CPython? That makes it
harder to justify changing it just so as to ease the compilation
process in CPython (as opposed to add new language features).

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Glyph Lefkowitz 

On Apr 4, 2011, at 2:00 PM, Guido van Rossum wrote:

> On Mon, Apr 4, 2011 at 10:05 AM, [email protected]
>  wrote:
>> As a re-implementor of ast.py that tries to be node for node
>> compatible, I'm fine with #1 but would really like to have tests that
>> will fail in test_ast.py to alert me!
> 
> [and]
> 
> On Mon, Apr 4, 2011 at 10:38 AM, Michael Foord
>  wrote:
>> A lot of tools that work with Python source code use ast - so even though
>> other implementations may not use the same ast "under the hood" they will
>> probably at least *want* to provide a compatible implementation. IronPython
>> is in that boat too (although I don't know if we *have* a compatible
>> implementation yet - we certainly feel like we *should* have one).
> 
> Ok, so it sounds like ast is *not* limited to CPython?

Oh, definitely not.  I would be pretty dismayed if tools like 

 would not run on Jython & PyPy.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Dino Viehland 

Guido wrote:
> On Mon, Apr 4, 2011 at 10:05 AM, [email protected]
>  wrote:
> > As a re-implementor of ast.py that tries to be node for node
> > compatible, I'm fine with #1 but would really like to have tests that
> > will fail in test_ast.py to alert me!
> 
> [and]
> 
> On Mon, Apr 4, 2011 at 10:38 AM, Michael Foord
>  wrote:
> > A lot of tools that work with Python source code use ast - so even
> > though other implementations may not use the same ast "under the hood"
> > they will probably at least *want* to provide a compatible
> > implementation. IronPython is in that boat too (although I don't know
> > if we *have* a compatible implementation yet - we certainly feel like we
> *should* have one).
> 
> Ok, so it sounds like ast is *not* limited to CPython? That makes it harder to
> justify changing it just so as to ease the compilation process in CPython (as
> opposed to add new language features).

Even so I think adding new features does allow new changes to the AST.  We'll
need to do the work to add support for the new features anyway so updating the
AST accordingly won't be much more work.  I agree with Frank that as long as 
there
are tests for the new features it's fine.  I think it'll also be better for 
consumers who
would probably prefer to see a YieldFrom node rather than its expansion (and not
all new language features will necessarily have a reasonable  expansion - 
consider if
goto ever happened ;) ).

Also, IronPython doesn't have ast yet but I think it has been requested that we 
implement it - we just haven't gotten around to it yet.

So I'm +1 on allowing changes to it.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Policy for versions of system python

2011-04-04 Thread Ned Deily 
In article <[email protected]>,
 Antoine Pitrou  wrote:
> To answer Eugene's question, there's no official policy but the
> comment at the top of Python/makeopcodetargets.py can indeed serve as
> an useful guideline. I wonder if we still have buildbots with 2.3 as
> the system Python, by the way.

The system Python on Mac OS X 10.4 (Tiger) is Python 2.3.  For 10.5 
(Leopard) it's 2.5.  10.6 (Snow Leopard) has both 2.6 and 2.5.

-- 
 Ned Deily,
 [email protected]

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Floris Bruynooghe 
On 4 April 2011 19:07, Glyph Lefkowitz  wrote:
>
> On Apr 4, 2011, at 2:00 PM, Guido van Rossum wrote:
>
>> On Mon, Apr 4, 2011 at 10:05 AM, [email protected]
>>  wrote:
>>> As a re-implementor of ast.py that tries to be node for node
>>> compatible, I'm fine with #1 but would really like to have tests that
>>> will fail in test_ast.py to alert me!
>>
>> [and]
>>
>> On Mon, Apr 4, 2011 at 10:38 AM, Michael Foord
>>  wrote:
>>> A lot of tools that work with Python source code use ast - so even though
>>> other implementations may not use the same ast "under the hood" they will
>>> probably at least *want* to provide a compatible implementation. IronPython
>>> is in that boat too (although I don't know if we *have* a compatible
>>> implementation yet - we certainly feel like we *should* have one).
>>
>> Ok, so it sounds like ast is *not* limited to CPython?
>
> Oh, definitely not.  I would be pretty dismayed if tools like 
> 
>  would not run on Jython & PyPy.

Add py.test as an application that uses the AST to support Jython,
PyPy and CPython in a portable way.  I always assumed AST was created
*because* bytecode was too CPython specific (but then I've never
implemented a language).

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Terry Reedy 

On 4/4/2011 2:00 PM, Guido van Rossum wrote:

On Mon, Apr 4, 2011 at 10:05 AM, [email protected]
  wrote:

As a re-implementor of ast.py that tries to be node for node
compatible, I'm fine with #1 but would really like to have tests that
will fail in test_ast.py to alert me!


[and]

On Mon, Apr 4, 2011 at 10:38 AM, Michael Foord
  wrote:

A lot of tools that work with Python source code use ast - so even though
other implementations may not use the same ast "under the hood" they will
probably at least *want* to provide a compatible implementation. IronPython
is in that boat too (although I don't know if we *have* a compatible
implementation yet - we certainly feel like we *should* have one).


Ok, so it sounds like ast is *not* limited to CPython? That makes it
harder to justify changing it just so as to ease the compilation
process in CPython (as opposed to add new language features).


Harder, but not impossible. Moving optimizations from bytecode (where 
they are demonstrably a bit fragile) to ast manipulations (where we 
presume they will be more robust and can be broader) should be a win in 
itself and it also makes them potentially available to *other* 
implementations. (There would have been some advantage to making this 
change for 3.0 But there was also reason for as little change as needed, 
just as with unittest.)


Are at least some of the implementation methods similar enough that they 
could use the same AST? It is, after all, a *semantic* translation into 
another language, and that need not depend on subsequent transforation 
and compilation to the ultimate target. A multiple-implementation AST 
could still be x.y dependent.


--
Terry Jan Reedy

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Dino Viehland 
Terry wrote:
> Are at least some of the implementation methods similar enough that they
> could use the same AST? It is, after all, a *semantic* translation into 
> another
> language, and that need not depend on subsequent transforation and
> compilation to the ultimate target. A multiple-implementation AST could still
> be x.y dependent.

For IronPython we have our own AST which is closely tied to the DLR ASTs (our
AST nodes are actually subclasses of the core DLR Expression node which then
"reduce" to the core DLR nodes on-demand).  We already do a huge amount of
manipulation of those ASTs from optimizations (constant folding being the 
primary
one) to re-writing them completely for things like generators or sys.settrace 
support and 
other optimizations like runtime optimized fast exception support.  But our 
ASTs are 
probably sufficiently different and sufficiently tied to the DLR that we 
couldn't 
share the exact same optimizations  on the ASTs but it would probably make it 
easier to steal ideas from CPython if you did them at the AST level as well.

They also have other differences such as the fact that they're effectively 
immutable.
Likely when we implement the _ast module it'll just transform our ASTs into the 
shared
ASTs via some additional attributes we attach to our ASTs rather than making 
them the
core AST implementation.



___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Revert the Lib/test/test_bigmem.py changes from commit 17891566a478 (and a

2011-04-04 Thread Michael Foord 
On 25 March 2011 17:09, Antoine Pitrou  wrote:

> On Fri, 25 Mar 2011 17:44:26 +0100
> Éric Araujo  wrote:
> > Hi,
> >
> > > changeset:   68921:11dc3f270594
> > > user:Thomas Wouters 
> > > date:Fri Mar 25 11:42:37 2011 +0100
> > > summary:
> > >   Revert the Lib/test/test_bigmem.py changes from commit 17891566a478
> (and a
> > > few other assertEqual tests that snuck in), and expand the docstrings
> and
> > > comments explaining why and how these tests are supposed to work.
> >
> > Your commit message does not explain why you reverted the changes.  The
> > specific assert* methods give more useful messages than assertEqual in
> > case of failure.
>
> Because they don't go well with huge inputs?
>
> >>> s = "x" * (2**29)
> >>> case.assertEqual(s + "a", s + "b")
> Traceback (most recent call last):
>  File "", line 1, in 
>  File "/home/antoine/cpython/default/Lib/unittest/case.py", line 643,
> in assertEqual assertion_func(first, second, msg=msg)
>  File "/home/antoine/cpython/default/Lib/unittest/case.py", line 984,
> in assertMultiLineEqual secondlines = [second + '\n']
> MemoryError
>
>
http://bugs.python.org/issue11763

Michael


>
> (of course, the functions could just be fixed)
>
>
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>



-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Guido van Rossum 
On Mon, Apr 4, 2011 at 12:56 PM, Terry Reedy  wrote:
> Moving optimizations from bytecode (where they
> are demonstrably a bit fragile) to ast manipulations (where we presume they
> will be more robust and can be broader) should be a win in itself

I am still doubtful of that. While in theory it is easier to become
confused about what the bytecode means, in practice the bugs we had
due to bytecode optimization were based on misunderstandings and
unintended consequences that would have caused the *exact* same bug if
the optimization was done at the AST level. (E.g. various
mistreatments of -0, ignoring possible floating point misbehavior for
extreme values or situations.)

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Supporting Visual Studio 2010

2011-04-04 Thread Brian Curtin 
Would it be reasonable to begin supporting Visual Studio 2010 for Windows
builds of 3.3? I now have a personal interest in this happening for some
stuff at work, and there's been a lot of questions in the last few months
about when we'll support it coming from python-list, #python-dev, and in
person at PyCon.

I wasn't around for the transition from 2005 to 2008, but I see we have a
few sub-folders in PC/ for previous versions, so apparently we may support
multiple versions at one time. Does it make sense to start this process now
for a change to 2010?

If it's not feasible to release 3.3 from a 2010 build, when might we be able
to make the change? Keep in mind the 3.3 final release is almost a year and
a half away, and we already know that Microsoft is likely to pull the cord
on VS2008 Express at some point now that 2010 has been out for a while.


I'm willing to do the work on this, but I just want to make sure it's a
worthwhile effort.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Supporting Visual Studio 2010

2011-04-04 Thread Martin v. Löwis 
Am 04.04.2011 22:38, schrieb Brian Curtin:
> Would it be reasonable to begin supporting Visual Studio 2010 for
> Windows builds of 3.3?

Interesting question. The following concerns have played a role in the
past:
- depending on the timing of the next VS release, Python may actually
  want to skip VS 2010, and move right to VS 2012 (say).
- users have expressed concerns that they constantly need to upgrade
  VS releases when developing for Python. With VS Express, that concern
  may be reduced - but you still need to buy a full license if you want
  to support AMD64.
- users have also expressed concerns that old VS versions become
  unavailable; of course, these are different users (since the first
  ones have already bought copies of VS 2008). The counter-argument is
  that you can still get cheap copies on Ebay, but that may be a red
  herring.
- every time this comes up, people also suggest that we should stop
  building with VS, and use gcc in the first place.

> Does it make sense to start this
> process now for a change to 2010?

I'll abstain from a vote here, and I think it essentially comes down to
voting (or somebody putting the foot down saying "I want this now",
which really was the way it worked the last time(s)).

Somebody would need to take charge of this, and fix all the issues that
come up: incompatibilities, generation of backwards-compatible project
files, MSI packaging, getting licenses to buildbot operators.

So if you want to lead this, and the votes are generally in favor,
go ahead. Be prepared to do this *again* before the 3.3 release when
switching to the next VS release (and yes, Microsoft's timing had
been so unfortunate in the past that such a switch would have occurred
just before the first beta release of Python).

> If it's not feasible to release 3.3 from a 2010 build, when might we be
> able to make the change?

If we don't switch for 3.3, we'll definitely switch to VS 2012.

> I'm willing to do the work on this, but I just want to make sure it's a
> worthwhile effort.

See above: it may or may not.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Supporting Visual Studio 2010

2011-04-04 Thread Antoine Pitrou 
On Mon, 04 Apr 2011 23:40:33 +0200
"Martin v. Löwis"  wrote:
> - users have expressed concerns that they constantly need to upgrade
>   VS releases when developing for Python.

Isn't that kind of a misguided argument? It's not Python who decides the
lifecycle of MSVC releases, it's Microsoft. We can't be blamed for the
churn.

If getting old (Microsoft-unsupported) MSVC releases is difficult, then
I think switching to the newest MSVC as soon as possible is the best
strategy, since it minimizes the annoyance for people wanting to build
extensions several years after a release is made.

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Supporting Visual Studio 2010

2011-04-04 Thread Michael Foord 

On 04/04/2011 23:21, Antoine Pitrou wrote:

On Mon, 04 Apr 2011 23:40:33 +0200
"Martin v. Löwis"  wrote:

- users have expressed concerns that they constantly need to upgrade
   VS releases when developing for Python.

Isn't that kind of a misguided argument? It's not Python who decides the
lifecycle of MSVC releases, it's Microsoft. We can't be blamed for the
churn.

If getting old (Microsoft-unsupported) MSVC releases is difficult, then
I think switching to the newest MSVC as soon as possible is the best
strategy, since it minimizes the annoyance for people wanting to build
extensions several years after a release is made.


Won't that still be an issue despite the stable ABI? Extensions on 
Windows should be linked to the same version of MSVCRT used to compile 
Python - and every time we switch version of Visual Studio it is usually 
accompanied by a switch in MSVCRT version. So for C extensions compiled 
with a specific version of Python will need to be recompiled for later 
versions of Python, even if they only use the stable ABI, if the newer 
version of Python is compiled against a different MSVCRT. (?)


This would seem to circumvent one of the core use-cases of the stable 
ABI which was not needing to recompile extensions for new versions of 
Python. Of course I could be completely wrong about all this.


All the best,

Michael

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk



--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Supporting Visual Studio 2010

2011-04-04 Thread Scott Dial 
On 4/4/2011 6:43 PM, Michael Foord wrote:
> Won't that still be an issue despite the stable ABI? Extensions on
> Windows should be linked to the same version of MSVCRT used to compile
> Python - and every time we switch version of Visual Studio it is usually
> accompanied by a switch in MSVCRT version.

My understanding (but I haven't looked closely) was that the stable ABI
specifically excluded anything that would expose a problem due to a CRT
mismatch -- making this a moot point. I'm sure Martin will correct me if
I am wrong.

-Scott

-- 
Scott Dial
[email protected]
[email protected]
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Terry Reedy 

On 4/4/2011 4:05 PM, Dino Viehland wrote:


"reduce" to the core DLR nodes on-demand).  We already do a huge amount of
manipulation of those ASTs from optimizations (constant folding being the 
primary
one) to re-writing them completely for things like generators or sys.settrace 
support and
other optimizations like runtime optimized fast exception support.  But our 
ASTs are


I meant to add that doing optimization (and other manipulations) with 
AST would also make it easier to borrow from other implementations.


--
Terry Jan Reedy

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 399: Pure Python/C Accelerator Module Compatibiilty Requirements

2011-04-04 Thread Brett Cannon 
At both the VM and language summits at PyCon this year, the issue of
compatibility of the stdlib amongst the various VMs came up. Two issues came
about in regards to modules that use C code. One is that code that comes in
only as C code sucks for all other VMs that are not CPython since they all
end up having to re-implement that module themselves. Two is that modules
that have an accelerator module (e.g., heapq, warnings, etc.) can end up
with compatibility options (sorry, Raymond, for picking on heapq, but is was
what bit the PyPy people most recently =).

In lieu of all of this, here is a draft PEP to more clearly state the policy
for the stdlib when it comes to C code. Since this has come up before and
this was discussed so much at the summits I have gone ahead and checked this
in so that even if this PEP gets rejected there can be a written record as
to why.

And before anyone asks, I have already run this past the lead devs of PyPy,
Jython, and IronPython and they all support what this PEP proposes. And with
the devs of the other VMs gaining push privileges there shouldn't be an
added developer burden on everyone to make this PEP happen.

==
PEP: 399
Title: Pure Python/C Accelerator Module Compatibiilty Requirements
Version: $Revision: 88219 $
Last-Modified: $Date: 2011-01-27 13:47:00 -0800 (Thu, 27 Jan 2011) $
Author: Brett Cannon 
Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: 04-Apr-2011
Python-Version: 3.3
Post-History:

Abstract


The Python standard library under CPython contains various instances
of modules implemented in both pure Python and C. This PEP requires
that in these instances that both the Python and C code *must* be
semantically identical (except in cases where implementation details
of a VM prevents it entirely). It is also required that new C-based
modules lacking a pure Python equivalent implementation get special
permissions to be added to the standard library.


Rationale
=

Python has grown beyond the CPython virtual machine (VM). IronPython_,
Jython_, and PyPy_ all currently being viable alternatives to the
CPython VM. This VM ecosystem that has sprung up around the Python
programming language has led to Python being used in many different
areas where CPython cannot be used, e.g., Jython allowing Python to be
used in Java applications.

A problem all of the VMs other than CPython face is handling modules
from the standard library that are implemented in C. Since they do not
typically support the entire `C API of Python`_ they are unable to use
the code used to create the module. Often times this leads these other
VMs to either re-implement the modules in pure Python or in the
programming language used to implement the VM (e.g., in C# for
IronPython). This duplication of effort between CPython, PyPy, Jython,
and IronPython is extremely unfortunate as implementing a module *at
least* in pure Python would help mitigate this duplicate effort.

The purpose of this PEP is to minimize this duplicate effort by
mandating that all new modules added to Python's standard library
*must* have a pure Python implementation _unless_ special dispensation
is given. This makes sure that a module in the stdlib is available to
all VMs and not just to CPython.

Re-implementing parts (or all) of a module in C (in the case
of CPython) is still allowed for performance reasons, but any such
accelerated code must semantically match the pure Python equivalent to
prevent divergence. To accomplish this, the pure Python and C code must
be thoroughly tested with the *same* test suite to verify compliance.
This is to prevent users from accidentally relying
on semantics that are specific to the C code and are not reflected in
the pure Python implementation that other VMs rely upon, e.g., in
CPython 3.2.0, ``heapq.heappop()`` raises different exceptions
depending on whether the accelerated C code is used or not::

from test.support import import_fresh_module

c_heapq = import_fresh_module('heapq', fresh=['_heapq'])
py_heapq = import_fresh_module('heapq', blocked=['_heapq'])


class Spam:
"""Tester class which defines no other magic methods but
__len__()."""
def __len__(self):
return 0


try:
c_heapq.heappop(Spam())
except TypeError:
# "heap argument must be a list"
pass

try:
py_heapq.heappop(Spam())
except AttributeError:
# "'Foo' object has no attribute 'pop'"
pass

This kind of divergence is a problem for users as they unwittingly
write code that is CPython-specific. This is also an issue for other
VM teams as they have to deal with bug reports from users thinking
that they incorrectly implemented the module when in fact it was
caused by an untested case.


Details
===

Starting in Python 3.3, any modules added to the standard library must
have a pure Python implementation. This rule can only be ig

Re: [Python-Dev] Supporting Visual Studio 2010

2011-04-04 Thread Michael Foord 

On 05/04/2011 00:12, Scott Dial wrote:

On 4/4/2011 6:43 PM, Michael Foord wrote:

Won't that still be an issue despite the stable ABI? Extensions on
Windows should be linked to the same version of MSVCRT used to compile
Python - and every time we switch version of Visual Studio it is usually
accompanied by a switch in MSVCRT version.

My understanding (but I haven't looked closely) was that the stable ABI
specifically excluded anything that would expose a problem due to a CRT
mismatch -- making this a moot point. I'm sure Martin will correct me if
I am wrong.


Ah, wouldn't surprise me at all to know he'd already thought of that. :-)

Michael


-Scott




--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Policy for making changes to the AST

2011-04-04 Thread Eugene Toder 
> Ok, so it sounds like ast is *not* limited to CPython? That makes it
> harder to justify changing it just so as to ease the compilation
> process in CPython (as opposed to add new language features).

The changes above are not just for CPython, but to simplify processing
of AST in general, by reducing redundancy and separating syntax from
semantics. It just happens that the current structure of AST doesn't
allow important cases of constant folding at all, so I had to make
*some* changes. However, if the goal is to preserve the current AST as
much as possible, I can instead make a very simple completely backward
compatible change -- add one new node type that will never be present
in unoptimized AST. This is much less elegant and will add more cruft
to cpython's code (rather than removing it like the current patch
does), but it will work.

Eugene
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Policy for versions of system python

2011-04-04 Thread Eugene Toder 
> To answer Eugene's question, there's no official policy but the
> comment at the top of Python/makeopcodetargets.py can indeed serve as
> an useful guideline. I wonder if we still have buildbots with 2.3 as
> the system Python, by the way.

Ok, I'll use 2.3 as my target. Thanks.

Eugene
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com