[Python-Dev] Latest draft of PEP 399 (Pure Python/C Accelerator Module Compatibility Requirements)
While at a mini-PyPy sprint w/ Alex Gaynor of PyPy and Phil Jenvey of Jython, I decided to finally put the time in to update this PEP yet again. The biggest changes is that the 100% branch coverage requirement has been replaced with "comprehensive" coverage from the tests. I think we are all enough grown-ups to not have to specify anything tighter than this. I also added a paragraph in the Details section about using the abstract C APIs (e.g., PyObject_GetItem) over type-specific ones (e.g., PyList_GetItem) in order to be more supportive of duck typing from the Python code. I figure the "be API compatible" assumes this, but mentioning it doesn't hurt (and should help make Raymond less angry =). PEP: 399 Title: Pure Python/C Accelerator Module Compatibility Requirements Version: $Revision: 88219 $ Last-Modified: $Date: 2011-01-27 13:47:00 -0800 (Thu, 27 Jan 2011) $ Author: Brett Cannon Status: Draft Type: Informational Content-Type: text/x-rst Created: 04-Apr-2011 Python-Version: 3.3 Post-History: 04-Apr-2011, 12-Apr-2011, 17-Jul-2011 Abstract The Python standard library under CPython contains various instances of modules implemented in both pure Python and C (either entirely or partially). This PEP requires that in these instances that the C code *must* pass the test suite used for the pure Python code so as to act as much as a drop-in replacement as reasonably possible (C- and VM-specific tests are exempt). It is also required that new C-based modules lacking a pure Python equivalent implementation get special permission to be added to the standard library. Rationale = Python has grown beyond the CPython virtual machine (VM). IronPython_, Jython_, and PyPy_ are all currently viable alternatives to the CPython VM. The VM ecosystem that has sprung up around the Python programming language has led to Python being used in many different areas where CPython cannot be used, e.g., Jython allowing Python to be used in Java applications. A problem all of the VMs other than CPython face is handling modules from the standard library that are implemented (to some extent) in C. Since other VMs do not typically support the entire `C API of CPython`_ they are unable to use the code used to create the module. Often times this leads these other VMs to either re-implement the modules in pure Python or in the programming language used to implement the VM itself (e.g., in C# for IronPython). This duplication of effort between CPython, PyPy, Jython, and IronPython is extremely unfortunate as implementing a module *at least* in pure Python would help mitigate this duplicate effort. The purpose of this PEP is to minimize this duplicate effort by mandating that all new modules added to Python's standard library *must* have a pure Python implementation _unless_ special dispensation is given. This makes sure that a module in the stdlib is available to all VMs and not just to CPython (pre-existing modules that do not meet this requirement are exempt, although there is nothing preventing someone from adding in a pure Python implementation retroactively). Re-implementing parts (or all) of a module in C (in the case of CPython) is still allowed for performance reasons, but any such accelerated code must pass the same test suite (sans VM- or C-specific tests) to verify semantics and prevent divergence. To accomplish this, the test suite for the module must have comprehensive coverage of the pure Python implementation before the acceleration code may be added. Details === Starting in Python 3.3, any modules added to the standard library must have a pure Python implementation. This rule can only be ignored if the Python development team grants a special exemption for the module. Typically the exemption will be granted only when a module wraps a specific C-based library (e.g., sqlite3_). In granting an exemption it will be recognized that the module will be considered exclusive to CPython and not part of Python's standard library that other VMs are expected to support. Usage of ``ctypes`` to provide an API for a C library will continue to be frowned upon as ``ctypes`` lacks compiler guarantees that C code typically relies upon to prevent certain errors from occurring (e.g., API changes). Even though a pure Python implementation is mandated by this PEP, it does not preclude the use of a companion acceleration module. If an acceleration module is provided it is to be named the same as the module it is accelerating with an underscore attached as a prefix, e.g., ``_warnings`` for ``warnings``. The common pattern to access the accelerated code from the pure Python implementation is to import it with an ``import *``, e.g., ``from _warnings import *``. This is typically done at the end of the module to allow it to overwrite specific Python objects with their accelerated equivalents. This kind of import can also be done before the end of the module when needed, e.g., an accelerated base class is provided but i
Re: [Python-Dev] Issue10271 - warnings.showwarning should allow any callable object - request commiter
Just so people know, I went ahead and fixed this for 3.3 (but not for 3.2 since it changes the API in a subtle way). On Sat, Jul 16, 2011 at 01:33, lekmalek wrote: > Hello all, > > Can any of you core devs have a look at > http://bugs.python.org/issue10271. It seems Brett is really busy right > now and this uncontroversial (AFAICT) one liner only needs someone to > review it and commit it. The pb is, it's holding me back a little bit, > and I really would like to have it in the next 3.2 release if possible. > > Thanks for your help, > > lekma > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] [PEPs] Rebooting PEP 394 (aka Support the /usr/bin/python2 symlink upstream)
Hi, These are two emails I sent a short while ago about finalizing PEP 394. There was no response, so in case the messages didn't go through, I'm resending them. Thanks, Kerrick Staley -- Forwarded message -- From: Kerrick Staley Date: Sat, Jul 9, 2011 at 7:45 PM Subject: Re: [Python-Dev] [PEPs] Support the /usr/bin/python2 symlink upstream To: [email protected] Sorry that I dropped the ball on this. I'd still like to see this get implemented, but I got distracted with school and forgot about it. Updates I have made to the PEP will be sent as a patch immediately after this email. Here's a summary of what was happenening when we left off: * The draft SVN version from March 4 was pretty much complete. * Some were concerned about addressing Windows, but it was generally agreed to leave the Windows issue to another PEP. PEP 397 was started on March 15 to address the Windows side of the issue. PEP 397 recommends that the Windows Python launcher read the shebang and use it to determine which Python version to use; this allows one syntax for both operating systems that is compatible with the current PEP 394 recommendation. * There were concerns from Ned Deily about the naming of other binaries such as idle, pydoc, and python-config; these need to be created as idle2, pydoc2, and python2-config, with links created at the locations of the original binaries. * There were concerns from Glenn Linderman that the shebang line doesn't encode enough information to flexibly handle Windows launching (or even launching in general). Here are my thoughts: * For Ned's comments, I agree. Although the issue isn't as large with these programs, there's no reason we can't handle them in the same way. I updated the PEP. * For Glenn's comments, I think the method you propose adds too much complexity. Regardless, if the #@ syntax is implemented, it can be described in PEP 397; it won't have an impact on the contents of this PEP. I think, though, that having multiple syntaxes may cause many scripts to be incompatible with multiple platforms when they don't have to be, since Unix coders will rarely add a #@ line, and Windows coders will likely forget the #! line. Also, #@ really ignores the purpose of a shebang: shebangs simply indicate an interpreter that works with the script; the shebang allows users to run arbitrary scripts without worrying about which interpreter they should specify. There's no reason that a script should use one interpreter on Unix, but be incompatible with that interpreter on Windows yet compatible with another. The name of the Unix binary is enough to determine the implementation and version of the interpreter to be used on Unix, and a Windows launcher should always invoke the same implementation/version on Windows (and this won't require hard-coding anything into the launcher if done right). If you want the script to run with a specific interpreter and version, possibly contingent on which operating system you're running the script under, then you can just invoke the interpreter by name with the script as an argument (e.g. python3 myprogram.py). TL;DR: shebangs encode a default implementation/version, and if you need something special, you can just manually run python3 myprogram.py or use a .bat file. Also, I updated the PEP with the clarification that commands like python3 should be hard links (because they'll be invoked from code and are more efficient; also, hard links are just as flexible as symlinks here), while commands like python should be soft links (because this makes it clear to sysadmins that they can be "switched", and it's needed for flexibility if python3 changes). This really doesn't matter, but can we keep it this way unless there are serious objections? Regards, Kerrick Staley -- Forwarded message -- From: Kerrick Staley Date: Sat, Jul 9, 2011 at 7:46 PM Subject: Re: [Python-Dev] [PEPs] Support the /usr/bin/python2 symlink upstream To: [email protected] $ svn diff Index: pep-0394.txt === --- pep-0394.txt(revision 88860) +++ pep-0394.txt(working copy) @@ -1,5 +1,5 @@ PEP: 394 -Title: The "python" command on Unix-Like Systems +Title: The "python" Command on Unix-Like Systems Version: $Revision$ Last-Modified: $Date$ Author: Kerrick Staley , @@ -53,10 +53,14 @@ * When reinvoking the interpreter from a Python script, querying ``sys.executable`` to avoid hardcoded assumptions regarding the interpreter location remains the preferred approach. +* The ``idle``, ``pydoc``, and ``python-config`` binaries from Python 2.0 +should likewise be available as ``idle2``, ``pydoc2``, and ``python2-config``, +with the original commands invoking these binaries by default, but possibly +invoking the Python 3.0 versions instead. These recommendations are the outcome of the relevant python-dev discussion in -March 2011 [1] (NOTE: More accurately, they will
