Re: [Python-Dev] A bit about the GIL
Hello, On Mon, 1 Apr 2013 01:14:11 +0200 Alfredo Solano Martínez wrote: > > The logic would then be something like this: > - when increasing the refcount, a thread writes only to its own subcounter, > creating one first if necessary. > - similarly, when decreasing the refcount, there is no need to access other > subcounters until that subcounter reaches zero. > - when a subcounter gets to zero, delete it, and read the other subcounters > to check if it was the last one. But then you will decrement another subcounter, right? Meaning you need a lock around all increments / decrements. > Unfortunately, in a crude test of mine there is already a severe performance > degradation, and that is without rwlocks. Yes, there will be. Given how often INCREF and DECREF are called, any complication of their implementation will significantly decrease single-threaded performance. See also http://mail.python.org/pipermail/python-ideas/2009-November/006599.html Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] What code in Python creates .exe launchers for 'entry_points' in 'setup.py'?
Hello. On Windows, 'setuptools' and 'distribute' package systems allows to add 'entry_points' definition into 'setup.py' python distribution script. For each entry in this definition, some kind of bootstrapper '.exe' file will be created and placed into 'Scripts' python dir. For example, if i install 'pip' via 'setup.py', a file named 'pip.exe' will be created and placed into 'C:\Python27\Scripts'. But what python code is responsible for creation of this bootstrapper executables? I have searched python 2.7.3 source code for some time, but can't find a place. Of course i can spend lots of time to debug 'setup.py' installation (really lots of source code), but maybe someone just knows the place and can pinpoint it? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] What code in Python creates .exe launchers for 'entry_points' in 'setup.py'?
Hello, Le 01/04/2013 16:51, Grigory Petrov a écrit : > But what python code is responsible for creation of this bootstrapper > executables? I have searched python 2.7.3 source code for some time, but > can't find a place. Setuptools is not in the standard library, so you would need to search the setuptools or distribute codebases. Regards ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] What code in Python creates .exe launchers for 'entry_points' in 'setup.py'?
The new "distlib" also provides that feature On Apr 1, 2013 4:58 PM, "Éric Araujo" wrote: > Hello, > > Le 01/04/2013 16:51, Grigory Petrov a écrit : > > But what python code is responsible for creation of this bootstrapper > > executables? I have searched python 2.7.3 source code for some time, but > > can't find a place. > > Setuptools is not in the standard library, so you would need to search > the setuptools or distribute codebases. > > Regards > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/dholth%40gmail.com > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] relative import circular problem
I just ran into the issue described in http://stackoverflow.com/questions/6351805/cyclic-module-dependencies-and-relative-imports-in-python. This is unfortunate, because we have been trying to move towards relative imports in order to aid flexibility in package and library design. The relative import syntax (from foo import bar) is a getattr type of lookup (i.e. import foo, then get attr from it). This is in contrast with absolute import import foo.bar (get the module foo.bar from sys.modules or import it) bar = foo.bar the latter works with partially initialized modules, but not the former, rendering two sibling modules unable to import each other using the relative syntax. as far as I know, relative imports are only supported using the former (import from) syntax. Are there any plans to alleviate this by allowing proper relative imports? After all, relative imports and packages go hand in hand. K ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] relative import circular problem
On Apr 01, 2013, at 08:20 PM, Kristján Valur Jónsson wrote: >The relative import syntax > > (from foo import bar) is a getattr type of lookup (i.e. import foo, then get > attr from it). > >This is in contrast with absolute import > > import foo.bar (get the module foo.bar from sys.modules or import it) > > bar = foo.bar I always thought of both syntaxes as absolute imports because they both name the full dotted path to the module you want. This contrasts with true PEP 328 relative imports such as: from ..foo import bar >the latter works with partially initialized modules, but not the former, >rendering two sibling modules unable to import each other using the relative >syntax. > >as far as I know, relative imports are only supported using the former >(import from) syntax. Are there any plans to alleviate this by allowing >proper relative imports? After all, relative imports and packages go hand in >hand. I personally dislike PEP 328 relative imports, since they seem fragile, but that's a different discussion. -Barry ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] relative import circular problem
On Mon, Apr 1, 2013 at 4:20 PM, Kristján Valur Jónsson < [email protected]> wrote: > I just ran into the issue described in > http://stackoverflow.com/questions/6351805/cyclic-module-dependencies-and-relative-imports-in-python > . > > This is unfortunate, because we have been trying to move towards relative > imports in order to aid flexibility in package and library design. > > The relative import syntax > > (from foo import bar) is a getattr type of lookup (i.e. import foo, then > get attr from it). > > This is in contrast with absolute import > > import foo.bar (get the module foo.bar from sys.modules or import it) > > bar = foo.bar > Or ``import foo.bar as bar`` > > > the latter works with partially initialized modules, but not the former, > rendering two sibling modules unable to import each other using the > relative syntax. > Clarification on terminology: the ``from .. import`` syntax is in no way relative. Relative imports use leading dots to specify relative offsets from your current position (i.e. as Barry said). It's more of a syntax for facilitating binding long names (e.g. foo.bar) to shorter names (bar). It's just unfortunate that it can lead to circular import issues when people start pulling in objects off of modules instead of modules alone. > > > as far as I know, relative imports are only supported using the former > (import from) syntax. Are there any plans to alleviate this by allowing > proper relative imports? After all, relative imports and packages go hand > in hand. > No, there are no plans to either tweak ``from ... import`` statements nor introduce a new syntax to deal help alleviate circular imports. -Brett > > > K > > > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] relative import circular problem
On Tue, Apr 2, 2013 at 6:20 AM, Kristján Valur Jónsson < [email protected]> wrote: > I just ran into the issue described in > http://stackoverflow.com/questions/6351805/cyclic-module-dependencies-and-relative-imports-in-python > . > > This is unfortunate, because we have been trying to move towards relative > imports in order to aid flexibility in package and library design. > > The relative import syntax > > (from foo import bar) is a getattr type of lookup (i.e. import foo, then > get attr from it). > > This is in contrast with absolute import > > import foo.bar (get the module foo.bar from sys.modules or import it) > > bar = foo.bar > > > > the latter works with partially initialized modules, but not the former, > rendering two sibling modules unable to import each other using the > relative syntax. > This is really a quality-of-implementation issue in the import system rather than a core language design problem. It's just that those of us with the knowledge and ability to fix it aren't inclined to do so because circular imports usually (although not quite always) indicate a need to factor some common code out into a third support module imported by both of the original modules. At that point, the code is cleaner and more decoupled, and the uneven circular import support ceases to be a problem for that application. If you're interested in digging further, see http://bugs.python.org/issue992389 (this should also be a *lot* easier to fix now we're using importlib than it ever would have been while we were still relying on import.c) Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
On Mon, Apr 1, 2013 at 12:28 AM, Mark Dickinson wrote:
> As written, int_check would do the wrong thing for bools, too: I
> definitely want int(True) to be 1, not True.
>
> For (2) and (4), it's not so clear. Are there use-cases for an __index__
> return value that's not directly of type int? I can't think of any offhand.
>
int() and operator.index() are both type coercion calls to produce true
Python integers - they will never return a subclass, and this is both
deliberate and consistent with all the other builtin types that accept an
instance of themselves as input to the constructor. Passing a subclass
instance to the base class constructor is the way you convert a subclass to
an ordinary instance of the base class:
>>> for base in (str, bytes, bytearray, int, float, complex, dict, tuple,
list, set, frozenset):
... class subclass(base): pass
... print("'type(base(subclass()))' is", type(base(subclass(
...
'type(base(subclass()))' is
'type(base(subclass()))' is
'type(base(subclass()))' is
'type(base(subclass()))' is
'type(base(subclass()))' is
'type(base(subclass()))' is
'type(base(subclass()))' is
'type(base(subclass()))' is
'type(base(subclass()))' is
'type(base(subclass()))' is
'type(base(subclass()))' is
There's code in the slot wrappers so that if you return a non-int object
from either __int__ or __index__, then the interpreter will complain about
it, and if you return a subclass, it will be stripped back to just the base
class.
If the language and library reference aren't clear on this, it's a
documentation issue.
Cheers,
Nick.
--
Nick Coghlan | [email protected] | Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 4XX: pyzaa "Improving Python ZIP Application Support"
https://docs.google.com/document/d/1MKXgPzhWD5wIUpoSQX7dxmqgTZVO6l9iZZis8dnri78/edit?usp=sharing PEP: 4XX Title: Improving Python ZIP Application Support Author: Daniel Holth Status: Draft Type: Standards Track Python-Version: 3.4 Created: 30 March 2013 Post-History: 30 March 2013, 1 April 2013 Improving Python ZIP Application Support Python has had the ability to execute directories or ZIP-format archives as scripts since version 2.6. When invoked with a zip file or directory as its first argument the interpreter adds that directory to sys.path and executes the __main__ module. These archives provide a great way to publish software that needs to be distributed as a single file script but is complex enough to need to be written as a collection of modules. This feature is not as popular as it should be, mainly because no one’s heard of it because it wasn’t promoted as part of Python 2.6, but also because Windows users don’t have a file extension (other than .py) to associate with the launcher. This PEP proposes to fix these problems by re-publicising the feature, defining the .pyz and .pyzw extensions as “Python ZIP Applications” and “Windowed Python ZIP Applications”, and providing some simple tooling to manage the format. A New Python ZIP Application Extension The Python 3.4 installer will associate .pyz and .pyzw “Python ZIP Applications” with the platform launcher so they can be executed. A .pyz archive is a console application and a .pyzw archive is a windowed application, indicating whether the console should appear when running the app. Why not use .zip or .py? Users expect a .zip file would be opened with an archive tool, and users expect .py to be opened with a text editor. Both would be confusing for this use case. For UNIX users, .pyz applications should be prefixed with a #! line pointing to the correct Python interpreter and an optional explanation. #!/usr/bin/env python3 # This is a Python application stored in a ZIP archive. (binary contents of archive) As background, ZIP archives are defined with a footer containing relative offsets from the end of the file. They remain valid when concatenated to the end of any other file. This feature is completely standard and is how self-extracting ZIP archives and the bdist_wininst installer format work. Minimal Tooling: The pyzaa Module This PEP also proposes including a simple application for working with these archives: The Python Zip Application Archiver “pyzaa” (rhymes with “huzzah” or “pizza”). “pyzaa” can archive or extract these files, compile bytecode, and can write the __main__ module if it is not present. Usage python -m pyzaa (pack | compile) python -m pyzaa pack [-o path/name] [-m module.submodule:callable] [-c] [-w] [-p interpreter] directory: ZIP the contents of directory as directory.pyz or [-w] directory.pyzw. Adds the executable flag to the archive. -c compile .pyc files and add them to the archive -p interpreter include #!interpreter as the first line of the archive -o path/name archive is written to path/name.pyz[w] instead of dirname. The extension is added if not specified. -m module.submodule:callable __main__.py is written as “import module.submodule; module.submodule.callable()” pyzaa pack will warn if the directory contains C extensions or if it doesn’t contain __main__.py. python -m pyzaa compile arcname.pyz[w] The Python files in arcname.pyz[w] are compiled and appended to the ZIP file. A standard ZIP utility or Python’s zipfile module can unpack the archives. FAQ Q. Isn’t pyzaa just a very thin wrapper over zipfile and compileall? A. Yes. Q. How does this compete with existing sdist/bdist formats? A. There is some overlap, but .pyz files are especially interesting as a way to distribute an installer. They may also prove useful as a way to deliver applications when users shouldn’t be asked to perform virtualenv + “pip install”. References [1] http://bugs.python.org/issue1739468 “Allow interpreter to execute a zip file” [2] http://bugs.python.org/issue17359 “Feature is not documented” Copyright This document has been placed into the public domain. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 4XX: pyzaa "Improving Python ZIP Application Support"
On 4/1/2013 5:47 PM, Daniel Holth wrote: users expect .py to be opened with a text editor. This user expects .py to be executed as an executable script, and thinks that is the default after an installation of Python on Windows. Windows has a separate option, Edit, to use to edit things. But, I'm glad to see you write the PEP. I have an even thinner method of doing this, using .py extensions, that I've been using for several years now with Python 3, and wondered why it wasn't more popular. My equivalent of pyzaa, though, only performs the pack operation, and requires a bit of cooperation from the application (as a convenient way of storing the application-specific parameters, I build the invocation of pyzaa-equivalent into the application itself using a non-documented command-line option, and build to a different directory, to avoid overwriting application.py). Feel free to incorporate all or parts of that idea if it makes sense to you and sounds convenient. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
