[issue2135] Restructure import.c into PEP 302 importer objects
New submission from Douglas Greiman: This patch reorganizes import.c to move functionality into two new PEP 302-style Importer objects. I attempted to change as little as feasible, but the patch is still ~4700 lines long, about 1000 of which is new tests. BuiltinImporter: handles C_BUILTIN and PY_FROZEN modules DirectoryImporter: handles PY_SOURCE, PY_COMPILED, PKG_DIRECTORY, C_EXTENSION modules BuiltinImporter is put on sys.meta_path, DirectoryImporter is put on sys.path_hooks. To preserve backward compatibility of methods like imp.find_module(), they use new variables sys.builtin_meta_path and sys.builtin_path_hooks which are analogous to sys.meta_path and sys.path_hook but contain only the two importer objects above. Character encoding issues were substantial. The existing code was somewhat inscrutable in this regard. The tests disabled in issue 1377 were re-added with more safeguards and harder tests. It is possible to import modules with non-ascii names from non-ascii paths. Tested on Windows XP and Linux. Areas for discussion: Naming: Names of the importer classes, names of variables, etc sys: I added three variables to sys. Is there a better alternative? ModuleInfo: The importers use a somewhat tricky way to store an open file between calls to importer.find_module() and importer.load_module(), so that the new code doesn't do any more system calls that the old code. semantics: There are many import corner cases where the semantics are unknown or unclear. Frozen packages: the __path__ of a frozen package is a string rather than a list, and it is possible to replace that string to change how imports inside that package are resolved. I have attempted to keep that functionality intact but it's not clear that it's a feature rather than a bug. -- components: Interpreter Core files: builtin_importer-20080217.diff messages: 62510 nosy: dgreiman severity: normal status: open title: Restructure import.c into PEP 302 importer objects type: behavior versions: Python 3.0 Added file: http://bugs.python.org/file9453/builtin_importer-20080217.diff __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2135> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2135] Restructure import.c into PEP 302 importer objects
Douglas Greiman added the comment: Brett, I wrote my patch thinking that the next step would be to rewrite DirectoryImporter in Python. If you're already working on that and just want to skip straight from point A to point C and skip this point B, I'm fine with that. Basically, tell me if you want me to pursue this further or drop it. Are you also reimplementing the code for builtin and frozen modules in Python? Alexander, The const thing: I went on a bit of a const rampage after an especially irritating debugging session involving function side effects. They could be removed. The non-subclassable aspect was an oversight. I thought about having a common base class but ended up creating the ModuleInfo helper class instead. There's still a few bits of code duplication between the two importer classes. 16x2 lines in *_find_module() could be pulled into a common place, 4x2 lines of *_require_module(), and a few lines in the various methods. I wanted to err on the side of introducing as few new classes as possible to minimize the general scariness of this patch. I don't have a Mac setup, but I think the error you're seeing is actually an error in my test code. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2135> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2297] Patch for fatal stack overflow in Windows caused by -v
New submission from Douglas Greiman <[EMAIL PROTECTED]>: When python is invoked with -v or -vv under Windows, the process of importing the codec for sys.stderr causes a message to be written to stderr, which in turn causes the codec to be recursively imported. Sometimes the stack overflow exception is swallowed, other times it is not. The bug depends on the particular locale settings of the Windows machine. To reproduce: python_d.exe -v and look for many repeated imports of encodings. Patch is attached. -- components: Interpreter Core files: py3k-win-codec-recursion-20080315.diff keywords: patch messages: 63570 nosy: dgreiman severity: normal status: open title: Patch for fatal stack overflow in Windows caused by -v type: crash versions: Python 3.0 Added file: http://bugs.python.org/file9678/py3k-win-codec-recursion-20080315.diff __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2297> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2298] Patch for "string without null bytes" check in getargs.c
New submission from Douglas Greiman <[EMAIL PROTECTED]>:
Code in getargs.c:convertsimple incorrectly uses PyUnicode_GetSize
instead of PyString_GET_SIZE when checking whether a bytes object
(encoded string) contains a null byte.
To reproduce: __import__('\u0080')
Incorrect behavior:
TypeError: __import__() argument 1 must be string without null bytes,
not str
Correct behavior:
ImportError
Note the lack of null bytes:
>>> '\u0080'.encode('utf-8')
b'\xc2\x80'
--
components: Interpreter Core
files: py3k-getargs-null-bytes-20080315.diff
keywords: patch
messages: 63571
nosy: dgreiman
severity: normal
status: open
title: Patch for "string without null bytes" check in getargs.c
type: behavior
versions: Python 3.0
Added file:
http://bugs.python.org/file9679/py3k-getargs-null-bytes-20080315.diff
__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue2298>
__
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2299] Minor typos in newtypes.rst
New submission from Douglas Greiman <[EMAIL PROTECTED]>: Fix three minor typos in Doc/extending/newtypes.rst -- assignee: georg.brandl components: Documentation files: Doc-newtypes-typos-20080315.diff keywords: patch messages: 63572 nosy: dgreiman, georg.brandl severity: minor status: open title: Minor typos in newtypes.rst type: behavior versions: Python 2.6 Added file: http://bugs.python.org/file9680/Doc-newtypes-typos-20080315.diff __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2299> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2297] Patch for fatal stack overflow in Windows caused by -v
Douglas Greiman <[EMAIL PROTECTED]> added the comment: Good call. I've attached an updated patch which includes a testcase in test_cmd_line.py. Added file: http://bugs.python.org/file9685/py3k-win-codec-recursion-20080316.diff __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2297> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28643] Broken makefile depends for profile-opt target
Changes by Douglas Greiman : -- nosy: +dgreiman ___ Python tracker <http://bugs.python.org/issue28643> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28425] Python3 ignores __init__.py that are links to /dev/null
Douglas Greiman added the comment: Is there a reasonable place to document that __init__.py (and probably source files in general) must be a "regular file" by the Unix definition, and not a device file, socket, etc? -- ___ Python tracker <http://bugs.python.org/issue28425> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28425] Python3 ignores __init__.py that are links to /dev/null
Douglas Greiman added the comment: To be thorough, I looked at non __init__.py source files as well, with erratic results. And who knows what would happen on other OS's. So a blanket "must be a regular file" seems reasonable. $ ls -l b.* lrwxrwxrwx 1 dgreiman eng 9 Dec 9 16:05 b.py -> /dev/null $ python2.7 -m b # Success $ python2.7 -c 'import b'# Success $ python2.7 b.py # Success $ python3.5 b.py # Success $ python3.5 -m b # Failure /opt/python3.5/bin/python3.5: No module named b $ python3.5 -c 'import b'# Failure Traceback (most recent call last): File "", line 1, in ImportError: No module named 'b' $ uname -a Linux 53334eb940e6 3.13.0-101-generic #148-Ubuntu SMP Thu Oct 20 22:08:32 UTC 2016 x86_64 GNU/Linux -- ___ Python tracker <http://bugs.python.org/issue28425> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28424] pkgutil.get_data() doesn't work with namespace packages
Douglas Greiman added the comment: Patch attached. Feel free to wordsmith. -- keywords: +patch Added file: http://bugs.python.org/file45825/issue28424.patch ___ Python tracker <http://bugs.python.org/issue28424> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29417] Sort entries in foo.dist-info/RECORD
New submission from Douglas Greiman: Feature Request: It would make reproducible builds easier if the [package].dist-info/RECORD files were sorted in some deterministic way. For example, building a Docker image twice and comparing the results. The current ordering seems to be arbitrary, and I didn't find any PEPs that specify one way or the other. I'm not sure what would need to be updated for this change. The whole pip/distutils/setuptools ecosystem is a bit murky. -- components: Distutils messages: 286729 nosy: dgreiman, dstufft, eric.araujo priority: normal severity: normal status: open title: Sort entries in foo.dist-info/RECORD type: enhancement ___ Python tracker <http://bugs.python.org/issue29417> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28425] Python3 ignores __init__.py that are links to /dev/null
Douglas Greiman added the comment: Bazel has been updated to no longer create symlinks to /dev/null https://github.com/bazelbuild/bazel/issues/1458 -- ___ Python tracker <http://bugs.python.org/issue28425> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28424] pkgutil.get_data() doesn't work with namespace packages
New submission from Douglas Greiman:
pkg_util.get_data('mypackage', 'resourcename')
returns None if 'mypackage' is a namespace package, because the package object
has no __file__ attribute. A shell script is attached to reproduce this.
This is either a bug to be fixed (my preference) or behavior to be documented.
If there is equivalent, working functionality in the new importlib, that would
also be a good to document.
--
components: Library (Lib)
files: namespace_package_get_data.sh
messages: 278543
nosy: dgreiman
priority: normal
severity: normal
status: open
title: pkgutil.get_data() doesn't work with namespace packages
type: behavior
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file45068/namespace_package_get_data.sh
___
Python tracker
<http://bugs.python.org/issue28424>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28425] Python3 ignores __init__.py that are links to /dev/null
New submission from Douglas Greiman: This manifests in the following way: A package directory containing an __init__.py that is a symlink to /dev/null is treated as a namespace package instead of a regular package. The Bazel build tool creates many __init__.py files in this way, which is how I even ran into this. Symlinks to regular files seem fine. -- components: Interpreter Core files: namespace_package_dev_null.sh messages: 278544 nosy: dgreiman priority: normal severity: normal status: open title: Python3 ignores __init__.py that are links to /dev/null type: behavior versions: Python 3.4, Python 3.5 Added file: http://bugs.python.org/file45069/namespace_package_dev_null.sh ___ Python tracker <http://bugs.python.org/issue28425> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28425] Python3 ignores __init__.py that are links to /dev/null
Douglas Greiman added the comment: See associated bug filed against Bazel: https://github.com/bazelbuild/bazel/issues/1458 As for why Bazel does that, it's related to the sandboxing implementation but I don't know any details beyond that. -- ___ Python tracker <http://bugs.python.org/issue28425> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1520879] make install change: Allow $DESTDIR to be relative
Changes by Douglas Greiman : -- status: open -> closed ___ Python tracker <http://bugs.python.org/issue1520879> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1520879] make install change: Allow $DESTDIR to be relative
Douglas Greiman added the comment: Duplicate of http://bugs.python.org/issue11411 -- ___ Python tracker <http://bugs.python.org/issue1520879> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
