[issue38091] Import deadlock detection causes deadlock

New submission from Ronan Lamy : There seems to be a race condition in importlib._bootstrap._ModuleLock that can cause a deadlock. The sequence of operations is as follows: * Thread 1 calls lock.acquire() * Thread 1 sets itself as lock.owner and begins importing the module * Thread 2 calls

[issue38095] Multi-threaded circular import fails with _DeadlockError when using importlib

New submission from Ronan Lamy : The IMPORT_NAME bytecode has a fast path calling PyImport_ImportModuleLevelObject() that behaves differently from importlib. In particular, test_circular_imports() in test_threaded_import.py fails due to a _DeadlockError if you replace some import statements

[issue38109] Missing constants in Lib/stat.py

New submission from Ronan Lamy : According to the docs, stat should include constants S_IFDOOR, S_IFPORT, S_IFWHT as well as the related S_IS... functions. However, these are only defined in _stat. I know that stat is a bit special (see bpo-11016), but that goes against PEP 399

[issue25130] Make tests more PyPy compatible

Change by Ronan Lamy : -- nosy: +Ronan.Lamy ___ Python tracker <https://bugs.python.org/issue25130> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue38109] Missing constants in Lib/stat.py

Change by Ronan Lamy : -- keywords: +patch pull_requests: +16248 stage: -> patch review pull_request: https://github.com/python/cpython/pull/16665 ___ Python tracker <https://bugs.python.org/issu

[issue38109] Missing constants in Lib/stat.py

Ronan Lamy added the comment: Well, my interest in this is to reduce the divergence between PyPy and CPython, and, potentially, to help other implementations. So I actually care more about 3.7 than about 3.9, which I probably won't look at before 2021. That said, I don't *need

[issue38109] Missing constants in Lib/stat.py

Ronan Lamy added the comment: Thanks Victor! -- ___ Python tracker <https://bugs.python.org/issue38109> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue37645] Replace PyEval_GetFuncName/PyEval_GetFuncDesc

Change by Ronan Lamy : -- nosy: +Ronan.Lamy ___ Python tracker <https://bugs.python.org/issue37645> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue26917] unicodedata.normalize(): bug in Hangul Composition

Ronan Lamy added the comment: Victor's patch is correct. I implemented the same fix in PyPy in https://bitbucket.org/pypy/pypy/commits/92b4fb5b9e58 -- nosy: +Ronan.Lamy ___ Python tracker <https://bugs.python.org/is

[issue31984] startswith and endswith leak implementation details

New submission from Ronan Lamy : One would think that u.startswith(v, start, end) would be equivalent to u[start: end].startswith(v), but one would be wrong. And the same goes for endswith(). Here is the actual spec (for bytes, but str and bytearray are the same), in the form of passing

[issue31984] startswith and endswith leak implementation details

Ronan Lamy added the comment: The problem is the complexity of the actual behaviour of these methods. It is impossible to get it right without looking at the source (at least, it was for me), and I doubt any ordinary user can correctly make use of the v='' behaviour, or predic

[issue31984] startswith and endswith leak implementation details

Ronan Lamy added the comment: Ah, thanks, I noticed the discrepancy between unicode and str in 2.7, but wondered when it was fixed. I guess I'm arguing that it was resolved in the wrong direction, then. Now, your first expression is wrong, even after fixing the obvious typo. The co

[issue14938] 'import my_pkg.__init__' creates duplicate modules

New submission from Ronan Lamy : If an __init__.py file contains relative imports, doing 'import my_pkg.__init__' or calling __import__('my_pkg.__init__') creates duplicate versions of the relatively imported modules, which (I believe) causes cryptic errors in some case

[issue14938] 'import my_pkg.__init__' creates duplicate modules

Ronan Lamy added the comment: my_pkg.__init__ isn't treated as just another module but as a package. That is the reason why the bogus my_pkg.__init__.module1 is created, I guess: >>> import sys >>> sorted(name for name in sys.modules if name.startswith('my_'

[issue14938] 'import my_pkg.__init__' creates duplicate modules

Ronan Lamy added the comment: Grmf. I didn't mean to change the status. -- resolution: -> wont fix status: open -> closed ___ Python tracker <http://bugs.python.

[issue14938] 'import my_pkg.__init__' creates duplicate modules

Ronan Lamy added the comment: Well, I can try. I have trouble wrapping my head around all the finders and loaders, but I'm slowly understanding what does what. What I don't know is what the expected behaviour is. Should my_pkg.__init__ be allowed to exist as a separate modu

[issue14938] 'import my_pkg.__init__' creates duplicate modules

Ronan Lamy added the comment: Reverting to the previous behaviour, then? OK. As I understand it, the issue comes from a DRY violation: both FileFinder.find_loader() and _LoaderBasics.is_package() have their own notion of what is a package and they disagree. Since the finder needs to know

[issue14938] 'import my_pkg.__init__' creates duplicate modules

Ronan Lamy added the comment: That would force the Loaders to know how to convert a module name into a file path, which isn't the case now since FileLoader.get_filename() is just a shim that returns self.path. So I'd rather add an optional argument to FileLoader. Actually, I fee

[issue14938] 'import my_pkg.__init__' creates duplicate modules

Ronan Lamy added the comment: Here's a preliminary patch, without tests or documentation, implementing my approach (adding an optional is_package=False to FileLoader.__init__()). Next question (quite independent of the chosen implementation): how should this be tested? -- key

[issue14938] 'import my_pkg.__init__' creates duplicate modules

Ronan Lamy added the comment: Your second solution is simple and solves the problem, so I guess that's it, then. Still, I don't really understand the purpose of SourceLoader, which makes me uneasy about the implementation of _LoaderBasics.is_package(). It seems to be meant

[issue14938] 'import my_pkg.__init__' creates duplicate modules

Ronan Lamy added the comment: OK, that makes sense pragmatically. Here's the patch then. I wasn't sure where to put the test, it doesn't actually have much in common with the rest of Lib/importlib/test/import_/test_packages.py but the name fits, so... -- A

[issue14982] pkgutil.walk_packages seems to not work properly on Python 3.3a

Changes by Ronan Lamy : -- nosy: +Ronan.Lamy ___ Python tracker <http://bugs.python.org/issue14982> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue14938] 'import my_pkg.__init__' creates duplicate modules

Ronan Lamy added the comment: I'm not sure that it's enough to test is_package() because that only involves the loader and not the interaction between it and FileFinder. That's the reason why my test works at a higher level. BTW, I sent the contri

[issue15030] PyPycLoader can't read cached .pyc files

New submission from Ronan Lamy : PyPycLoader can't read or write .pyc files created by the core import machinery. I'm attaching a failing test case demonstrating the issue: if you import a .py file using standard mechanisms, thus creating a .pyc, and then (in a separate process, sa

[issue15031] Split .pyc parsing from module loading

New submission from Ronan Lamy : I think it would be beneficial to extract the handling of the binary format of bytecode files from the rest of the loader/finder code in importlib._bootstrap. That means exposing, internally at least, functions that do nothing more than convert the binary

[issue15030] PyPycLoader can't read cached .pyc files

Ronan Lamy added the comment: 2.7 doesn't have PyPycLoader. For 3.2, it's such a pain to create a working concrete subclass of PyPycLoader that I gave up. But just by reading the code, it doesn't seem affected, as there's no discrepancy with importlib._bootstrap: the

[issue15030] PyPycLoader can't read cached .pyc files

Ronan Lamy added the comment: My preferred solution would be to use to use the functions I describe in issue 15031. -- ___ Python tracker <http://bugs.python.org/issue15

[issue15031] Split .pyc parsing from module loading

Ronan Lamy added the comment: I see. However, I think that breaking code noisily is better than breaking it silently, which is what happens when the same protocol is reimplemented many times. And _loads_pyc() could be made more forward-compatible by returning (path_stats, code) instead of

[issue15030] PyPycLoader can't read cached .pyc files

Ronan Lamy added the comment: Well, working on a deprecated ABC is certainly low-priority, but it seems rather bad to have 2 incompatible implementations of the .pyc format in the stdlib. If this is to stay like this, it should at least come with a strong warning that users of PyPycLoader

[issue12982] Document that importing .pyo files needs python -O

Changes by Ronan Lamy : -- nosy: +Ronan.Lamy ___ Python tracker <http://bugs.python.org/issue12982> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue12982] Document that importing .pyo files needs python -O

Ronan Lamy added the comment: Doing it at the interpreter level is trivial (cf. patch), except for an annoying bug I noticed (see below). Doing it from user code might require some care to avoid disrupting existing import hooks, but AFAICT something like sys.path_hooks.append

[issue15297] pkgutil.iter_importers() includes an ImpImporter

Ronan Lamy added the comment: AFAICT, the intent of this function was to help provide a fully PEP-302 compliant import process wrapping the builtin C-implemented import mechanism. Nowadays, I believe that iterating over sys.meta_path should probably be enough. -- nosy: +Ronan.Lamy

[issue15288] Clarify the pkgutil.walk_packages() note

Ronan Lamy added the comment: It seems that most, if not all, uses of "importer" in pkgutil refer to finders, e.g. ImpImporter is a actually only a finder, not an importer. So s/importer/finder/ is needed, and perhaps also a note explaining that ImpImporter isn't in fact

[issue15482] __import__() change between 3.2 and 3.3

New submission from Ronan Lamy: I noticed a change in the behaviour of __import__() between 3.2 and 3.3. It looks like in 3.2 __import__() does a Py2-style combo relative/absolute import. Here's a minimal test case: $ ls foo bar.py __init__.py __pycache__ $ cat foo/__init__.py __imp