[issue38263] [Windows] multiprocessing: DupHandle.detach() race condition on DuplicateHandle(DUPLICATE_CLOSE_SOURCE)
Pedro Algarvio added the comment: Any possible workaround for this issue? I seem to be consistingly hitting this issue. -- nosy: +s0undt3ch ___ Python tracker <https://bugs.python.org/issue38263> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38263] [Windows] multiprocessing: DupHandle.detach() race condition on DuplicateHandle(DUPLICATE_CLOSE_SOURCE)
Pedro Algarvio added the comment:
What I'm able to copy from the console(though I doubt I'm getting all of
the traceback):
Traceback (most recent call last):
File "", line 1, in
File
"c:\hostedtoolcache\windows\python\3.5.4\x64\lib\multiprocessing\spawn.py",
line 106, in spawn_main
exitcode = _main(fd)
File
"c:\hostedtoolcache\windows\python\3.5.4\x64\lib\multiprocessing\spawn.py",
line 116, in _main
self = pickle.load(from_parent)
File
"c:\hostedtoolcache\windows\python\3.5.4\x64\lib\multiprocessing\connection.py",
line 942, in rebuild_pipe_connection
handle = dh.detach()
File
"c:\hostedtoolcache\windows\python\3.5.4\x64\lib\multiprocessing\reduction.py",
line 128, in detach
self._access, False, _winapi.DUPLICATE_CLOSE_SOURCE)
PermissionError: [WinError 5] Access is denied
Pedro Algarvio @ Phone
A sábado, 22/02/2020, 13:44, Eryk Sun escreveu:
>
> Eryk Sun added the comment:
>
> > I seem to be consistingly hitting this issue.
>
> It will help with test development if you can provide a minimal example
> that reliably reproduces the problem.
>
> In msg353064 I see DuplicateHandle calls failing with ERROR_ACCESS_DENIED
> (5). Assuming the process handles have the required PROCESS_DUP_HANDLE
> access, it's most likely the case that the underlying NtDuplicateObject
> system call is failing with STATUS_PROCESS_IS_TERMINATING (0xC10A). For
> example:
>
> import ctypes
> ntdll = ctypes.WinDLL('ntdll')
> from subprocess import Popen, PIPE
> from _winapi import GetCurrentProcess, TerminateProcess
> from _winapi import DuplicateHandle, DUPLICATE_SAME_ACCESS
>
> p = Popen('cmd.exe', stdin=PIPE, stdout=PIPE, stderr=PIPE)
> TerminateProcess(p._handle, 0)
>
> Try to duplicate the process handle into the terminated process:
>
> >>> source = GetCurrentProcess()
> >>> target = handle = p._handle
> >>> try:
> ... DuplicateHandle(source, handle, target,
> ... 0, False, DUPLICATE_SAME_ACCESS)
> ... except:
> ... status = ntdll.RtlGetLastNtStatus()
> ... print(f'NTSTATUS: {status & (2**32-1):#010x}')
> ... raise
> ...
> NTSTATUS: 0xc10a
> Traceback (most recent call last):
> File "", line 2, in
> PermissionError: [WinError 5] Access is denied
>
> --
> nosy: +eryksun
>
> ___
> Python tracker
> <https://bugs.python.org/issue38263>
> ___
>
--
___
Python tracker
<https://bugs.python.org/issue38263>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42382] No easy way to get the distribution which provided a importlib.metadata.EntryPoint
Pedro Algarvio added the comment: Our software uses a plug-in based approach. Plugins are able to add/modify internal behavior, and, as part of bug submission process we have a CLI flag which provides information about the core app as well as any intervening plugins. This is where we need to "map" an entry point to it's distribution, so we know the name and version of it to display on this report. -- ___ Python tracker <https://bugs.python.org/issue42382> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42382] No easy way to get the distribution which provided a importlib.metadata.EntryPoint
New submission from Pedro Algarvio :
With `pkg_resources` an `EntryPoint` has a dist attribute which allows you to
get the distribution that provided that specific entry-point, however, with
`importlib.metafata` and `importlib_metadata` that's not an east task.
```python
USE_IMPORTLIB_METADATA_STDLIB = USE_IMPORTLIB_METADATA = False
try:
# Py3.8+
import importlib.metadata
USE_IMPORTLIB_METADATA_STDLIB = True
except ImportError:
# < Py3.8 backport package
import importlib_metadata
USE_IMPORTLIB_METADATA = True
def get_distribution_from_entry_point(entry_point):
loaded_entry_point = entry_point.load()
if isinstance(loaded_entry_point, types.ModuleType):
module_path = loaded_entry_point.__file__
else:
module_path = sys.modules[loaded_entry_point.__module__].__file__
if USE_IMPORTLIB_METADATA_STDLIB:
distributions = importlib.metadata.distributions
else:
distributions = importlib_metadata.distributions
for distribution in distributions():
try:
relative = pathlib.Path(module_path).relative_to(
distribution.locate_file("")
)
except ValueError:
pass
else:
if relative in distribution.files:
return distribution
```
The above solution has the additional drawback that you're iterating the
distributions list, once per EntryPoint, which, was already iterated to get the
entry-points listing.
I propose we attach the Distribution instance to each of the found EntryPoint
to avoid this low performance workaround.
I don't have an issue with providing a pull-request, but should that pull
request be done against `importlib_metadata` or `importlib.metadata`?
--
components: Library (Lib)
messages: 381211
nosy: jaraco, s0undt3ch
priority: normal
severity: normal
status: open
title: No easy way to get the distribution which provided a
importlib.metadata.EntryPoint
type: enhancement
___
Python tracker
<https://bugs.python.org/issue42382>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42382] No easy way to get the distribution which provided a importlib.metadata.EntryPoint
Change by Pedro Algarvio : -- keywords: +patch pull_requests: +2 stage: -> patch review pull_request: https://github.com/python/cpython/pull/23334 ___ Python tracker <https://bugs.python.org/issue42382> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42382] No easy way to get the distribution which provided a importlib.metadata.EntryPoint
Pedro Algarvio added the comment: Guess I jumped too fast :) Will the changes in CPythom be included in `importlib_metadata`? -- ___ Python tracker <https://bugs.python.org/issue42382> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7300] Unicode arguments in str.format()
Pedro Algarvio added the comment:
This is not a 2.7 issue only:
>>> import sys
>>> sys.version_info
(2, 6, 5, 'final', 0
>>> 'Foo {0}'.format(u'bár')
Traceback (most recent call last):
File "", line 1, in
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 1:
ordinal not in range(128)
>>>
--
nosy: +Pedro.Algarvio
___
Python tracker
<http://bugs.python.org/issue7300>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21423] concurrent.futures.ThreadPoolExecutor/ProcessPoolExecutor should accept an initializer argument
Changes by Pedro Algarvio : -- nosy: +Pedro.Algarvio ___ Python tracker <http://bugs.python.org/issue21423> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21423] concurrent.futures.ThreadPoolExecutor/ProcessPoolExecutor should accept an initializer argument
Pedro Algarvio added the comment: Would also love to see this in the stdlib soon. My use case is logging setup(forward logs to the main process). -- ___ Python tracker <http://bugs.python.org/issue21423> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
