[issue42938] ctypes double representation BoF

[Jordy Zomer]

New submission from Jordy Zomer :

Hi, 

There's a buffer overflow in the PyCArg_repr() function in _ctypes/callproc.c.

The buffer overflow happens due to not checking the length of th sprintf() 
function on line: 

case 'd':
sprintf(buffer, "",
self->tag, self->value.d);
break;

Because we control self->value.d we could make it copy _extreme_ values. For 
example we could make it copy 1e300 which would be a 1 with 300 zero's  to 
overflow the buffer.

This could potentially cause RCE when a user allows untrusted input in these 
functions.

A minimal PoC:

>>> from ctypes import *
>>> c_double.from_param(1e300)
*** buffer overflow detected ***: terminated
Aborted


I recommend __always__ controlling how much you copy so I'd use snprintf with a 
size argument instead.

Best Regards,

Jordy Zomer

--
components: ctypes
messages: 385136
nosy: JordyZomer
priority: normal
severity: normal
status: open
title: ctypes double representation BoF
type: security
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42885] Optimize re.search() for \A (and maybe ^)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

^ matches not just the beginning of the string. It matches the beginning of a 
line, i.e. an anchor just after '\n'. If the input string contains '\n', the 
result cannot be found less than by linear time. If you want to check if the 
beginning of the string matches a regular expression, it is better to use 
match(). If you want the check if the whole string matches it, it is better to 
use fullmatch().

But in some cases you cannot choose what method to use. If you have a set of 
patterns, and only some of them should be anchored to the start of the string, 
you have to use search(). And while linear complexity for ^ is expected, 
search() is not optimized for \A.

So the original report is rejected, the behavior is expected and cannot be 
changed. It is not a bug. But some optimization can be added for \A, and 
perhaps the constant multiplier for ^ can be reduced too.

--
title: Regex performance problem with ^ aka AT_BEGINNING -> Optimize 
re.search() for \A (and maybe ^)
versions:  -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7946] Convoy effect with I/O bound threads and New GIL

[Charles-François Natali]

Change by Charles-François Natali :


--
nosy:  -neologix

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42885] Optimize re.search() for \A (and maybe ^)

[Steven D'Aprano]

Steven D'Aprano  added the comment:

On Sat, Jan 16, 2021 at 08:59:13AM +, Serhiy Storchaka wrote:
> 
> Serhiy Storchaka  added the comment:
> 
> ^ matches not just the beginning of the string. It matches the 
> beginning of a line, i.e. an anchor just after '\n'.

Only in MULTILINE mode.

I completely agree that in multiline mode '^a' has to search the entire 
string. But in single line mode, if the first character isn't an 'a', 
there is no need to continue searching.

As far as I can see from the docs, ^ in single line mode and \A always 
should be constant time.

> So the original report is rejected, the behavior is expected and 
> cannot be changed. It is not a bug.

I disagree that it is expected behaviour. "Match the start of the 
string" cannot possibly match anything except the start of the string, 
so who would expect it to keep scanning past the start of the string?

(Multiline mode is, of course, different.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42916] Support for DICOM image file format in imghdr module

[Ross Rhodes]

Change by Ross Rhodes :


--
keywords: +patch
nosy: +trrhodes
nosy_count: 2.0 -> 3.0
pull_requests: +23050
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/24227

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42643] http.server does not support HTTP range requests

[David Bordeynik]

Change by David Bordeynik :


--
keywords: +patch
nosy: +DavidBord
nosy_count: 2.0 -> 3.0
pull_requests: +23051
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24228

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42916] Support for DICOM image file format in imghdr module

[Ross Rhodes]

Ross Rhodes  added the comment:

Hello Charles,

Following the format provided, I've opened a PR to implement your proposal. 
Feedback welcome.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42935] Pickle can't import builtins at exit

[Pete Wicken]

Pete Wicken  added the comment:

Out of curiosity, why is there not much we can do? I'm not familiar enough with 
Python's C code to appreciate the difficulty of making the builtins available 
at the point where the pickle save is run when exiting. The fact that this 
segfaults in the 3.8 version tested was rather concerning.

Shelve's implementation without the context manager was how we found the 
original bug; but it feels the way pickle handles this should be a lot safer.

Some clarification might help me craft a warning for the documentation if there 
is nothing that can be done in the code base.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42939] Linux's chattr i.e. ioctl(FS_IOC_SETFLAGS) is not supported in os.chflags()

[Марк Коренберг]

New submission from Марк Коренберг :

https://man7.org/linux/man-pages/man2/ioctl_iflags.2.html

Seems the ioctl is only way to implement chattr(). I propose to add all these 
flags to os.chflags() and to gain its support in Linux.

--
components: Library (Lib)
messages: 385141
nosy: socketpair
priority: normal
severity: normal
status: open
title: Linux's chattr i.e. ioctl(FS_IOC_SETFLAGS) is not supported in 
os.chflags()
versions: Python 3.10, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42885] Optimize re.search() for \A (and maybe ^)

[Arnim Rupp]

Arnim Rupp  added the comment:

some more observations, which might be helpful in tracking it down:

x$ is much faster than ^x
A$ is as slow as ^x

$ python3 -m timeit -s "a = 'A'*1" "import re" "re.search('x$', a)"
10 loops, best of 5: 32.9 msec per loop

$ python3 -m timeit -s "a = 'A'*1" "import re" "re.search('A$', a)"
1 loop, best of 5: 802 msec per loop

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42940] Incorrect behavior of inspect.signature(f).bind

[Slava Kostrov]

New submission from Slava Kostrov :

>>> def foo(a, /, b=1, **kwargs): pass
...
>>> foo(a=1)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: foo() missing 1 required positional argument: 'a'

>>> inspect.signature(foo).bind(a=1)
Traceback (most recent call last):
  File "", line 1, in 
  File "\Python\Python39\lib\inspect.py", line 3062, in bind
return self._bind(args, kwargs)
  File "\Python\Python39\lib\inspect.py", line 2958, in _bind
raise TypeError(msg) from None
TypeError: 'a' parameter is positional only, but was passed as a keyword

There is a different behaviour between a simple function call and 
inspect.signature(foo).bind - different TypeError messages.

I think inspect should check for VARKEYWORDS before throwing an error and throw 
only if there are not.

--
messages: 385143
nosy: slavkostrov
priority: normal
severity: normal
status: open
title: Incorrect behavior of inspect.signature(f).bind
type: behavior
versions: Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42941] Infinite loop in asyncio sslproto

[Hingyuen Lam]

New submission from Hingyuen Lam :

There is an infinite loop in feed_appdata() of sslproto.py.
The OpenSSL documents said when some fatal I/O error occurred, 
SSL_ERROR_SYSCALL is returned and no further I/O operations should be 
performed. This error is ignored in the exception handling of feed_appdata(), 
and the while loop will never be break.
This error should be raised, the same for feed_ssldata().

--
components: asyncio
messages: 385144
nosy: asvetlov, linxy95, yselivanov
priority: normal
severity: normal
status: open
title: Infinite loop in asyncio sslproto
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42531] importlib.resources.path() raises TypeError for packages without __file__

[miss-islington]

miss-islington  added the comment:


New changeset f08c66467d56c71f75c6659ede9177449fe9d2e6 by William Schwartz in 
branch '3.8':
[3.8] bpo-42531: Teach importlib.resources.path to handle packages without 
__file__ (GH-23611)
https://github.com/python/cpython/commit/f08c66467d56c71f75c6659ede9177449fe9d2e6


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42531] importlib.resources.path() raises TypeError for packages without __file__

[miss-islington]

Change by miss-islington :


--
pull_requests: +23052
pull_request: https://github.com/python/cpython/pull/24229

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42531] importlib.resources.path() raises TypeError for packages without __file__

[Jason R. Coombs]

Change by Jason R. Coombs :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42941] Infinite loop in asyncio sslproto

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
nosy: +python-dev
nosy_count: 3.0 -> 4.0
pull_requests: +23053
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24230

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42531] importlib.resources.path() raises TypeError for packages without __file__

[Jason R. Coombs]

Change by Jason R. Coombs :


--
versions:  -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33809] Expose `capture_locals` parameter in `traceback` convenience functions

[Irit Katriel]

Change by Irit Katriel :


--
keywords: +patch
pull_requests: +23054
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24231

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42163] _replace() no longer works on platform.uname_result objects

[Jason R. Coombs]

Change by Jason R. Coombs :


--
pull_requests: +23055
pull_request: https://github.com/python/cpython/pull/24232

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42659] Objects of uname_result Class Cannot be Successfully Pickled

[Jason R. Coombs]

Change by Jason R. Coombs :


--
pull_requests: +23057
pull_request: https://github.com/python/cpython/pull/24232

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42189] copy.deepcopy() no longer works on platform.uname_result objects

[Jason R. Coombs]

Change by Jason R. Coombs :


--
pull_requests: +23056
pull_request: https://github.com/python/cpython/pull/24232

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42659] Objects of uname_result Class Cannot be Successfully Pickled

[Jason R. Coombs]

Jason R. Coombs  added the comment:


New changeset 799722cb0ddb90752cde7798cab543f30623ebf2 by Jason R. Coombs in 
branch '3.9':
[3.9] bpo-42163, bpo-42189, bpo-42659: Support uname_tuple._replace (for all 
but processor) (GH-23010) (#24232)
https://github.com/python/cpython/commit/799722cb0ddb90752cde7798cab543f30623ebf2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42189] copy.deepcopy() no longer works on platform.uname_result objects

[Jason R. Coombs]

Jason R. Coombs  added the comment:


New changeset 799722cb0ddb90752cde7798cab543f30623ebf2 by Jason R. Coombs in 
branch '3.9':
[3.9] bpo-42163, bpo-42189, bpo-42659: Support uname_tuple._replace (for all 
but processor) (GH-23010) (#24232)
https://github.com/python/cpython/commit/799722cb0ddb90752cde7798cab543f30623ebf2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42163] _replace() no longer works on platform.uname_result objects

[Jason R. Coombs]

Jason R. Coombs  added the comment:


New changeset 799722cb0ddb90752cde7798cab543f30623ebf2 by Jason R. Coombs in 
branch '3.9':
[3.9] bpo-42163, bpo-42189, bpo-42659: Support uname_tuple._replace (for all 
but processor) (GH-23010) (#24232)
https://github.com/python/cpython/commit/799722cb0ddb90752cde7798cab543f30623ebf2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42163] _replace() no longer works on platform.uname_result objects

[Jason R. Coombs]

Change by Jason R. Coombs :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42189] copy.deepcopy() no longer works on platform.uname_result objects

[Jason R. Coombs]

Change by Jason R. Coombs :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42659] Objects of uname_result Class Cannot be Successfully Pickled

[Jason R. Coombs]

Change by Jason R. Coombs :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42935] Pickle can't import builtins at exit

[Christian Heimes]

Christian Heimes  added the comment:

At the end of a Python process, the interpreter is shut down in multiple steps. 
Object finalizers such as __del__ may be executed late in the interpreter shut 
down process. In your case, most of the interpreter is already gone. There 
isn't anything we can do to solve the problem.

That's why we recommend explicit resource management with the "with" statement. 
The atexit module is another way to execute cleanup hooks before the 
interpreter is shut down.

--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42885] Optimize re.search() for \A (and maybe ^)

[Arnim Rupp]

Change by Arnim Rupp :


--
components: +Regular Expressions -Library (Lib)
nosy: +ezio.melotti, mrabarnett

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42942] Feature request: Add decdigest() to hashlib

[Arnim Rupp]

New submission from Arnim Rupp :

Problem: hashlib only offers digest() and hexdigest() but the fastest way to 
work with hashes is as integer.

The first thing loki does after getting the hashes is to convert them to int:
md5, sha1, sha256 = generateHashes(fileData)
md5_num=int(md5, 16)
sha1_num=int(sha1, 16)
sha256_num=int(sha256, 16)
https://github.com/Neo23x0/Loki/blob/master/loki.py

All the ~5 hashes to compare are also converted to int after reading them 
from a file. The comparison is about twice as fast compared to hexdigest in 
strings because it uses just half the memory. 

(The use case here is to compare these 50,000 hashes to the hashes of all the 
200,000 files on a system that gets scanned for malicious files.)

Solution: Add decdigest() to hashlib which returns the int version of the hash. 
This has 2 advantages: 
1. It saves the time for converting the hash to hex and back
2. Having decdigest() in the documentation inspires more programmers to work 
with hashes as int opposed to slow strings (where it's performance relevant.)

Should be just few lines of code for each algorithm, I could do the PR.

static PyObject *
_sha3_shake_128_hexdigest(SHA3object *self, PyObject *arg)
{
PyObject *return_value = NULL;
unsigned long length;

if (!_PyLong_UnsignedLong_Converter(arg, &length)) {
goto exit;
}
return_value = _sha3_shake_128_hexdigest_impl(self, length);

https://github.com/python/cpython/blob/63298930fb531ba2bb4f23bc3b915dbf1e17e9e1/Modules/_sha3/clinic/sha3module.c.h

--
components: Library (Lib)
messages: 385150
nosy: 2d4d
priority: normal
severity: normal
status: open
title: Feature request: Add decdigest() to hashlib
type: performance
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42943] singledispatchmethod should expose registry of all known overloads

[Ilya Kulakov]

New submission from Ilya Kulakov :

The wrapper created by singledispatchmethod does not (trivially) expose 
registry of all known overloads.
Consider the following example:


@singledispatchmethod
def on_message(message):
raise NotImplementedError

@on_message.register
def _(message: MessageA): ...

@on_message.register
def _(message: MessageB): ...

loop = ...
condition = ...
messages = []

def receive_message_on_thread(message):
if ...:  # only signal to asyncio if message can be handled
messages.append(message)
loop.call_soon_threadsafe(condition.notify_all)
else:
...

async def process_messages_on_asyncio():
while True:
await condition.wait_for(lambda: len(messages) > 0)

while len(messages):
m = messages.pop(0)
...


Here messages are delivered outside of asyncio in a separate thread and thus 
they need to be forwarded into asyncio for further processing in the app. If 
the flow of messages is high it is desirable to filter inside the thread 
handler before signaling asyncio.

There are multiple approaches to describe which messages can be handled by the 
asyncio processor, but singledispatchmethod is the most elegant as it allows to 
keep everything in one place without if/else or code duplication.

Having a registry (trivially) exposed would allow to write the condition as 
`isinstance(message, tuple(on_message.registry.keys()))`.

--
components: Library (Lib)
messages: 385151
nosy: Ilya.Kulakov, lukasz.langa, methane, ncoghlan
priority: normal
severity: normal
status: open
title: singledispatchmethod should expose registry of all known overloads
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42944] random.Random.sample bug when counts is not None

[Jon FRANCO]

New submission from Jon FRANCO :

Hello,

If I am reading right, random.Random.sample method has a bug if counts is not 
None.

Line 482 (of master):
"""
selections = sample(range(total), k=k)
"""
this is calling the module function 'sample' and not the instance method.

IMO this line should be:
"""
selections = self.sample(range(total), k=k)
"""

Python 3.9.1 (default, Dec 11 2020, 14:32:07) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from random import Random
>>> r = Random()
>>> r.seed('bug')
>>> r.sample(range(50), 5, counts=range(1,51))
[34, 39, 31, 42, 38]
>>> r.seed('bug')
>>> r.sample(range(50), 5, counts=range(1,51))
[39, 11, 3, 12, 29]  <== this should be [34, 39, 31, 42, 38]
>>> r.seed('nobug')
>>> r.sample(range(50), 5)
[0, 30, 23, 17, 49]
>>> r.seed('nobug')
>>> r.sample(range(50), 5)
[0, 30, 23, 17, 49]

Regards,
Jon FRANCO

--
components: Library (Lib)
messages: 385152
nosy: jonfranco, rhettinger
priority: normal
severity: normal
status: open
title: random.Random.sample bug when counts is not None
type: behavior
versions: Python 3.10, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42944] random.Random.sample bug when counts is not None

[Raymond Hettinger]

Change by Raymond Hettinger :


--
assignee:  -> rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42944] random.Random.sample bug when counts is not None

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Would you like to submit a PR for this?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32509] doctest syntax ambiguity between continuation line and ellipsis

[Jason R. Coombs]

Jason R. Coombs  added the comment:

I've encountered this issue again with a different use-case.

I'm attempting to add a doctest to a routine that emits the paths of the files 
it processes. I want to use ellipses to ignore the prefixes of the output 
because they're not pertinent to the test. Here's the test that might have 
worked: 
https://github.com/python/importlib_resources/commit/ca9d014e1b884ff7f8cee63a436832a3e6e809fb,
 but failed with:

```
___ ERROR collecting 
importlib_resources/tests/update-zips.py ___
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/doctest.py:939: 
in find
self._find(tests, obj, name, module, source_lines, globs, {})
.tox/python/lib/python3.9/site-packages/_pytest/doctest.py:522: in _find
doctest.DocTestFinder._find(  # type: ignore
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/doctest.py:1001:
 in _find
self._find(tests, val, valname, module, source_lines,
.tox/python/lib/python3.9/site-packages/_pytest/doctest.py:522: in _find
doctest.DocTestFinder._find(  # type: ignore
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/doctest.py:989: 
in _find
test = self._get_test(obj, name, module, globs, source_lines)
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/doctest.py:1073:
 in _get_test
return self._parser.get_doctest(docstring, globs, name,
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/doctest.py:675: 
in get_doctest
return DocTest(self.get_examples(string, name), globs,
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/doctest.py:689: 
in get_examples
return [x for x in self.parse(string, name)
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/doctest.py:651: 
in parse
self._parse_example(m, name, lineno)
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/doctest.py:709: 
in _parse_example
self._check_prompt_blank(source_lines, indent, name, lineno)
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/doctest.py:793: 
in _check_prompt_blank
raise ValueError('line %r of the docstring for %s '
E   ValueError: line 6 of the docstring for 
importlib_resources.tests.update-zips.main lacks blank after ...: 
'.../data01/utf-16.file -> ziptestdata/utf-16.file'
```

I was able to work around the issue by injecting a newline into the output 
(https://github.com/python/importlib_resources/commit/b8d48d5a86a9f5bd391c18e1acb39b5697f7ca40).

I notice also that in some environments that the test still fails due to the 
arbitrary ordering of the output, but that test does pass in some environments.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42945] weakref.finalize documentation contradicts itself RE: finalizer callback or args referencing object

[Ryan Heisler]

New submission from Ryan Heisler :

In the documentation for `weakref.finalize` 
(https://docs.python.org/3.9/library/weakref.html#weakref.finalize), it says:

"Note It is important to ensure that func, args and kwargs do not own any 
references to obj, either directly or indirectly, since otherwise obj will 
never be garbage collected. In particular, func should not be a bound method of 
obj."

However, at the bottom of the document, in the section called "Comparing 
finalizers with __del__() methods" 
(https://docs.python.org/3.8/library/weakref.html#comparing-finalizers-with-del-methods),
 the following code is part of an example of how to use `weakref.finalize`:

class TempDir:
def __init__(self):
self.name = tempfile.mkdtemp()
self._finalizer = weakref.finalize(self, shutil.rmtree, self.name)

I believe this code violates the rule that func, args, and kwargs should not 
have a reference to obj. In the example, obj is the instance of TempDir, and 
one of the arguments to finalize's callback is an attribute of the same 
instance of TempDir.

I do not know how to fix this example code. I found it while trying to figure 
out how to use `weakref.finalize`.

--
assignee: docs@python
components: Documentation
messages: 385155
nosy: docs@python, ryan.a.heisler
priority: normal
severity: normal
status: open
title: weakref.finalize documentation contradicts itself RE: finalizer callback 
or args referencing object
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42945] weakref.finalize documentation contradicts itself RE: finalizer callback or args referencing object

[Tim Peters]

Tim Peters  added the comment:

Not a problem. Arguments to a function are evaluated before the function is 
invoked.  So in

self._finalizer = weakref.finalize(self, shutil.rmtree, self.name)

self.name is evaluated before weakref.finalize is called(). `self.name` 
_extracts_ the `.name` attribute of `self`, and the result is simply a pathname 
(returned by the earlier call to `mkdtemp()`), from which `self` cannot be 
accessed.

Just like, e.g., for just about any old object O, after

O.name = 42

a later `O.name` extracts the int 42, with no trace of that 42 had anything to 
do with `O`.

This isn't so when for a bound method object: `O.method_name` constructs and 
returns an object that _does_ reference `O`. That's why the earlier docs 
highlight bound method objects. For an attribute `name` bound to a chunk of 
data, `O.name` just retrieves that data, which _generally_ has nothing to do 
with `O` beyond that it happens to be reachable from `O` (but also generally 
NOT the reverse).

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42945] weakref.finalize documentation contradicts itself RE: finalizer callback or args referencing object

[Ryan Heisler]

Ryan Heisler  added the comment:

Perfect, thanks for your quick response. I was passing a bound method of obj as 
the func to `weakref.finalize(obj, func, /, *args, **kwargs)`. It slipped my 
mind that an instance variable like self.name and a bound method like 
self.clean_up, though they both belong to self, would be evaluated differently.

For anyone wondering, I was looking for weakref.proxy 
(https://docs.python.org/3/library/weakref.html#weakref.proxy) or 
weakref.WeakMethod 
(https://docs.python.org/3/library/weakref.html#weakref.WeakMethod)

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com