[issue42340] KeyboardInterrupt should come with a warning
Change by Benjamin Fogle : Added file: https://bugs.python.org/file49596/sigint_condition_2.py ___ Python tracker <https://bugs.python.org/issue42340> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42340] KeyboardInterrupt should come with a warning
New submission from Benjamin Fogle : This is related to bpo-29988, and I'm happy to move this to there. I made this a separate issue because this is a workaround, not a fix as was being discussed there. Also unlike bpo-29988, this is not restricted to context managers or finally blocks. TL;DR: Raising exceptions from interrupt handlers (most notably KeyboardInterrupt) can wreak havoc in ways that are impossible to fix. This should be noted in the documentation, with a workaround. I've attached a few example scripts that cause various strange behavior on Linux when a KeyboardInterrupt is raised at just the right time. There are likely many, many more possible examples: - sigint_condition_1.py: Cause a deadlock with threading.Condition - sigint_condition_2.py: Cause a double-release and/or notify on unacquired threading.Condition - sigint_tempfile.py: Cause NamedTemporaryFiles to not be deleted - sigint_zipfile.py: Cause ZipExtFile to corrupt its state When a user presses Ctrl+C, a KeyboardInterrupt will be raised on the main thread at some later time. This exception may be raised after any bytecode, and most Python code, including the standard library, is not designed to handle exceptions that spring up from nowhere. As a simple example, consider threading.Condition: def __enter__(self): return self._lock.__enter__() The KeyboardInterrupt could be raised just prior to return. In this case, __exit__ will never be called, and the underlying lock will remain acquired. A similar problem occurs if KeyboardInterrupt occurs at the start of __exit__. This can be mitigated by attempting to catch a KeyboardInterrupt *absolutely everywhere*, but even then, it can't be fixed completely. def __enter__(self): try: # it could happen here, in which case we should not unlock ret = self._lock.__enter__() # it could happen here, in which case we must unlock except KeyboardInterrupt: # it could, in theory, happen again right here ... raise return ret # it could happen here, which is the same problem we had before This is not restricted to context handlers or try/finally blocks. The zipfile module is a good example of code that is almost certain to enter an inconsistent state if a KeyboardInterrupt is raised while it's doing work: class ZipExtFile: ... def read1(self, n): ... self._readbuffer = b'' # what happens if KeyboardInterrupt happens here? self._offset = 0 ... Due to how widespread this is, it's not worth "fixing". (And honestly, it seems to be a rare problem in practice.) I believe that it would be better to clearly document that KeyboardInterrupt (or any exception propagated from a signal handler) may leave the system in an inconsistent state. Complex or high reliability applications should avoid catching KeyboardInterrupt as a way of gracefully shutting down, and should prefer registering their own SIGINT handler. They should also avoid raising exceptions from signal handlers at all. -- assignee: docs@python components: Documentation messages: 380868 nosy: benfogle, docs@python priority: normal severity: normal status: open title: KeyboardInterrupt should come with a warning type: enhancement versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue42340> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42340] KeyboardInterrupt should come with a warning
Change by Benjamin Fogle : Added file: https://bugs.python.org/file49595/sigint_condition_1.py ___ Python tracker <https://bugs.python.org/issue42340> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42340] KeyboardInterrupt should come with a warning
Change by Benjamin Fogle : Added file: https://bugs.python.org/file49597/sigint_tempfile.py ___ Python tracker <https://bugs.python.org/issue42340> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42340] KeyboardInterrupt should come with a warning
Change by Benjamin Fogle : Added file: https://bugs.python.org/file49598/sigint_zipfile.py ___ Python tracker <https://bugs.python.org/issue42340> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42340] KeyboardInterrupt should come with a warning
Change by Benjamin Fogle : -- keywords: +patch pull_requests: +22151 stage: -> patch review pull_request: https://github.com/python/cpython/pull/23255 ___ Python tracker <https://bugs.python.org/issue42340> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31976] Segfault when closing BufferedWriter from a different thread
New submission from Benjamin Fogle :
To reproduce:
```
import threading
import io
import time
import _pyio
class MyFileIO(io.FileIO):
def flush(self):
# Simulate a slow flush. Slow disk, etc.
time.sleep(0.25)
super().flush()
raw = MyFileIO('test.dat', 'wb')
#fp = _pyio.BufferedWriter(raw)
fp = io.BufferedWriter(raw)
t1 = threading.Thread(target=fp.close)
t1.start()
time.sleep(0.1) # Ensure t1 is sleeping in fp.close()/raw.flush()
fp.write(b'test')
t1.join()
```
_pyio.BufferedWriter ignores the error completely rather than throwing a
ValueError("write to closed file").
--
components: Interpreter Core
messages: 305803
nosy: benfogle
priority: normal
severity: normal
status: open
title: Segfault when closing BufferedWriter from a different thread
versions: Python 3.6, Python 3.7
___
Python tracker
<https://bugs.python.org/issue31976>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31976] Segfault when closing BufferedWriter from a different thread
Change by Benjamin Fogle : -- keywords: +patch pull_requests: +4287 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue31976> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31976] Segfault when closing BufferedWriter from a different thread
Change by Benjamin Fogle : -- type: -> crash ___ Python tracker <https://bugs.python.org/issue31976> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27517] LZMACompressor and LZMADecompressor raise exceptions if given empty strings twice
New submission from Benjamin Fogle: To reproduce: >>> import lzma >>> c = lzma.LZMACompressor() >>> c.compress(b'') b'\xfd7zXZ\x00\x00\x04\xe6\xd6\xb4F' >>> c.compress(b'') b'' >>> c.compress(b'') Traceback (most recent call last): File "", line 1, in _lzma.LZMAError: Insufficient buffer space >>> d = lzma.LZMADecompressor() >>> d.decompress(b'') b'' >>> d.decompress(b'') Traceback (most recent call last): File "", line 1, in _lzma.LZMAError: Insufficient buffer space This can occur anytime during compression/decompression when an empty string is passed twice in a row. The problem is that that liblzma interprets a call to lzma_code() with avail_in == 0 as a buffer full condition. The second time in a row it encounters this, it returns LZMA_BUF_ERROR as per documentation. The attached patch prevents this condition from occurring. -- components: Extension Modules files: lzma.patch keywords: patch messages: 270451 nosy: benfogle priority: normal severity: normal status: open title: LZMACompressor and LZMADecompressor raise exceptions if given empty strings twice type: behavior versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6 Added file: http://bugs.python.org/file43727/lzma.patch ___ Python tracker <http://bugs.python.org/issue27517> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27517] LZMACompressor and LZMADecompressor raise exceptions if given empty strings twice
Benjamin Fogle added the comment: The above code demonstrates two separate issues. One with the decompressor, and one with the compressor. In the compressor example, the first output differs from the rest because it is a file header which is always emitted. That behavior is correct. -- ___ Python tracker <http://bugs.python.org/issue27517> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27517] LZMACompressor and LZMADecompressor raise exceptions if given empty strings twice
Benjamin Fogle added the comment: Ah, thank you. Good catch. I have reworked the patch to handle both cases. -- Added file: http://bugs.python.org/file45286/lzma_2.patch ___ Python tracker <http://bugs.python.org/issue27517> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
