[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows
Evgeny added the comment: Dear all, how can we realistically move this forward? This issue is 9 years old by now. Everybody from the discussion agrees, this is an issue. There were several proposals made, all of them slightly different. 7 months ago I have implemented solution, pretty much similar to the one, proposed by eryksun is the https://bugs.python.org/msg376656 https://github.com/python/cpython/pull/22431 So far I received no comments. I am not quite familiar with the decision making in Python development. Who can take a decision which way to go? -- ___ Python tracker <https://bugs.python.org/issue14243> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows
Evgeny added the comment: Dear all, thank you very much for the discussion, I will just try to summarize the results of it. In my PR I used solution, proposed by Eryk. My solution involves introduction of the extra flag delete_on_close and making sure, that new situation is fully backwards compatible, because I did not feel I have an authority to propose anything which would break backwards compatibility (as Python 4.0 I think is not on the horizon yet) As I see from the discussion, the following decisions need to be taken: WHICH FLAG TO USE Eryk was proposing that it shall be delete_on_close (which I have implemented) Jason thinks, that we shall introduce flag delete_when, however when I read his proposal, the functionality which he proposes is not that different from what I implemented with delete_on_close. Ethan however thinks, that no extra flags are necessary at all USAGE OF O_TEMPORARY ON WINDOWS Ethan, Steve thinks, it is not needed Eryk prefers to provide a way to omit O_TEMPORARY, but still use it by default, when it's omitted CHANGING OF THE CURRENT BEHAVIOUR / BREAKING BACKWARDS COMPATIBILITY Ethan thinks, that we shall slightly change backwards compatibility in a way that if CM is used, than file is deleted on CM exit and not on close as now Jason thinks that backwards compatibility shall be changed gradually over several releases with the advanced depreciation warning Question: does everybody definitely agree then, that backwards compatibility shall definitely be altered, whether immediately or gradually? Any other decision to be taken which I missed? CONCLUSION: There seems to be still disagreements and I don't really know how to move this forward as I am not sure I understand the decision making mechanism in the Python community (especially when it comes to breaking backwards compatibility). Any thoughts on this? -- ___ Python tracker <https://bugs.python.org/issue14243> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows
Evgeny added the comment: On Mon, Apr 12, 2021 at 12:51 AM Jason R. Coombs wrote: > Jason R. Coombs added the comment: > At least I and Ethan and Martin have expressed a desire for the default, > preferred usage work well in a portable environment. Requiring > `delete_on_close=False` violates that expectation. Separately to my previous message. If the only issue with my PR is that default solution is not portable, then this can be simply changed within current PR by setting default of delete_on_close to fe equal to False. As I mentioned, I was assuming I cannot break backwards compatibility. -- ___ Python tracker <https://bugs.python.org/issue14243> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows
Evgeny added the comment: >Eryk Sun added the comment: >I replied twice that I thought using the CM exit instead of O_TEMPORARY is >okay for NamedTemporaryFile() -- but only if a separate implementation of >TemporaryFile() that uses O_TEMPORARY is added at the same time. I want >guaranteed cleanup for TemporaryFile() since it's not intended to be reopened. At the moment, the TemporaryFile directly reuses NamedTemporaryFile for none-posix or cygwin systems. https://github.com/python/cpython/blob/a92d7387632de1fc64de51f22f6191acd0c6f5c0/Lib/tempfile.py#L552 Does it mean, that your suggestion to leave the O_TEMPORARY for TemporaryFile means, that NamedTemporaryFile needs to have a mechanism to know whether it was called as a TemporaryFile and then to have a different functionality in this case relative to the situation it would be called directly? -- ___ Python tracker <https://bugs.python.org/issue14243> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows
Evgeny added the comment: >Paul Moore added the comment: >Evgeny, would you be willing to update your PR (including adding the docs >change, and tests to catch as many edge cases as you can think up) to match >this behaviour? Paul, thank you for moving this forward. I will give it a try. -- ___ Python tracker <https://bugs.python.org/issue14243> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows
Evgeny added the comment: > Eryk Sun added the comment: > Just implement a separate function for TemporaryFile() instead of aliasing it > to NamedTemporaryFile(). See msg390814. Eryk, forgive my ignorance, but aren't in your msg390814 you are proposing yet another enhancement (separate from the issue14243, discussed here), in this case for TemporaryFile in Windows systems? I may be mistaken, but I see that you are proposing some trick with first unlinking the file and then employing it as a temporary file using previously known fd. This "trick" is not present in the current code and does not seem to address issue14243. Or am I talking total nonsense? -- ___ Python tracker <https://bugs.python.org/issue14243> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows
Evgeny added the comment: Eryk, I agree, that implementing TemporaryFile() in Windows goes hand in hand with the decision to stop using O_TEMPORARY in NamedTemporaryFile() The only thing I want to point out is that your suggestion also includes this "unlinking trick" (sorry, may be there is a better description for this), which seems to be separate/extra to the usage of O_TEMPORARY. Hence my newbie questions are: 1) What problem are you trying to solve by this "unlinking trick"? 2) Do we need to have a separate issue raised for this problem? 3) Is this "unlinking trick" using some documented features of Windows? (will it not stop working after some Windows patch)? 4) Will we need to create separate unit tests for this issue? 5) Do we also need to reach a consensus on implementing of this "unlinking trick"? -- ___ Python tracker <https://bugs.python.org/issue14243> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows
Change by Evgeny : -- keywords: +patch pull_requests: +22618 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/22431 ___ Python tracker <https://bugs.python.org/issue14243> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows
Evgeny added the comment: Hello, this is to let you know, that I have created a pull request for this issue https://github.com/python/cpython/pull/22431 I am not really an experienced programmer, but I will give it a try -- nosy: +ev2geny ___ Python tracker <https://bugs.python.org/issue14243> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16399] argparse: append action with default list adds to list instead of overriding
Evgeny added the comment: You don't need action='append'. For desired behavior you can pass action='store' with nargs='*'. I think it's a simplest workaround. -- nosy: +gun146 ___ Python tracker <https://bugs.python.org/issue16399> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12983] byte string literals with invalid hex escape codes raise ValueError instead of SyntaxError
Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker <http://bugs.python.org/issue12983> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12729] Python lib re cannot handle Unicode properly due to narrow/wide bug
Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker <http://bugs.python.org/issue12729> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12251] subprocess(..., stdout=sys.stderr) closes stderr for child
Changes by Evgeny Tarasov : -- nosy: +Evgeny.Tarasov ___ Python tracker <http://bugs.python.org/issue12251> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12464] tempfile.TemporaryDirectory.cleanup follows symbolic links
New submission from Evgeny Kapun :
TemporaryDirectory.cleanup follows symbolic links to directories and tries to
clean them as well. Try this (on Linux):
import os, tempfile
with tempfile.TemporaryDirectory() as d:
os.symlink("/proc", d + "/test")
--
components: Library (Lib)
messages: 139571
nosy: abacabadabacaba
priority: normal
severity: normal
status: open
title: tempfile.TemporaryDirectory.cleanup follows symbolic links
type: behavior
versions: Python 3.2, Python 3.3
___
Python tracker
<http://bugs.python.org/issue12464>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12465] gc.get_referents can be used to crash Python
New submission from Evgeny Kapun :
This code crashes Python:
import gc
gc.get_referents(object.__dict__)[0].clear()
gc.get_referents(type.__dict__)[0].clear()
type("A", (), {})()
--
components: Interpreter Core
messages: 139572
nosy: abacabadabacaba
priority: normal
severity: normal
status: open
title: gc.get_referents can be used to crash Python
type: crash
versions: Python 2.7, Python 3.3
___
Python tracker
<http://bugs.python.org/issue12465>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10451] memoryview can be used to write into readonly buffer
New submission from Evgeny Kapun : This code crashes Python: import io, mmap io.BytesIO(b' ').readinto(memoryview(mmap.mmap(-1, 1, prot=mmap.PROT_READ))) -- components: Interpreter Core messages: 121446 nosy: abacabadabacaba priority: normal severity: normal status: open title: memoryview can be used to write into readonly buffer type: crash versions: Python 3.1, Python 3.2 ___ Python tracker <http://bugs.python.org/issue10451> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38430] Memory leak in ThreadPoolExecutor + run_in_executor
New submission from Evgeny Nizhibitsky : I have run into a memory leak caused by using run_in_executor + ThreadPoolExecutor while running some stability tests with custom web services. It was 1 MB leaked for 1k requests made for my case and I've extracted the root cause and converted it into minimal script with both mentioned parts + just NOP function to "run". The script can easily eat up to 1 GB of memory in less then 1 minute now. It uses external psutil library to report the memory allocated but it can be easily commented out and the leak will stay anyway. One can found that script attached + Dockerfile/Makefile for reproducibility. I've also reproduced it in my own conda-based 3.7 environment as well as the master branch of cpython. -- components: asyncio files: python-leak.zip messages: 354351 nosy: Evgeny Nizhibitsky, asvetlov, yselivanov priority: normal severity: normal status: open title: Memory leak in ThreadPoolExecutor + run_in_executor type: resource usage versions: Python 3.6, Python 3.7, Python 3.9 Added file: https://bugs.python.org/file48654/python-leak.zip ___ Python tracker <https://bugs.python.org/issue38430> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38430] Memory leak in ThreadPoolExecutor + run_in_executor
Evgeny Nizhibitsky added the comment: Oh, I see that in the initial code with leakage (it was heavy ThreadPoolExecutor + xgboost thing) there was an await but I must have lost it somewhere while reducing it to the minimal example and finished in the wrong direction. Glad too see that it raised a discussion to prevent others from getting into this silent trap. -- ___ Python tracker <https://bugs.python.org/issue38430> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
New submission from Evgeny Boytsov : Hello everybody! We are using Python 3.7 running at CentOS 7 x64. Python is used as a library to create dynamic extensions for our app server. Some time ago we began to experience crashes in decimal module in some heavy-multithreaded scenarios. After some testing and debugging I was able to reproduce it without our own code using only pybind11 library to simplify embedding (in real app we are using boost.python). I've built python 3.8 with clang 7 and address sanitizer enabled and got error "use-after-free" with some additional data. Please find attached C++ source file, python module and ASAN output. Is it really a bug (most probably - data race) or there is something wrong with such embedding scenario? -- components: Interpreter Core files: decimal_crash.zip messages: 362807 nosy: boytsovea priority: normal severity: normal status: open title: Crash in decimal module in heavy-multithreaded scenario versions: Python 3.7, Python 3.8 Added file: https://bugs.python.org/file48923/decimal_crash.zip ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Evgeny Boytsov added the comment: Please note, that UnlockGILandSleep takes GIL back before returning. In a real production code there is a database query. In this example I emulate them with random sleep. So I don't see any problems here. -- ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Evgeny Boytsov added the comment: Thank you for feedback. I will try to reproduce the issue with 3.6. By the way, haven't you used gdb with python pretty-printers enabled to examine the state of the program? I've got the same error message, then I breaked the execution in debugger and tried to examine the callstack of threads, that stucked in UnlockGILandSleep. The reason for it is clear: then the debugger tries to build a callstack, some of pretty printers try to execute some python code to give a better representation of interpreter objects. The code is executed at the top of the stack of the examined thread. Since this thread explicitly released the GIL before going to sleep, these functions hit the assert about calling the memory allocator without holdng the GIL. Disabling pretty-printers makes these error messages to disappear. -- ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Evgeny Boytsov added the comment: I'am unable to reproduce neither my or your issues with python 3.6. The program runs infinitely as it meant to be. Can you please give me C++ traceback from the core dump, which was created when you ran my program? -- ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Evgeny Boytsov added the comment: Your callstack is very strange. At line 30 of main.cpp GIL is obviously locked: // importing module in this tread gstate = PyGILState_Ensure(); py::module crash_test = py::module::import( "crash_test" ); <-- import PyGILState_Release( gstate ); I suppose that there is something wrong with your setup. Maybe - wrong working directory for the main executable, which doesn't contain crash_test.py Also I've tried to revert this patch https://github.com/python/cpython/pull/5278 for 3.7. It makes problem to disappear, 1 hour of stable work under ASAN. So I suppose it is the source of the bug. I will try to tweak _testembed.c. -- resolution: -> not a bug ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Evgeny Boytsov added the comment: I rewrote my example without pybind and eliminated C++ module (I realized that time.sleep() also releases the GIL, so we achieve the same effect). Still the same results: with python 3.7.3 app crashes with attached ASAN output, with python 3.7.3 without https://github.com/python/cpython/pull/5278 works just fine. To run main.cpp you should add directory with crash_test.py to PYTHONPATH. -- Added file: https://bugs.python.org/file48930/threaded_crash.zip ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] Crash in decimal module in heavy-multithreaded scenario
Evgeny Boytsov added the comment: Also I understood the source of your crash with my initial example. Since you haven't used CMake to configure project, pybind didn't setup required macroses to enable threading support. So no issues in pybind. -- ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39776] PyContextVar_Get(): crash due to race condition in updating tstate->id
Evgeny Boytsov added the comment: I checked both my test example and real production code with your patch. I'm unable to reproduce the bug, so I think it is fixed now. Thank you! -- ___ Python tracker <https://bugs.python.org/issue39776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43141] `asdict` fails with frozen dataclass keys; tries to use raw dict form as key
New submission from Evgeny Naumov :
A simple example is documented here:
https://github.com/samuelcolvin/pydantic/issues/2233 but it doesn't look like
it was actually filed as a bug against CPython... Copy paste of the minimal
reproducing example:
from dataclasses import asdict, dataclass
@dataclass(eq=True, frozen=True)
class A:
a: str
@dataclass(eq=True, frozen=True)
class B:
b: dict[A, str]
asdict(B({A("x"):"y"}))
The offending code are these lines in dataclasses.py:
elif isinstance(obj, dict):
return type(obj)((_asdict_inner(k, dict_factory),
_asdict_inner(v, dict_factory))
The _asdict_inner(k, dict_factory) [correctly] turns dataclasses into dicts...
which are not hashable. I assume there is a good reason the _asdict_inner
function is called on the key, but its failure in this case looks like a
(rather annoying!) error.
--
components: Library (Lib)
messages: 386545
nosy: enaumov
priority: normal
severity: normal
status: open
title: `asdict` fails with frozen dataclass keys; tries to use raw dict form as
key
type: behavior
versions: Python 3.9
___
Python tracker
<https://bugs.python.org/issue43141>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1508475] transparent gzip compression in urllib
Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker <http://bugs.python.org/issue1508475> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30929] AttributeErrors after import in multithreaded environment
New submission from Evgeny Boytsov: Hello everybody! We are using Python 3.4 running at CentOS 7 x64 and experiencing some problems with simulatenous import of modules from different threads of execution. The attached archive contains simple example, which demonstrates the issue. There is python package called "mod". It contains two sub-packages called "foo" and "bar". They, in turn, contain modules "foo" and "bar" respectively. Another package called "threader" contains function "run_threads()", which runs two threads of execution (using threading) and joins them. First thread of execution imports "mod.foo.foo" from "mod" and calls its function. The second thread does the same, but for module "mod.bar.bar". Finally, there is module "test.py" which calls "threader.run_threads()" and when it returns, tries to import "mod.foo.foo" and call its function. To reproduce the bug you need to run "test.py" for several times. Most of the launches complete successfully. But sometimes we encounter AttributeError during import like that: 'module' object has no attribute 'foo' We've done some investigation in order to clarify this behaviour and added import hook (you can see it in test.py), which logs all import that take place. When the launch completes successfully, the order of imports looks like that: $ python3 test.py [py][import] threader [py][import] threading [py][import] time [py][import] traceback [py][import] linecache [py][import] tokenize [py][import] collections [py][import] _collections [py][import] operator [py][import] _operator [py][import] keyword [py][import] heapq [py][import] itertools [py][import] _heapq [py][import] reprlib [py][import] re [py][import] sre_compile [py][import] _sre [py][import] sre_parse [py][import] sre_constants [py][import] _locale [py][import] copyreg [py][import] token [py][import] mod [py][import] mod.foo [py][import] mod.bar [py][import] mod.foo.foo [py][import] mod.bar.bar print from foo print from bar When the issue arises we see the following: $ python3 test.py [py][import] threader [py][import] threading [py][import] time [py][import] traceback [py][import] linecache [py][import] tokenize [py][import] collections [py][import] _collections [py][import] operator [py][import] _operator [py][import] keyword [py][import] heapq [py][import] itertools [py][import] _heapq [py][import] reprlib [py][import] re [py][import] sre_compile [py][import] _sre [py][import] sre_parse [py][import] sre_constants [py][import] _locale [py][import] copyreg [py][import] token [py][import] mod [py][import] mod.foo [py][import] mod [py][import] mod.bar [py][import] mod.foo.foo [py][import] mod.bar.bar thread of foo: import error 'module' object has no attribute 'foo' print from bar That is when the issue arises there are two imports of package "mod". And the most confusing thing about this scenario is that even after completion of "run_threads()" interpreter can not import "mod.foo.foo" and gives the same AttributeErrors in for-loop inside test.py. -- components: Interpreter Core files: mod.zip messages: 298349 nosy: boytsovea priority: normal severity: normal status: open title: AttributeErrors after import in multithreaded environment type: behavior versions: Python 3.4 Added file: http://bugs.python.org/file47014/mod.zip ___ Python tracker <http://bugs.python.org/issue30929> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30929] AttributeErrors after import in multithreaded environment
Evgeny Boytsov added the comment: The same behaviour is reprodusible at ubuntu 16.04 with python 3.5.2. And a colleague of mine said that he was able to reproduce the issue with python 3.6. -- ___ Python tracker <http://bugs.python.org/issue30929> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30048] If a task is canceled at the right moment, the cancellation is ignored
New submission from Evgeny Kapun:
If I run this code:
import asyncio as a
@a.coroutine
def coro1():
yield from a.ensure_future(coro2())
print("Still here")
yield from a.sleep(1)
print("Still here 2")
@a.coroutine
def coro2():
yield from a.sleep(1)
res = task.cancel()
print("Canceled task:", res)
loop = a.get_event_loop()
task = a.ensure_future(coro1())
loop.run_until_complete(task)
I expect the task to stop shortly after a call to cancel(). It should surely
stop when I try to sleep(). But it doesn't. On my machine this prints:
Canceled task: True
Still here
Still here 2
So, cancel() returns True, but the task doesn't seem to be canceled.
--
components: asyncio
messages: 291522
nosy: abacabadabacaba, yselivanov
priority: normal
severity: normal
status: open
title: If a task is canceled at the right moment, the cancellation is ignored
type: behavior
versions: Python 3.5, Python 3.6
___
Python tracker
<http://bugs.python.org/issue30048>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30048] If a task is canceled at the right moment, the cancellation is ignored
Evgeny Kapun added the comment: The problem is that the task doesn't catch CancelledError, yet it disappears. -- ___ Python tracker <http://bugs.python.org/issue30048> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30064] BaseSelectorEventLoop.sock_{recv, sendall}() don't remove their callbacks when canceled
New submission from Evgeny Kapun:
Code:
import asyncio as a
import socket as s
@a.coroutine
def coro():
s1, s2 = s.socketpair()
s1.setblocking(False)
s2.setblocking(False)
try:
yield from a.wait_for(loop.sock_recv(s2, 1), 1)
except a.TimeoutError:
pass
yield from loop.sock_sendall(s1, b'\x00')
yield
s1.close()
s2.close()
loop = a.get_event_loop()
loop.run_until_complete(coro())
Result:
Exception in callback BaseSelectorEventLoop._sock_recv(,
True, , 1)
handle: , True,
, 1)>
Traceback (most recent call last):
File "/usr/lib/python3.6/asyncio/events.py", line 127, in _run
self._callback(*self._args)
File "/usr/lib/python3.6/asyncio/selector_events.py", line 378, in
_sock_recv
self.remove_reader(fd)
File "/usr/lib/python3.6/asyncio/selector_events.py", line 342, in
remove_reader
return self._remove_reader(fd)
File "/usr/lib/python3.6/asyncio/selector_events.py", line 279, in
_remove_reader
key = self._selector.get_key(fd)
File "/usr/lib/python3.6/selectors.py", line 189, in get_key
return mapping[fileobj]
File "/usr/lib/python3.6/selectors.py", line 70, in __getitem__
fd = self._selector._fileobj_lookup(fileobj)
File "/usr/lib/python3.6/selectors.py", line 224, in _fileobj_lookup
return _fileobj_to_fd(fileobj)
File "/usr/lib/python3.6/selectors.py", line 41, in _fileobj_to_fd
raise ValueError("Invalid file descriptor: {}".format(fd))
ValueError: Invalid file descriptor: -1
--
components: asyncio
messages: 291593
nosy: abacabadabacaba, yselivanov
priority: normal
severity: normal
status: open
title: BaseSelectorEventLoop.sock_{recv,sendall}() don't remove their callbacks
when canceled
type: behavior
versions: Python 3.5, Python 3.6
___
Python tracker
<http://bugs.python.org/issue30064>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34035] zipfile: AttributeError in "seek" method of "_SharedFile" class
New submission from Evgeny Prilepin : The misprint in the file lib/zipfile.py in the line 704 leads to AttributeError: '_SharedFile' object has no attribute 'writing' "self.writing()" should be replaced by "self._writing()". I also think this code shold be covered by tests. -- components: Library (Lib) messages: 320996 nosy: espdev priority: normal severity: normal status: open title: zipfile: AttributeError in "seek" method of "_SharedFile" class type: crash versions: Python 3.7 ___ Python tracker <https://bugs.python.org/issue34035> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34035] zipfile: AttributeError in "seek" method of "_SharedFile" class
Evgeny Prilepin added the comment: I think the line 1031 also contains the misprint and will raise NameError. > self._decompressor = zipfile._get_decompressor(self._compress_type) "zipfile." is not correct code in this place. -- ___ Python tracker <https://bugs.python.org/issue34035> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34035] zipfile: AttributeError in "seek" method of "_SharedFile" class
Evgeny Prilepin added the comment: Hi Rémi, it would be great if you posted a PR. -- ___ Python tracker <https://bugs.python.org/issue34035> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18784] minor uuid.py loading optimization
New submission from Evgeny Sologubov: Please see the patch attached. It quite is primitive and self-explanatory: the code wouldn't attempt to load *libc* via ctypes.CDLL, if all necessary functions are already found in *libuuid*. This patch also can serve as a work-around solution to issue #17213 (which I encountered on cygwin+python27) -- components: Library (Lib), Windows, ctypes files: uuid.patch keywords: patch messages: 195653 nosy: eugals priority: normal severity: normal status: open title: minor uuid.py loading optimization type: performance versions: Python 2.7 Added file: http://bugs.python.org/file31377/uuid.patch ___ Python tracker <http://bugs.python.org/issue18784> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19005] PyIter_Next crashes if passed a non-iterator
New submission from Evgeny Kapun: According to the documentation, PyIter_Next should raise TypeError if passed an object which is not an iterator as an argument. Instead, it calls a function through a null pointer, which leads to a crash. -- components: Interpreter Core messages: 197483 nosy: abacabadabacaba priority: normal severity: normal status: open title: PyIter_Next crashes if passed a non-iterator type: crash versions: Python 3.3 ___ Python tracker <http://bugs.python.org/issue19005> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19423] permutations len issue
New submission from Evgeny Luttsev:
Code:
n = 2
perms = permutations(range(n), n)
if list(perms) == [(0, 1), (1, 0)]:
print("==")
print("len(list(perms)):", len(list(perms)))
Result:
==
len(list(perms)): 0 # SHOULD BE 2
--
components: Library (Lib)
messages: 201556
nosy: DSblizzard
priority: normal
severity: normal
status: open
title: permutations len issue
type: behavior
versions: Python 2.6, Python 3.1
___
Python tracker
<http://bugs.python.org/issue19423>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7434] general pprint rewrite
Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker <http://bugs.python.org/issue7434> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22524] PEP 471 implementation: os.scandir() directory scanning function
Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker <http://bugs.python.org/issue22524> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6252] logging midnigh rotation
New submission from Turnaev Evgeny :
i have a very basic setup of logging in a long running daemon app..
I use TimedRotatingFileHandler with rotating at midnight.
The bug:
The last open logging file was for 2009-05-25..
app was running without logging anything for about a week or more.
After a week past something happened and daemon started to log messages,
but.. for each new message logging system creates new file.
In other words.. suppose we was idle since 2009-05-25, now it is
2009-06-10 and we want to log 10 messages..
current opened file is for 2009-05-25.. so when a first messages
arrives into logging system current must be closed. and a new file must
be opened where all 10 messages should be..
but instead logging system will log each subsequent message in files
2009-05-26, 2009-05-27, 2009-05-28..(in fact it will log in file
without a date and then when nect message arives rename current to
2009-05-26, 2009-05-27, 2009-05-28.. but is still the same) and so on,
one message per file.. i think until it reaches current date.
It is a logic error.
my logging setup looks like this:
-
import logging
from logging import debug,warning,info,error,critical
from logging.handlers import TimedRotatingFileHandler
log_path = '/var/log/cherry/smsd_reg'
log_level = logging.DEBUG
basic_log_handler = TimedRotatingFileHandler(log_path + "/
app_log_smsd_reg_server",'midnight',1)
basic_log_handler.setLevel(log_level)
basic_log_formatter = logging.Formatter('%(asctime)s pid: %(process)d,
%(levelname)s %(message)s')
basic_log_handler.setFormatter(basic_log_formatter)
logging.getLogger('').addHandler(basic_log_handler)
logging.getLogger('').setLevel(log_level)
system info:
Python 2.5.1
FreeBSD 6.2-RELEASE-p7 FreeBSD 6.2-RELEASE-p7
--
components: Extension Modules
messages: 89191
nosy: SanityIO
severity: normal
status: open
title: logging midnigh rotation
type: behavior
versions: Python 2.5
___
Python tracker
<http://bugs.python.org/issue6252>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6252] logging midnight rotation
Turnaev Evgeny added the comment: I am sorry for my poor english. You must be misunderstood me. I attached a file try it like this: wget -o /dev/null http://localhost:9022/ then 5-7 times wget -o /dev/null http://localhost:9022/s then 4-5 times wget -o /dev/null http://localhost:9022/ i bet this error persists in 3.1 -- status: pending -> open Added file: http://bugs.python.org/file14263/httpd.py ___ Python tracker <http://bugs.python.org/issue6252> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9334] argparse does not accept options taking arguments beginning with dash (regression from optparse)
Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker <http://bugs.python.org/issue9334> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9334] argparse does not accept options taking arguments beginning with dash (regression from optparse)
Evgeny Kapun added the comment: The way how argparse currently parses option arguments is broken. If a long option requires an argument and it's value isn't specified together with the option (using --option=value syntax), then the following argument should be interpreted as that value, no matter what it looks like. There should be no guesses or heuristics here. That the behavior depends on whether some argument "looks like" a negative number is the most horrible. Argument parsing should follow simple, deterministic rules, preferably the same that used by standard getopt(3). -- ___ Python tracker <http://bugs.python.org/issue9334> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11406] There is no os.listdir() equivalent returning generator instead of list
Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker <http://bugs.python.org/issue11406> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17899] os.listdir() leaks FDs if invoked on FD pointing to a non-directory
New submission from Evgeny Kapun: When called with a file descriptor as an argument, os.listdir() duplicates it to pass to fdopendir(3). If this call fails, the new file descriptor is not closed, which leads to file descriptor leak. -- components: Library (Lib) messages: 188322 nosy: abacabadabacaba priority: normal severity: normal status: open title: os.listdir() leaks FDs if invoked on FD pointing to a non-directory type: resource usage versions: Python 3.3, Python 3.4 ___ Python tracker <http://bugs.python.org/issue17899> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17899] os.listdir() leaks FDs if invoked on FD pointing to a non-directory
Evgeny Kapun added the comment: To make fdopendir fail, just pass any valid FD which points to a non-directory, such as a file or a pipe. -- ___ Python tracker <http://bugs.python.org/issue17899> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17899] os.listdir() leaks FDs if invoked on FD pointing to a non-directory
Evgeny Kapun added the comment: Simple test: while True: try: listdir(0) except NotADirectoryError: pass -- ___ Python tracker <http://bugs.python.org/issue17899> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22609] Constructors of some mapping classes don't accept `self` keyword argument
New submission from Evgeny Kapun: >>> import collections >>> collections.Counter(self=1) Traceback (most recent call last): File "", line 1, in TypeError: __init__() got multiple values for argument 'self' >>> collections.OrderedDict(self="test") Traceback (most recent call last): File "", line 1, in TypeError: __init__() got multiple values for argument 'self' >>> collections.UserDict(self="test") Traceback (most recent call last): File "", line 1, in TypeError: __init__() got multiple values for argument 'self' Python surely should have positional-only parameters. -- components: Library (Lib) messages: 229076 nosy: abacabadabacaba priority: normal severity: normal status: open title: Constructors of some mapping classes don't accept `self` keyword argument type: behavior versions: Python 3.4 ___ Python tracker <http://bugs.python.org/issue22609> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24129] Incorrect (misleading) statement in the execution model documentation
Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker <http://bugs.python.org/issue24129> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25454] operator.methodcaller should accept additional arguments during the call
New submission from Evgeny Kapun: Currently, operator.methodcaller behaves like this: def methodcaller(name, *args, **kwargs): def caller(obj): return getattr(obj, name)(*args, **kwargs) return caller That is, it is possible to supply arguments when the object is created but not during the call. I think that it will be more useful if it behaved like this: def methodcaller(name, *args, **kwargs): def caller(obj, *args2, **kwargs2): return getattr(obj, name)(*args, *args2, **kwargs, **kwargs2) return caller -- components: Extension Modules messages: 253307 nosy: abacabadabacaba priority: normal severity: normal status: open title: operator.methodcaller should accept additional arguments during the call type: enhancement versions: Python 3.5, Python 3.6 ___ Python tracker <http://bugs.python.org/issue25454> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25455] Some repr implementations don't check for self-referential structures
New submission from Evgeny Kapun:
Implementations of repr for some of the types in the standard library doesn't
check for self-referential structures. As a result, when calling repr() on such
objects, Python crashes due to infinite recursion.
Example:
>>> import functools
>>> x = functools.partial(min)
>>> x.__setstate__((x, (), {}, {}))
>>> repr(x)
Segmentation fault
Another example:
>>> import xml.etree.ElementTree
>>> x = xml.etree.ElementTree.Element('')
>>> x.tag = x
>>> repr(x)
Segmentation fault
One more example:
>>> import io
>>> class X(io.TextIOWrapper): __slots__ = 'name'
>>> x = X(open('/dev/null'))
>>> x.name = x
>>> repr(x)
Segmentation fault
--
components: Extension Modules
messages: 253309
nosy: abacabadabacaba
priority: normal
severity: normal
status: open
title: Some repr implementations don't check for self-referential structures
type: crash
versions: Python 3.5, Python 3.6
___
Python tracker
<http://bugs.python.org/issue25455>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25454] operator.methodcaller should accept additional arguments during the call
Evgeny Kapun added the comment:
There are methods that accept a single argument and behave like a binary
operation or a predicate. It would be useful to be able to turn them into
binary functions for use with higher-order functions like map and reduce:
reduce(methodcaller("combine"), objs) # reduce a sequence using "combine"
method
map(methodcaller("combine"), alist, blist) # combine pairs of elements
all(map(methodcaller("compatible_with"), alist, blist)) # check that pairs
of elements are compatible
This functionality is already available for overloaded operators, because
operator module provides them as functions. Some methods behave similarly to
operators, so I think that a similar functionality should be available for them
as well.
--
___
Python tracker
<http://bugs.python.org/issue25454>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18531] Undocumented different between METH_KEYWORDS and **kws
Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker <http://bugs.python.org/issue18531> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2142] difflib.unified_diff(...) produces invalid patches
Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker <http://bugs.python.org/issue2142> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26254] ssl server doesn't work with ECC certificates
New submission from Evgeny Kapun:
I tried to use ssl module to create a server with a certificate that uses an
ECC key. However, this didn't work. Here is how to reproduce this:
First, generate a key and a certificate:
$ openssl req -newkey ec -pkeyopt ec_paramgen_curve:prime256v1 -x509
-keyout key.pem -out cert.pem
(type some passphrase, then just press Enter in response to the questions
that it asks)
Then run this Python program:
from socket import socket
from ssl import wrap_socket
s = socket()
s.bind(('localhost', 12345))
s.listen()
wrap_socket(s.accept()[0], 'key.pem', 'cert.pem', True)
This program will wait for a connection, so try to connect:
$ openssl s_client -connect localhost:12345
The program will ask for a passphrase, so type it. After that, you will get an
exception:
Traceback (most recent call last):
File "test.py", line 6, in
wrap_socket(s.accept()[0], 'key.pem', 'cert.pem', True)
File "/usr/lib/python3.5/ssl.py", line 1064, in wrap_socket
ciphers=ciphers)
File "/usr/lib/python3.5/ssl.py", line 747, in __init__
self.do_handshake()
File "/usr/lib/python3.5/ssl.py", line 983, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib/python3.5/ssl.py", line 628, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:645)
If the certificate uses RSA key, it works. With ECC, I had no luck. I tried
creating a context explicitly and using set_ciphers method to enable more
ciphers. While it appears to support ECDSA ciphersuites, it can't use them for
some reason.
--
components: Extension Modules
messages: 259305
nosy: abacabadabacaba
priority: normal
severity: normal
status: open
title: ssl server doesn't work with ECC certificates
type: behavior
versions: Python 3.5
___
Python tracker
<http://bugs.python.org/issue26254>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26254] ssl should raise an exception when trying to load an unusable key (ECC key not using a named curve)
Evgeny Kapun added the comment: So, it looks like OpenSSL doesn't support keys using arbitrary curves at all. Then why don't I get an exception when trying to load such a key? Instead it just quietly disables all authenticated ciphersuites (anonymous ciphersuites still work) and then I get a confusing exception about lack of shared ciphers. I think that if it can't use a key, it should raise an exception right away. -- resolution: not a bug -> status: closed -> open title: ssl server doesn't work with ECC certificates -> ssl should raise an exception when trying to load an unusable key (ECC key not using a named curve) ___ Python tracker <http://bugs.python.org/issue26254> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27084] Add dir_fd and follow_symlinks kwargs to os.listdir and os.scandir
New submission from Evgeny Kapun: Many functions in os module support dir_fd and follow_symlinks keyword arguments. I think that os.listdir and os.scandir should do likewise. See also: issue25996. -- components: Library (Lib) messages: 266091 nosy: abacabadabacaba priority: normal severity: normal status: open title: Add dir_fd and follow_symlinks kwargs to os.listdir and os.scandir type: enhancement versions: Python 3.6 ___ Python tracker <http://bugs.python.org/issue27084> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25996] Add support of file descriptor in os.scandir()
Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker <http://bugs.python.org/issue25996> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27085] Make it possible to select return type for os.listdir
New submission from Evgeny Kapun: Currently, os.listdir returns a list of strings, unless called with a bytes argument, in which case a list of byte strings is returned. I think that there should be a keyword argument to override this selection. -- components: Library (Lib) messages: 266093 nosy: abacabadabacaba priority: normal severity: normal status: open title: Make it possible to select return type for os.listdir type: enhancement versions: Python 3.6 ___ Python tracker <http://bugs.python.org/issue27085> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27086] Add closefd argument to os.listdir
New submission from Evgeny Kapun: Currently, when given a file descriptor, os.listdir will duplicate it so that the original file descriptor is not closed. In many cases, a file descriptor is not needed anymore after directory is listed, so this is not necessary. I propose adding a keyword argument closefd to os.listdir which, if set to True, will make os.listdir take ownership of the passed file descriptor and close it at the end. -- components: Library (Lib) messages: 266094 nosy: abacabadabacaba priority: normal severity: normal status: open title: Add closefd argument to os.listdir type: enhancement versions: Python 3.6 ___ Python tracker <http://bugs.python.org/issue27086> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27084] Add dir_fd and follow_symlinks kwargs to os.listdir and os.scandir
Evgeny Kapun added the comment: 1. Yes, it's possible to emulate dir_fd this way, so this is just for convenience. 2. If follow_symlinks is False, O_NOFOLLOW is passed to the underlying open(2) syscall. Of course, this doesn't make sense if a file descriptor is passed already. -- ___ Python tracker <http://bugs.python.org/issue27084> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27086] Add closefd argument to os.listdir
Evgeny Kapun added the comment: Not a problem, just two unnecessary syscalls. Also, I think that many of those who pass a file descriptor to os.listdir don't need it afterwards, because after you fstat() a file descriptor (to discover that it points to a directory) and read the directory contents, there are few things you can use it for. -- ___ Python tracker <http://bugs.python.org/issue27086> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27084] Add dir_fd and follow_symlinks kwargs to os.listdir and os.scandir
Evgeny Kapun added the comment: Mostly for consistency with other functions. Also, this provides an easy way to walk a directory tree recursively: just call listdir on every member, and if it doesn't raise OSError, that member must be a directory. With follow_symlinks=False, this method wouldn't follow symlinks, unlike os.walk and os.fwalk, which always follow symlinks at least to differentiate between symlinks to directories and other symlinks. -- ___ Python tracker <http://bugs.python.org/issue27084> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27085] Make it possible to select return type for os.listdir
Evgeny Kapun added the comment: Unfortunately, this doesn't work if I pass a file descriptor. -- resolution: rejected -> status: closed -> open ___ Python tracker <http://bugs.python.org/issue27085> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27085] Make it possible to select return type for os.listdir
Evgeny Kapun added the comment: Unfortunately, on Linux, handling names as Unicode can cause some problems. For example, merely print()-ing such a name can cause UnicodeEncodeError. -- ___ Python tracker <http://bugs.python.org/issue27085> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23689] Memory leak in Modules/sre_lib.h
New submission from Evgeny Kapun: In Modules/sre_lib.h on line 882 [1], a block of memory is allocated. If SRE(match) function later terminates abruptly, either because of a signal or because subsequent memory allocation fails, this block is never released. [1] https://hg.python.org/cpython/file/c89f7c34e356/Modules/sre_lib.h#l882 -- components: Regular Expressions messages: 238313 nosy: abacabadabacaba, ezio.melotti, mrabarnett priority: normal severity: normal status: open title: Memory leak in Modules/sre_lib.h type: resource usage versions: Python 3.4 ___ Python tracker <http://bugs.python.org/issue23689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23690] re functions never release GIL
New submission from Evgeny Kapun: Looks like function in re module (match, fullmatch and so on) don't release GIL, even though these operations can take much time. As a result, other threads can't run while a pattern is being matched, and thread switching doesn't happen as well. -- components: Regular Expressions messages: 238316 nosy: abacabadabacaba, ezio.melotti, mrabarnett priority: normal severity: normal status: open title: re functions never release GIL type: resource usage versions: Python 3.4 ___ Python tracker <http://bugs.python.org/issue23690> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23691] re.finditer iterator is not reentrant, but doesn't protect against nested calls to __next__
New submission from Evgeny Kapun: Iterator returned by re.finditer includes a SRE_STATE value, which is not designed to be used concurrently. However, it is possible to call __next__ on such iterator while another such call is in progress, e.g. from a signal handler. This may result in corruption of SRE_STATE structure. -- components: Regular Expressions messages: 238323 nosy: abacabadabacaba, ezio.melotti, mrabarnett priority: normal severity: normal status: open title: re.finditer iterator is not reentrant, but doesn't protect against nested calls to __next__ type: crash versions: Python 3.4 ___ Python tracker <http://bugs.python.org/issue23691> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23689] Memory leak in Modules/sre_lib.h
Evgeny Kapun added the comment: Memory leak only happens if match operation terminates abruptly, e.g. because of SIGINT. In this case, DO_JUMP doesn't come back. -- ___ Python tracker <http://bugs.python.org/issue23689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23690] re functions never release GIL
Evgeny Kapun added the comment: Aren't Python strings immutable? Also, match functions still permit execution of signal handlers, which can execute any Python code. If GIL is needed during matching, can it be released temporarily to permit thread switching? -- ___ Python tracker <http://bugs.python.org/issue23690> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23689] Memory leak in Modules/sre_lib.h
Evgeny Kapun added the comment:
Tracemalloc code:
import re
import signal
import tracemalloc
class AlarmError(Exception):
pass
def handle_alarm(signal, frame):
raise AlarmError
signal.signal(signal.SIGALRM, handle_alarm)
s1 = tracemalloc.take_snapshot()
for _ in range(20):
try:
signal.alarm(1)
re.match('(?:a|a|(?=b)){1000}', 'a'*999)
raise RuntimeError
except AlarmError:
pass
s2 = tracemalloc.take_snapshot()
res = s2.compare_to(s1, 'lineno')
for e in res[:10]:
print(e)
For me, it shows almost 3 MiB allocated in re.py.
--
___
Python tracker
<http://bugs.python.org/issue23689>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue433030] SRE: Atomic Grouping (?>...) is not supported
Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker <http://bugs.python.org/issue433030> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9134] sre bug: lastmark_save/restore
Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker <http://bugs.python.org/issue9134> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23692] Undocumented feature prevents re module from finding certain matches
New submission from Evgeny Kapun:
This pattern matches:
re.match('(?:()|(?(1)()|z)){2}(?(2)a|z)', 'a')
But this doesn't:
re.match('(?:()|(?(1)()|z)){0,2}(?(2)a|z)', 'a')
The difference is that {2} is replaced by {0,2}. This shouldn't prevent the
pattern from matching anywhere where it matched before.
The reason for this misbehavior is a feature which is designed to protect re
engine from infinite loops, but in fact it sometimes prevents patterns from
matching where they should. I think that this feature should be at least
properly documented, by properly I mean that it should be possible to
reconstruct the exact behavior from documentation, as the implementation is not
particularly easy to understand.
--
components: Regular Expressions
messages: 238330
nosy: abacabadabacaba, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: Undocumented feature prevents re module from finding certain matches
type: behavior
versions: Python 3.4
___
Python tracker
<http://bugs.python.org/issue23692>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23689] Memory leak in Modules/sre_lib.h
Evgeny Kapun added the comment: This patch doesn't fix the issue. The problem is that the list starting with state->repeat doesn't necessarily contains all repeat contexts that are allocated. Indeed, here [1] and here [2] repeat contexts are temporarily removed from the list. If the match procedure terminates abruptly, they are not added back. [1] https://hg.python.org/cpython/file/c89f7c34e356/Modules/sre_lib.h#l963 [2] https://hg.python.org/cpython/file/c89f7c34e356/Modules/sre_lib.h#l1002 -- ___ Python tracker <http://bugs.python.org/issue23689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2636] Adding a new regex module (compatible with re)
Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker <http://bugs.python.org/issue2636> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22672] float arguments in scientific notation not supported by argparse
Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker <http://bugs.python.org/issue22672> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24260] TabError behavior doesn't match documentation
New submission from Evgeny Kapun: In the documentation, it is said: Indentation is rejected as inconsistent if a source file mixes tabs and spaces in a way that makes the meaning dependent on the worth of a tab in spaces; a TabError is raised in that case. But that's not true. For example, Python thinks that these two indentations are "consistent": <8 spaces> <8 spaces> However, their width would be different for any tab width except 1, 2, 4, and 8. Actually, it's not easy to check that indentation is "consistent" as it is defined currently, so I think that it is the documentation that should be changed. So, I think that the paragraph that I quoted above should be changed to match the actual behavior. -- components: Interpreter Core messages: 243794 nosy: abacabadabacaba priority: normal severity: normal status: open title: TabError behavior doesn't match documentation type: behavior versions: Python 3.4 ___ Python tracker <http://bugs.python.org/issue24260> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24261] Add a command line flag to suppress default signal handlers
New submission from Evgeny Kapun: Currently, Python always changes handling of certain signals on startup: SIGPIPE is ignored, and SIGINT is handled by a function that raises KeyboardInterrupt exception. As a result, if the user presses Ctrl-C, a backtrace is printed to stderr. Some program may want to change this behavior. For example, they may want to print a user-friendly message, or to terminate without printing anything. They may do that by two means: either by installing their own handler for SIGINT, or by using a try..except block to handle the KeyboardInterrupt exception. However, they both have a flaw: if SIGINT is received during startup, before the program begins execution, the ugly KeyboardInterrupt backtrace is printed nevertheless. So, I propose to add a way to stop Python from installing this default SIGINT handler. I think that the most obvious way is to add a command line switch. This switch should prevent Python from installing its SIGINT handler. It may also prevent it from ignoring SIGPIPE, so that all signal actions would stay the same as when the program was executed. After that, programs may use signal module to set up their own handlers, if that is desired. -- components: Interpreter Core messages: 243795 nosy: abacabadabacaba priority: normal severity: normal status: open title: Add a command line flag to suppress default signal handlers type: enhancement ___ Python tracker <http://bugs.python.org/issue24261> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14228] Don't display traceback when import site is interrupted by CTRL+c
Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker <http://bugs.python.org/issue14228> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24260] TabError behavior doesn't match documentation
Evgeny Kapun added the comment: Prohibiting tabs after spaces is not enough. For example, Python rejects this code: if 1: if 1: pass because its indentation is invalid if tab width is 1. However, it accepts this code: if 1: if 1: <10 spaces>pass despite its indentation being invalid if tab width is 10 or more. -- ___ Python tracker <http://bugs.python.org/issue24260> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28436] GzipFile doesn't properly handle short reads and writes on the underlying stream
New submission from Evgeny Kapun:
GzipFile's underlying stream can be a raw stream (such as FileIO), and such
streams can return short reads and writes at any time (e.g. due to signals).
The correct behavior in case of short read or write is to retry the call to
read or write the remaining data.
GzipFile doesn't do this. This program demonstrates the problem with reading:
import io, gzip
class MyFileIO(io.FileIO):
def read(self, n):
# Emulate short read
return super().read(1)
raw = MyFileIO('test.gz', 'rb')
gzf = gzip.open(raw, 'rb')
gzf.read()
Output:
$ gzip -c /dev/null > test.gz
$ python3 test.py
Traceback (most recent call last):
File "test.py", line 10, in
gzf.read()
File "/usr/lib/python3.5/gzip.py", line 274, in read
return self._buffer.read(size)
File "/usr/lib/python3.5/gzip.py", line 461, in read
if not self._read_gzip_header():
File "/usr/lib/python3.5/gzip.py", line 409, in _read_gzip_header
raise OSError('Not a gzipped file (%r)' % magic)
OSError: Not a gzipped file (b'\x1f')
And this shows the problem with writing:
import io, gzip
class MyIO(io.RawIOBase):
def write(self, data):
print(data)
# Emulate short write
return 1
raw = MyIO()
gzf = gzip.open(raw, 'wb')
gzf.close()
Output:
$ python3 test.py
b'\x1f\x8b'
b'\x08'
b'\x00'
b'\xb9\xea\xffW'
b'\x02'
b'\xff'
b'\x03\x00'
b'\x00\x00\x00\x00'
b'\x00\x00\x00\x00'
It can be seen that there is no attempt to write all the data. Indeed, the
return value of write() method is completely ignored.
I think that either gzip module should be changed to handle short reads and
writes properly, or its documentation should reflect the fact that it cannot be
used with raw streams.
--
components: Library (Lib)
messages: 278606
nosy: abacabadabacaba
priority: normal
severity: normal
status: open
title: GzipFile doesn't properly handle short reads and writes on the
underlying stream
type: behavior
versions: Python 3.5
___
Python tracker
<http://bugs.python.org/issue28436>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15994] memoryview to freed memory can cause segfault
Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker <http://bugs.python.org/issue15994> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13322] buffered read() and write() does not raise BlockingIOError
Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker <http://bugs.python.org/issue13322> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28445] Wrong documentation for GzipFile.peek
New submission from Evgeny Kapun: >From the documentation for GzipFile.peek(): At most one single read on the compressed stream is done to satisfy the call. If "compressed stream" means the underlying file object, then this is not true. The method tries to return at least one byte, unless the stream is at EOF. It is possible to create arbitrarily long compressed stream that would decompress to nothing, and the implementation would read the entire stream in this case. Because the length of the stream is not known in advance, several reads may be required for this. Perhaps the documentation for GzipFile.peek() should be made the same as that for BZ2File.peek() and LZMAFile.peek(). -- assignee: docs@python components: Documentation messages: 278656 nosy: abacabadabacaba, docs@python priority: normal severity: normal status: open title: Wrong documentation for GzipFile.peek versions: Python 3.5 ___ Python tracker <http://bugs.python.org/issue28445> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
