Changes by Nathaniel Smith :
--
resolution: -> fixed
stage: -> resolved
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue30038>
___
___
Nathaniel Smith added the comment:
Then maybe simplest solution is to scale back the claim :-).
The important semantic change would be that right now, interrupt_main() is
documented to cause KeyboardInterrupt to be raised in the main thread. So if
you register a custom SIGINT handler that
Changes by Nathaniel Smith :
--
nosy: +njs
___
Python tracker
<http://bugs.python.org/issue16487>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by Nathaniel Smith :
--
nosy: +njs
___
Python tracker
<http://bugs.python.org/issue14243>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from Nathaniel Smith:
Sometimes, CPython's signal handler (signalmodule.c:signal_handler) runs in a
different thread from the main thread. On Unix this is rare but it does happen
(esp. for SIGCHLD), and on Windows it happens 100% of the time. It turns out
that there is a s
Nathaniel Smith added the comment:
On further investigation (= a few hours staring at the ceiling last night), it
looks like there's another explanation for my particular bug... which is good,
because on further investigation (= a few hours squinting at google results) it
looks like
Nathaniel Smith added the comment:
@arigo: Technically we also need that the writes to memory are observed to
happen-before the write() call on the wakeup fd, which is not something that
Intel is going to make any guarantees about. But *probably* this is also safe
because the kernel has to
Nathaniel Smith added the comment:
> I haven't dug in deeply, but it sounds like we handle IDNs in CNs and SANs
> differently?
No -- Python's ssl module uses exactly the same hostname checking logic in both
cases, and it's equally broken regardless. But, since CAs do all
Changes by Nathaniel Smith :
--
pull_requests: +3043
___
Python tracker
<http://bugs.python.org/issue28414>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from Nathaniel Smith:
socket.getaddrinfo accepts None as a port argument, and translates it into 0.
This is handy, because bind() understands 0 to mean "pick a port for me", and
if you want bind to pick a port for you and port=None is a slightly more
obvious way t
Changes by Nathaniel Smith :
--
nosy: +giampaolo.rodola
___
Python tracker
<http://bugs.python.org/issue31198>
___
___
Python-bugs-list mailing list
Unsubscribe:
Nathaniel Smith added the comment:
Ugh, apparently this weird behavior is actually mandated by the RFC :-(.
RFC 3493:
The nodename and servname arguments are either null pointers or
pointers to null-terminated strings. One or both of these two
arguments must be a non-null pointer
Nathaniel Smith added the comment:
Was this actually fixed, or did everyone just get tired and give up on the
original patch?
--
nosy: +njs
___
Python tracker
<http://bugs.python.org/issue11
Nathaniel Smith added the comment:
This would still provide value even if you have to do a GET_AWAITABLE in the
protected region: the most common case is that __aenter__ is a coroutine
function, which means that its __await__ is implemented in C and already
protected against interrupts.
Also
Nathaniel Smith added the comment:
In bpo-30703 Antoine fixed signal handling so it doesn't use Py_AddPendingCall
anymore. At this point the only time the interpreter itself uses
Py_AddPendingCall is when there's an error writing to the signal_fd, which
should never happen in no
Changes by Nathaniel Smith :
--
nosy: +njs
___
Python tracker
<http://bugs.python.org/issue29302>
___
___
Python-bugs-list mailing list
Unsubscribe:
Nathaniel Smith added the comment:
Adding Ned to CC in case he wants to comment on the utility of per-opcode
tracing from the perspective of coverage.py.
--
nosy: +nedbat, njs
___
Python tracker
<http://bugs.python.org/issue31
Nathaniel Smith added the comment:
My reading of the man page is that if SSL_shutdown returns 0, this means that
it succeeded at doing the first phase of shutdown. If there are errors then
they should be ignored, because it actually succeeded.
If you want to then complete the second phase of
Nathaniel Smith added the comment:
For purposes of writing a test case, can you install a custom Python-level
signal handler, and make some assertion about where it runs? I.e., the calling
frame should be inside the __aexit__ body, not anywhere earlier
Nathaniel Smith added the comment:
To make sure I understand correctly: your concern is that the event loop is not
implemented in C. So if you have this patch + an async CM that's implemented in
C + the async CM never *actually* yields to the event loop, then that will be
signal safe..
Nathaniel Smith added the comment:
FWIW trio's strategy for handling this is to install a clever signal handle
that routes the signal to the event loop IF the signal arrives while the event
loop is running, or while particularly sensitive code like trio.Lock.__aexit__
is running. The re
Nathaniel Smith added the comment:
(tl;dr: this patch is more awesome than you realize, thanks for working on it
:-))
--
___
Python tracker
<https://bugs.python.org/issue29
Nathaniel Smith added the comment:
Some prior discussion on the old asyncio tracker:
https://github.com/python/asyncio/pull/305#issuecomment-168714572
https://github.com/python/asyncio/issues/341
--
___
Python tracker
<ht
Nathaniel Smith added the comment:
Should I open a new issue for the __aexit__ part of this issue then, or...? The
reason neither asyncio nor trio use the approach you describe is that it makes
it impossible to break out of infinite loops, and trio in particular has
entirely solved this
Nathaniel Smith added the comment:
Note also that adding a check to ceval is not sufficient -- there are also lots
of calls to PyErr_CheckSignals scattered around the tree, e.g. next to every
syscall that can return EINTR.
I've considered proposing something like this in the past, sin
Nathaniel Smith added the comment:
On another topic: you should add a test where the 'with' block exits by raising
an exception, because in this case the interpreter jumps straight to
WITH_CLEANUP_START, potentially skipping the DEFER_PENDING_UNTI
Nathaniel Smith added the comment:
> a special case for WITH_CLEANUP_START
I guess this works for the sync case, but it seems kinda fragile, and obviously
won't work for the async case. Do you think it's worth revisiting this idea
from the OP?: "The eval loop's &qu
Nathaniel Smith added the comment:
Yes... low-level routines like coroutine schedulers where you need to defer
interrupts inside the scheduler but not inside the coroutines being scheduled.
It's not gold plating, it's actually the original motivation :-)
https://vorpus.org/blog
Nathaniel Smith added the comment:
... it's true I did write such an example. And a few lines up in the same
message I wrote an example where I was protecting an event loop from
interrupts. In both cases the tricky question is how to manage the transitions
between protected and unprot
Nathaniel Smith added the comment:
Here's the patch I mentioned:
https://github.com/njsmith/cpython/commit/62547dc5ea323a07c25c2047636a02241f518013
It has a crude but effective technique for handling the hanging issue :-).
Alternatively one could use the World's Stupidest Corout
Nathaniel Smith added the comment:
Sure, but whatever overhead it has, it has. Once we're paying for it, new keys
are free (at yield/resume time).
Compared to a bare thread-local it probably has somewhat higher overhead when
we have to check it, but (a) that's why PEP 550 has a clev
Nathaniel Smith added the comment:
How does a trace function or debugger tell the difference between a closure
cell and a fast local whose value happens to be a cell object?
--
___
Python tracker
<https://bugs.python.org/issue30
Nathaniel Smith added the comment:
Doesn't this proposal break every debugger, including pdb?
--
___
Python tracker
<https://bugs.python.org/issue30744>
___
___
Nathaniel Smith added the comment:
Would it be more acceptable to use a DeprecationWarning? It's not really the
right thing because we're not planning to ever actually remove the
functionality. But, we've already gone to a lot of trouble to try to show
DeprecationWarnings
Nathaniel Smith added the comment:
> In my experience people are more likely to run code through a linter than
> they are to ever run an interpreter with DeprecationWarning enabled.
This used to be true, and it was a disaster. So there's been a lot of work to
fix it, and it
Nathaniel Smith added the comment:
> I'm thinking of submitting a talk at PyCon to explain what we've discover so
> far in IPython.
You totally should!
Or actually there are two options to think about: you can submit a general
talk, or submit a talk to the language summi
Nathaniel Smith added the comment:
It's confusing, but AFAICT what happened is that Mozilla started to disable it
in release builds, but got a bunch of pushback from users and changed their
minds and decided to keep it enabled. But then there was a snafu tracking the
patch for tha
New submission from Nathaniel Smith :
Suppose we want to test how a program responds to control-C. We'll want to
write a test that delivers a SIGINT to our process at a time of our choosing,
and then checks what happens. For example, asyncio and Trio both have tests
like this, and Trio
Nathaniel Smith added the comment:
It probably doesn't matter too much either way, but almost all the
signal-related wrappers are in signal. os.kill is the only exception AFAIK.
--
___
Python tracker
<https://bugs.python.org/is
Nathaniel Smith added the comment:
Vladimir Matveev pointed out that there's already a wrapper in
_testcapimodule.c:
https://github.com/python/cpython/blob/f06fba5965b4265c42291d10454f387b76111f26/Modules/_testcapimodule.c#L3862-L3879
So if we do add this to signalmodule.c, we can drop
Nathaniel Smith added the comment:
I'd like to see it in 3.8, but don't know if I'll get to it, and it'd be a good
onramp issue for someone who wants to get into cpython development. So, let's
put the keyword on it for now, and see what happens...
Nathaniel Smith added the comment:
Am I right that this is considered fixed (or at least as fixed as it can be),
and the issue should be closed?
Also, small note in case anyone stumbles across this in the future and is
confused: Python does *not* handle this correctly on Windows. I suspect
New submission from Nathaniel Smith :
~$ unzip -l /tmp/python-3.7.2-embed-amd64.zip
Archive: /tmp/python-3.7.2-embed-amd64.zip
Length DateTimeName
- -- -
99856 2018-12-23 23:10 python.exe
98320 2018-12-23 23:10 pythonw.exe
3780624
Nathaniel Smith added the comment:
You're right, good catch!
--
resolution: duplicate -> fixed
stage: resolved ->
superseder: Fatal Python error: initfsencoding: unable to load the file system
codec zipimport.ZipImportError: can't find m
New submission from Nathaniel Smith :
Travis provides a "3.8-dev" python, which is updated regularly to track cpython
master. When running our tests on this Python, specifically version
python: 3.8.0a0 (heads/master:f75d59e, Feb 3 2019, 07:27:24)
we just started getting
Nathaniel Smith added the comment:
Oh, that's not my code, it's the core of IPython's REPL :-). I just filed a bug
with them, referencing this one: https://github.com/ipython/ipython/issues/11590
--
___
Python tracker
<https
New submission from Nathaniel Smith:
https://github.com/python/cpython/pull/477
--
components: Library (Lib)
messages: 289041
nosy: njs
priority: normal
severity: normal
status: open
title: Expose TCP_NOTSENT_LOWAT
___
Python tracker
<h
Changes by Nathaniel Smith :
--
pull_requests: +396
___
Python tracker
<http://bugs.python.org/issue29728>
___
___
Python-bugs-list mailing list
Unsubscribe:
Nathaniel Smith added the comment:
I think ideally on numpy's end we would reseed iff the RNG was unseeded. Now
that I think about it I'm a little surprised that we haven't had more
complaints about this, so I guess it's not a super urgent issue, but that would
be an
Nathaniel Smith added the comment:
It turns out that this bug is more general than signal.pause, and has caused
problems for a few different people:
https://github.com/dabeaz/curio/issues/118#issuecomment-287735781
https://github.com/dabeaz/curio/issues/118#issuecomment-287798241
https
Nathaniel Smith added the comment:
I don't really have a specific use case personally -- for trio, I haven't found
a way to make use of set_wakeup_fd because of various issues[1], but I'm also
not planning to use SIGCHLD, so this isn't very urgent.
In general set_wakeup_fd
Nathaniel Smith added the comment:
@haypo: It's a socketpair. It works fine when I set up a toy test case using
set_wakeup_fd + select, and it works fine in my real code when I use CFFI
cleverness to register a signal handler that manually writes a byte to my
wakeup socket, but when I
Nathaniel Smith added the comment:
@haypo: okay, looked over things over for a third time and this time I found my
very silly error :-). So I'm now able to use set_wakeup_fd on Windows
(https://github.com/python-trio/trio/pull/108), but not on Unix
(https://github.com/python-trio/trio/i
Nathaniel Smith added the comment:
Letting Python-level signal handlers run in arbitrary threads is an interesting
idea, but it's a non-trivial change to Python semantics that may well break
some programs (previously it was guaranteed that signal handlers couldn't race
with main t
Nathaniel Smith added the comment:
If you want to trigger the standard signal handling logic, then raise(SIGINT)
is also an option. On unix it probably won't help because of issue 21895, but
using raise() here + fixing 21895 by using pthread_kill in the c level signal
handler would tog
Nathaniel Smith added the comment:
(oh, in case it wasn't obvious: the advantage of raise() over kill() and
pthread_kill() is that raise() works everywhere, including Windows, so it would
avoid platform specific logic. Or if you don't like raise() for some reason
then you can ge
New submission from Nathaniel Smith:
In the process of fixing issue 27867, a new function PySlice_AdjustIndices was
added, and PySlice_GetIndicesEx was converted into a macro that calls this new
function. The patch was backported to both the 3.5 and 3.6 branches, was
released in 3.6.1, and is
Nathaniel Smith added the comment:
It does make sense to skip emitting a warning if there's no try or with block
active.
Don't we already have the ability to check for this, though, without any
extensions to the frame or code objects? That's what the public
PyGen_NeedsFinaliz
New submission from Nathaniel Smith:
You might hope the interpreter would enforce the invariant that for 'with' and
'async with' blocks, either '__(a)enter__' and '__(a)exit__' are both called,
or else neither of them is called. But it turns out that
Nathaniel Smith added the comment:
> Can we consider 3.6.0 rather than 3.6.1 as broken release?
In the last week, pypi downloads were about evenly split between 3.6.0 and
3.6.1 (2269969 for "3.6.1", 1927189 for "3.6.0", and those two were ~2 orders
of magnitude more co
Nathaniel Smith added the comment:
It looks like PyTorch got bitten by this:
https://discuss.pytorch.org/t/pyslice-adjustindices-error/1475/11
--
___
Python tracker
<http://bugs.python.org/issue29
New submission from Nathaniel Smith:
In trip_signal [1], the logic goes:
1) set the flag saying that this particular signal was tripped
2) write to the wakeup fd
3) set the global is_tripped flag saying "at least one signal was tripped", and
do Py_AddPendingCall (which sets some gl
New submission from Nathaniel Smith:
If we have a chain of generators/coroutines that are 'yield from'ing each
other, then resuming the stack works like:
- call send() on the outermost generator
- this enters _PyEval_EvalFrameDefault, which re-executes the YIELD_FROM opcode
- which
Changes by Nathaniel Smith :
--
pull_requests: +1224
___
Python tracker
<http://bugs.python.org/issue30039>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by Nathaniel Smith :
--
pull_requests: +1226
___
Python tracker
<http://bugs.python.org/issue30038>
___
___
Python-bugs-list mailing list
Unsubscribe:
Nathaniel Smith added the comment:
Right. My claim would be that the PR I just submitted is the correct fix for
bpo-21645 as well.
The approach asyncio uses is very elegant, but unfortunately it assumes that
the wakeup fd has infinite buffer, which isn't true. If enough signals or
Nathaniel Smith added the comment:
Err, libuv obviously doesn't use a Python-level signal handler. I just meant to
include them as another example of a library I checked that uses a self-pipe to
handle signals but relies on out-of-band information to transmit what the
actual sign
Nathaniel Smith added the comment:
I think the idea in c13ef6664998 wasn't so much that we wanted the wakeup fd to
be written to first, as that the way the code was written back then, the
presence of 'if (is_tripped) return;' meant that it wasn't getting written to
*a
Nathaniel Smith added the comment:
If it helps, notice that the SetEvent(sigint_event) call used to wake up the
main thread on windows is also performed unconditionally and after the call to
Py_AddPendingEvent. From the point of view of twisted/tornado/trio, this is
exactly the same as the
Nathaniel Smith added the comment:
The attached script wakeup-fd-racer.py fails consistently for me using cpython
3.6.0 on my windows 10 vm:
> python wakeup-fd-racer.py
Attempt 0: start
Attempt 0: FAILED, took 10.0160076 seconds
select_calls = 2
(It may help that the VM only has 1
New submission from Nathaniel Smith:
When a wakeup fd is registered via signal.set_wakeup_fd, then the C level
signal handler writes a byte to the wakeup fd on each signal received. If this
write fails, then it prints an error message to the console.
Some projects use the wakeup fd as a way
Nathaniel Smith added the comment:
I haven't noticed the error in the wild because I don't use set_wakeup_fd on
Linux/MacOS, because of this issue :-). But on MacOS literally all it would
take is to receive two signals in quick succession, or to receive one signal at
a moment when s
Nathaniel Smith added the comment:
FYI: https://github.com/pandas-dev/pandas/pull/16066
--
___
Python tracker
<http://bugs.python.org/issue29943>
___
___
Pytho
New submission from Nathaniel Smith:
Basically what it says in the title... if you create an SSL object via
wrap_socket with do_handshake_on_connect=False, or via wrap_bio, and then
forget to call do_handshake and just go straight to sending and receiving data,
then the encrypted connection
Changes by Nathaniel Smith :
--
title: If you forget to call do_handshake, then everything seems to work but
hostname is disabled -> If you forget to call do_handshake, then everything
seems to work but hostname checking is disabled
___
Pyt
New submission from Nathaniel Smith:
As pointed out in this stackoverflow answer:
http://stackoverflow.com/a/43578450/
and since I seem to be collecting signal-handling bugs these days :-), there's
a race condition in how the interpreter uses _PyOS_SigintEvent to allow
control-C to brea
Nathaniel Smith added the comment:
Oh, I should also say that this isn't actually affecting me, I just figured
that once I was aware of the bug it was worth making a record here. Might be a
good starter bug for someone trying to get into CPython's
Nathaniel Smith added the comment:
Apparently this also broke pyqt for multiple users; here's the maintainers at
conda-forge struggling to figure out the best workaround:
https://github.com/conda-forge/pyqt-feedstock/pull/25
I really think it would be good to fix this in 3.6 sooner r
Nathaniel Smith added the comment:
Pillow also had broken wheels up on pypi for a while; they've now put out a bug
fix release that #undef's PySlice_GetIndicesEx, basically monkeypatching out
the bugfix to get back to the 3.6.0 behavior:
https://github.com/python-pillow/Pillow/i
Nathaniel Smith added the comment:
More collateral damage: apparently the workaround that Pandas used for this bug
(#undef'ing PySlice_GetIndicesEx) broke PyPy, so now they need a workaround for
the workaround: https://github.com/pandas-dev/pandas/pull/16192
Recording this here partly
Nathaniel Smith added the comment:
I don't find it helpful to think of it as declaring 3.6.0 broken vs declaring
3.6.1 broken. 3.6.0 is definitely good in the sense that if you build a module
on it then it will import on both 3.6.0 and 3.6.1, and 3.6.1 is definitely good
in the sense th
Nathaniel Smith added the comment:
Looks interesting! What's the advantage over running the server and the test in
the same loop? The ability to use blocking operations in the tests, and to
re-use an expensive-to-start server over multiple tests? (I've mostly used
threads in te
Nathaniel Smith added the comment:
@Jonathan: Even 3.6.1 was careful to retain compatibility with code built by
3.6.0. And your proposed 3.6.1-patched will generate binaries equivalent to the
ones 3.6.0 generates. So I don't think you need to worry; 3.6.2 is not going to
add a new and
New submission from Nathaniel Smith:
sphinxcontrib-trio [1] does a few things; one of them is to enhance sphinx's
autodoc support by trying to sniff out the types of functions so that it can
automatically determine that something is, say, a generator, or an async
classmethod.
This runs
Nathaniel Smith added the comment:
@Dima: are you volunteering to fix and maintain it? I can see why it's useful
to have some way to get at the fpu flags, but I don't see how fpectl
specifically helps with that issue, and fpectl has always been broken
Nathaniel Smith added the comment:
Also fixing the abi issues that started this, and probably making an argument
for why it makes sense for all of cpython's built-in float operations to check
the fpu flags, and to do so using a weird longjmp-based mechanism that only
some platforms su
Nathaniel Smith added the comment:
Another option you might want to consider is proposing to add a proper fpu
control flag setting/checking API to the math module.
--
___
Python tracker
<http://bugs.python.org/issue29
Nathaniel Smith added the comment:
> While I suggest you to *not* use an event loop (wakeup fd pipe/socket handle
> with select) and signal.signal(), you are true that there is a race condition
> if you use select() with signal.signal() so I merged your change.
Unfortunately this is
Nathaniel Smith added the comment:
> (BTW do you happen to know any tricks to force CPython to do an immediate
> PyErr_CheckSignals on Windows?)
Never mind on this... it looks like calling repr() on any object is sufficient.
--
___
Python t
Nathaniel Smith added the comment:
@Dima:
> @njs: to point out that usefulness of this module is not just wishful
> thinking. I just used it to locate, up to the line in a Python extension
> module written in C, a bug in Sagemath (that has perhaps 20 FPU-using
> extensions, som
New submission from Nathaniel Smith:
The SSL_shutdown man page says that if it returns 0, and an SSL_ERROR_SYSCALL
is set, then SSL_ERROR_SYSCALL should be ignored - or at least I think that's
what it's trying to say. See the RETURN VALUES section. I think this means we
should only
Nathaniel Smith added the comment:
Oddly, I expected to run into this with my code using SSLObject in trio [1],
but if I connect to python.org:443 and then 'await
trio_ssl_stream.do_handshake(); trio_ssl_stream.getpeercert()' it works just
fine ... even though when I run the sslbugs
Nathaniel Smith added the comment:
I'd also like to make use of this in trio, as a way to get safer and less
complicated control-C handling without having to implement things in C.
(Exhaustive discussion:
https://vorpus.org/blog/control-c-handling-in-python-and-trio/)
@Nick: I under
Nathaniel Smith added the comment:
Right, fixing this bug alone can't make programs control-C safe in general. But
there exist techniques to make __enter__/__exit__ methods safe WRT control-C,
and if we fix this bug *and* apply those techniques then you can get some
meaningful guara
Nathaniel Smith added the comment:
Debian testing, x86-64, with:
Python 3.5.3rc1 (default, Jan 3 2017, 04:40:57)
[GCC 6.3.0 20161229] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>
Nathaniel Smith added the comment:
> If all you need is that with foo: pass guarantees that either both or neither
> of __enter__ and __exit__ are called, for C context managers, and only C
> context managers, then the fix is trivial.
It would be nice to have it for 'async with
Nathaniel Smith added the comment:
Certainly which frame is being executed is an implementation detail, and I can
see an argument from that that we shouldn't have a frame introspection API at
all... but we do have one, and it has some pretty important use cases, like
traceback pri
Nathaniel Smith added the comment:
> I'm not sure I understand how `f_func` would help to better handle Control-C
> in Trio. Nathaniel, could you please elaborate on that?
Sure. The issue is that I need to mark certain frames as "protected" from
KeyboardInterrupt, in
Nathaniel Smith added the comment:
On further thought, I think the way I'd write a test for this is:
(1) add a testing primitive that waits for N instructions and then injects a
SIGINT. Probably this would require tweaking the definition of
Py_MakePendingCalls like I described in my pre
Nathaniel Smith added the comment:
> Yes, whenever you touch frames you're disabling the JIT for the call site
> (and maybe for more call sites up the stack, idk). So it doesn't matter what
> you use, `f_func` or `f_locals`, the performance will suffer big time. Is
&g
101 - 200 of 488 matches
Mail list logo
