New submission from Nathaniel Smith :
cPickle.dump by default does not properly encode unicode characters
outside the BMP -- it throws away the high bits:
>>> cPickle.loads(cPickle.dumps(u"\U00012345"))
u'\u2345'
The problem is in dump, not load:
>>> pi
New submission from Nathaniel Smith :
Quoting from the docs for ssl.SSLContext:
"Changed in version 3.6: The context is created with secure default values."
- https://docs.python.org/3/library/ssl.html#ssl.SSLContext
This is not true. If you call ssl.SSLContext(), you get a context
Nathaniel Smith added the comment:
It's something I'm still interested in, but I'm not actively working on it (as
you noticed :-)), and there are some other CPython changes that I'll probably
prioritize first. Do you want to close this and I can re-open it when I do
Nathaniel Smith added the comment:
Yury asked me to weigh in here, since I guess between him and Andrew there's
some uncertainty about whether reverting is the right choice or not. I can't
answer that, but I can share some thoughts.
Unfortunately, I wasn't aware of the Stream
Nathaniel Smith added the comment:
That seems a bit extreme, and I don't think conflicts with Trio are the most
important motivation here. (And they shouldn't be.) But, we should probably
write down what the motivations actually are.
Yury might have a different perspective, but
Nathaniel Smith added the comment:
> I think we need something *like* asyncio.Stream in the mix, for compatibility
> and bridging between the two worlds, but right now it's not quite clear how
> that should look, and pushing it off to 3.9 would give us time to figure out
>
New submission from Nathaniel Smith :
Just noticed this while looking at the code to asyncio.Task.__step
asyncio Futures have two different error states: they can have an arbitrary
exception set, or they can be marked as cancelled.
asyncio Tasks handle this by detecting what exception was
Nathaniel Smith added the comment:
I saw Yury on Saturday, and we spent some time working through the implications
of different options here.
For context: the stream ABCs will be a bit different from most third-party
code, because their value is proportional to how many projects adopt them
Nathaniel Smith added the comment:
BTW Andrew, while writing that I realized that there's also overlap between
your new Server classes and Trio's ABC for servers, and I think it'd be
interesting to chat about how they compare maybe? But it's not relevant to this
issue
Nathaniel Smith added the comment:
> I hope that the Trio project can minimize the number of methods they want to
> add to those ABCs so that we don't need to duplicate a lot of functionality
> in asyncio.Stream. E.g. in the new asyncio.Stream there's a Stream.write()
&
Nathaniel Smith added the comment:
It's also possible to implement 'select' on top of IOCP using undocumented
private APIs. That's what libuv does (and how 'select' works internally).
There's a bunch of information on this collected here:
https://gi
Nathaniel Smith added the comment:
The proposal is to be able to run io module operations in two modes: the
regular one, and one where performing actual I/O is forbidden – so if they go
down the stack and can fulfill the operation from some in-memory buffer, great,
they do that, and if not
Nathaniel Smith added the comment:
> If you wanted to keep async disk access separate from the io module, then
> what we'd have to do is to create a fork of all the code in the io module,
> and add this feature to it.
Thinking about this again today, I realized there *mig
Nathaniel Smith added the comment:
I think conceptually, cancelling all tasks and waiting for them to exit is the
right thing to do. That way you run as much shutdown code as possible in
regular context, and also avoid this problem – if there are no tasks, then
there can't be any
Nathaniel Smith added the comment:
Yeah, that's the question. I've dug into the AFD_POLL stuff more now, and...
There's a lot of edge cases to think about. It certainly *can* be done, because
this is the primitive that 'select' and friends are built on, so obviously
Nathaniel Smith added the comment:
Thanks for the CC.
It would be nice to get `pidfd_send_signal` as well, while we're at it. But
that could be a separate PR.
AFAICT the other bits of the pidfd API right now are some extra flags to clone,
and an extra flag to waitid. The waitid fl
Nathaniel Smith added the comment:
hasattr is useful for supporting old versions of the os module, but asyncio
doesn't have to care about that. You should probably try calling pidfd_open and
see whether you get -ESYSCALL, and also maybe try passing it to poll(), since I
think there
Nathaniel Smith added the comment:
It sounds like there's actually nothing to do here? (Except maybe eventually
switch to pidfd or similar, but Victor says he wants to use a different issue
for that.) Can this be closed?
--
nosy: +njs
___
P
Nathaniel Smith added the comment:
But then deeper in the thread it sounded like you concluded that this didn't
actually help anything, which is what I would expect :-).
--
___
Python tracker
<https://bugs.python.org/is
Nathaniel Smith added the comment:
You can't solve a time-of-check-to-time-of-use race by adding another check. I
guess your patch might narrow the race window slightly, but it was a tiny
window before and it's a tiny window after, so I don't really get it.
The test doesn&
Nathaniel Smith added the comment:
Hmm, you know, on further thought, it is actually possible to close this race
for real, without pidfd.
What you do is split 'wait' into two parts: first it waits for me process to
become reapable without actually reaping it. On Linux you can do
Nathaniel Smith added the comment:
I guess a bikeshed question is whether it should be `os.pidfd_send_signal`
(like `os.kill`) or `signal.pidfd_send_signal` (like `signal.pthread_kill`).
I don't actually care either way myself :-)
--
___
P
Nathaniel Smith added the comment:
I don't know :-( After filing this upstream we started ignoring these
exceptions, to make our CI less flaky:
https://github.com/python-trio/trio/pull/365/files
But unfortunately that means I don't know if we've been hitting them sin
Nathaniel Smith added the comment:
Funny, I was actually looking at this a bit last week, because I was trying to
figure out if I could trick `ssl` into doing DTLS...
The two big problems I ran into are:
- for DTLS you need to instantiate the SSLContext with PROTOCOL_DTLS, and idk
how you
Nathaniel Smith added the comment:
Python's 'id' function exposes raw memory addresses constantly. As long as
they're just integers, they can't do much harm.
(In Rust, taking a pointer to a random object is considered totally safe, can
be done anywhere. It's *
Nathaniel Smith added the comment:
@types.coroutine is still useful as a tool for writing "foundation" coroutines
that directly interact with the coroutine runner, e.g.:
https://github.com/python-trio/trio/blob/94562c54d241859eb05ed85c88fd6073d6752ff6/trio/_core/_traps.py#L12-
Change by Nathaniel Smith :
--
Removed message: https://bugs.python.org/msg392594
___
Python tracker
<https://bugs.python.org/issue28708>
___
___
Python-bug
Change by Nathaniel Smith :
--
Removed message: https://bugs.python.org/msg392595
___
Python tracker
<https://bugs.python.org/issue28708>
___
___
Python-bug
Change by Nathaniel Smith :
--
Removed message: https://bugs.python.org/msg390813
___
Python tracker
<https://bugs.python.org/issue28708>
___
___
Python-bug
Change by Nathaniel Smith :
--
nosy: +njs
___
Python tracker
<https://bugs.python.org/issue42514>
___
___
Python-bugs-list mailing list
Unsubscribe:
Nathaniel Smith added the comment:
I think this is a dupe of issue41271?
--
___
Python tracker
<https://bugs.python.org/issue44738>
___
___
Python-bugs-list m
New submission from Nathaniel Smith :
Inside 'except' and 'finally' blocks, the interpreter keeps track of the
'active exception in the thread-state. It can be introspected via
`sys.exc_info()`, it enables bare `raise`, and it triggers implicit context
propa
Nathaniel Smith added the comment:
Ugh, the gnarly ExitStack bug is bpo-44594, not whatever I wrote above
--
___
Python tracker
<https://bugs.python.org/issue45
Nathaniel Smith added the comment:
New changeset e6d1aa1ac65b6908fdea2c70ec3aa8c4f1dffcb5 by John Belmonte in
branch 'main':
bpo-44594: fix (Async)ExitStack handling of __context__ (gh-27089)
https://github.com/python/cpython/commit/e6d1aa1ac65b6908fdea2c70ec3aa8
Change by Nathaniel Smith :
--
resolution: -> duplicate
stage: -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Nathaniel Smith added the comment:
> I saw the article as well, but think auto-sorting would have just provided a
> thin mask over their serious data pipeline bugs.
This seems like an inappropriately elitist attitude. I'm sure their code has
bugs; so does mine and yours. But in fa
Nathaniel Smith added the comment:
Ouch, that's nasty. It also has security implications. For example, suppose you
have a multi-user computer, where one user is has a video call going, which
transfers packets over UDP. Another user could check what port they're using,
bind to the
Nathaniel Smith added the comment:
> Now your updated docs and warning read more like we are working around a
> Linux security bug which is not really the case - this behavior was
> intentionally added to the kernels and some of the code I do for a living
> relies on it to w
Nathaniel Smith added the comment:
I don't know about podman, but it sounds like mock and docker both use buggy
sandboxing: https://bugzilla.redhat.com/show_bug.cgi?id=1770154
--
___
Python tracker
<https://bugs.python.org/is
Nathaniel Smith added the comment:
I was assuming we'd only do this on Linux, since that's where the bug is...
though now that you mention it the Windows behavior is probably wonky too.
SO_REUSEADDR and SO_REUSEPORT have different semantics on all three of
Windows/BSD/Linux. A hi
Nathaniel Smith added the comment:
There's another important use case for this, that hasn't been discussed here.
If you want to use openssl for TLS + the system trust store to verify
certificates, then you need to disable openssl's certificate validation,
perform the hand
Nathaniel Smith added the comment:
Trio uses 65535 as the default backlog argument. There's a big comment here
explaining why:
https://github.com/python-trio/trio/blob/master/trio/_highlevel_open_tcp_listeners.py
This doesn't actually set the backlog to 65635; instead it tells the
Nathaniel Smith added the comment:
Another subtlety that that code handles, and that the stdlib socket module
might also want to handle: if the user passes in a custom backlog argument
that's >65535, then we silently replace it with 66535 before passing it to the
system. The reason
Nathaniel Smith added the comment:
> Thread B thinks the process is still running, so it calls waitid+WNOHANG on
> a stale PID, with unpredictable results.
I'm pretty sure you mean WNOWAIT, right? The actual reaping step (which might
use WNOHANG) is already protected by a lock,
Nathaniel Smith added the comment:
Yeah, I don't think it matters that much. There are lots of random gotchas that
you have to watch out for using the socket module.
To be clear: I do still think that using a large value by default, and clamping
user input values at 65535, wou
Nathaniel Smith added the comment:
> revalidate pid licenses
It means autocorrect mangled the text... I don't remember what word that's
supposed to be, but basically I meant "revalidate the pid hasn't been reaped"
> Wouldn't it be sufficient t
Change by Nathaniel Smith :
--
nosy: +njs
___
Python tracker
<https://bugs.python.org/issue39148>
___
___
Python-bugs-list mailing list
Unsubscribe:
Nathaniel Smith added the comment:
> For the concern issue, as I understand it, the ability to call getpeercert()
> or the proposed getpeercertchain() is only after the TLS session has been
> established. As such, the SSL socket already established that there exists a
> va
Nathaniel Smith added the comment:
I'm not sure I agree about assuming that users will be able to work around
these issues... I mean, nothing personal, I'm sure you're well-informed and
maybe your code would work fine, but if you don't understand my example then
how
New submission from Nathaniel Smith :
In bpo-39386, the 'aclose' method for async generators was fixed so that the
following broken code would correctly raise an error:
# -- bad code --
async def agen_fn():
yield
async def do_bad_thing():
agen = agen_fn()
a
Change by Nathaniel Smith :
--
keywords: +patch
pull_requests: +17843
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/18475
___
Python tracker
<https://bugs.python.org/issu
Change by Nathaniel Smith :
--
pull_requests: +17844
pull_request: https://github.com/python/cpython/pull/18475
___
Python tracker
<https://bugs.python.org/issue39
Change by Nathaniel Smith :
--
pull_requests: +17877
pull_request: https://github.com/python/cpython/pull/18502
___
Python tracker
<https://bugs.python.org/issue39
Change by Nathaniel Smith :
--
pull_requests: +17878
pull_request: https://github.com/python/cpython/pull/18502
___
Python tracker
<https://bugs.python.org/issue39
Nathaniel Smith added the comment:
New changeset f464edf3239f7867fe31c9cd238a68fb3b90feaa by Nathaniel J. Smith in
branch '3.7':
bpo-39606: allow closing async generators that are already closed (GH-18475)
(GH-18502)
https://github.com/python/cpyt
Nathaniel Smith added the comment:
New changeset f464edf3239f7867fe31c9cd238a68fb3b90feaa by Nathaniel J. Smith in
branch '3.7':
bpo-39606: allow closing async generators that are already closed (GH-18475)
(GH-18502)
https://github.com/python/cpyt
Change by Nathaniel Smith :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Nathaniel Smith added the comment:
> was Tornado the only project experiencing this pain
At least twisted and anyio are still broken on 3.8+Windows because of this
change:
https://twistedmatrix.com/trac/ticket/9766
https://github.com/agronholm/anyio/issues
Change by Nathaniel Smith :
--
nosy: +ncoghlan, yselivanov
___
Python tracker
<https://bugs.python.org/issue40213>
___
___
Python-bugs-list mailing list
Unsub
Nathaniel Smith added the comment:
I don't have any particular insight into the bug, but I do have an
observation: this seems like an awful lot of energy to spend on some code
that's not even used by default. Would it make sense to deprecate it and
stick with ThreadedChildWatcher?
O
Nathaniel Smith added the comment:
How do you know that your reproducer is showing the same bug, or anything
related to signals? IIUC subprocess waiting by default doesn't involve a
signal handler.
On Sat, May 9, 2020, 14:40 Chris Jerdonek wrote:
>
> Chris Jerdonek added
Nathaniel Smith added the comment:
makes sense to me
On Tue, May 12, 2020 at 10:14 PM Chris Jerdonek wrote:
>
>
> Chris Jerdonek added the comment:
>
> Also adding Nathaniel since he's the one that filed #32751. Nathaniel, do you
> agree that if an exception occur
Change by Nathaniel Smith :
--
pull_requests: +19469
pull_request: https://github.com/python/cpython/pull/20170
___
Python tracker
<https://bugs.python.org/issue39
Nathaniel Smith added the comment:
New changeset 58205a0217e91232cc1e945dbfe4e387d636eb76 by Nathaniel J. Smith in
branch 'master':
bpo-39148: fixup to account for IPV6_ENABLED being moved (GH-20170)
https://github.com/python/cpython/commit/58205a0217e91232cc1e945dbfe4e3
Nathaniel Smith added the comment:
I think I fixed the buildbot issues in GH-20170, but I can't seem to reach the
buildbot site right now, so it's hard to know for sure!
--
versions: -Python 3.6
___
Python tracker
<https://bu
Nathaniel Smith added the comment:
If I remember correctly, cython-generated C code calls those macros at
appropriate places, so there are probably a bunch of projects that are
using them and the developers have no idea.
On Thu, Feb 18, 2021, 04:16 STINNER Victor wrote:
>
> STINNER
New submission from Nathaniel Smith :
Consider the following short program. demo() is a trivial async function that
creates a QObject instance, connects a Python signal, and then exits. When we
call `send(None)` on this object, we expect to get a StopIteration exception.
-
from PySide2
Nathaniel Smith added the comment:
I don't think I understand what you mean by "reentrant"... we don't have any
single piece of code that's calling back into itself. The question is about
what the tp_dealloc contract is.
Digging into it more, it looks
Nathaniel Smith added the comment:
Filed with PySide2: https://bugreports.qt.io/browse/PYSIDE-1313
--
resolution: -> not a bug
stage: -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Nathaniel Smith added the comment:
Just found this while searching to see if we had an existing way to combine
WeakValueDictionary + defaultdict. The use case I've run into a few times is
where you need a keyed collection of mutexes, like e.g. asyncio.Lock objects.
You want to make
Change by Nathaniel Smith :
--
pull_requests: -19900
___
Python tracker
<https://bugs.python.org/issue31254>
___
___
Python-bugs-list mailing list
Unsubscribe:
Nathaniel Smith added the comment:
FWIW, this seems like a pretty straightforward improvement to me.
--
___
Python tracker
<https://bugs.python.org/issue40
New submission from Nathaniel Smith :
So Windows finally has pty support:
https://devblogs.microsoft.com/commandline/windows-command-line-introducing-the-windows-pseudo-console-conpty/
However, the API is a bit weird. Unlike Unix, when you create a Windows pty,
there's no way to directl
Nathaniel Smith added the comment:
Yeah, writing a trivial "event loop" to drive actually-synchronous code is
easy. Try it out:
-
async def f():
print("hi from f()")
await g()
async def g():
print("hi from g()")
# This is our event loop:
coro
Nathaniel Smith added the comment:
> Whether or not one buys that, the point of my approach is that SQLAlchemy
> itself *will* publish async methods. End user code *will not* ever context
> switch to another task without them explicitly calling using an await.
Oh, I thought th
Nathaniel Smith added the comment:
> 90% of everything people are doing here are in the context of HTTP services.
> The problem of, "something.a now creates state that other tasks might see"
> is not a real "problem" that is solved by using IO-only explicit c
Nathaniel Smith added the comment:
Huh, this is very weird. I can confirm that the async generator objects aren't
cleaned up until loop shutdown on asyncio.
On the trio main branch, we don't yet use the `set_asyncgen_hooks` mechanism,
and the async generator objects are
Nathaniel Smith added the comment:
...On closer examination, it looks like that Trio PR has at least one test that
checks that async generators are collected promptly after they stop being
referenced, and that test passes:
https://github.com/python-trio/trio/pull/1564/files#diff
Nathaniel Smith added the comment:
Oh! I see it. This is actually working as intended.
What's happening is that the event loop will clean up async generators when
they're garbage collected... but, this requires that the event loop get a
chance to run. In the demonstration program
Nathaniel Smith added the comment:
I made a record of my investigations here:
https://github.com/python-trio/trio/issues/1586
One reason we might care about this is that asyncio ought to be using a clock
that stops ticking while suspended.
(On which note: Please don't switch macOS t
Nathaniel Smith added the comment:
It does seem pretty harmless in this case.
--
___
Python tracker
<https://bugs.python.org/issue41543>
___
___
Python-bug
Nathaniel Smith added the comment:
> Perhaps it's time to restart the original discussion, whether `aclose()`
> should cancel pending `anext`.
I'm still not aware of any way to make this work, technically. So it's a moot
point unless s
Nathaniel Smith added the comment:
As far as things like run/run_until_complete/etc go, I think doctests should
have the same semantics as async REPLs, whatever those end up being.
Given that we don't actually have mature async REPLs, that the core feature to
enable them only landed
New submission from Nathaniel Smith :
Maybe we should expose the SSL_CTX_set_num_tickets function:
https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_num_tickets.html
This is a new function added in OpenSSL 1.1.1, that lets you control the number
of tickets issued by a TLS 1.3 connection
Nathaniel Smith added the comment:
> Regarding your comment on client_context.num_ticket getter: IMHO it's not a
> good idea to raise an exception on attribute access. It may break
> introspection.
Hmm, I see what you mean.
Basically my intuition is: it's a bi
Nathaniel Smith added the comment:
Traditionally on Unix, sockets are represented by file descriptors. File
descriptors are small integers. (POSIX actually mandates the "small" part:
whenever a new fd is created, the OS has to assign it the smallest integer
value that's no
Nathaniel Smith added the comment:
One of Guido's mentees (I don't know her bpo account name) also ran into this:
https://python.zulipchat.com/#narrow/stream/116742-core.2Fhelp/topic/Weird.20sphinx.20error
Some analysis shows that we do have a bug in the docs – the sphinx &
Nathaniel Smith added the comment:
There are two different axes of sync/async here, and it's important not to mix
them up: the context manager could be sync or async, and the function that's
being decorated could be sync or async. This issue is about the case where you
have a sy
Nathaniel Smith added the comment:
I think we haven't *actually* done a proper DeprecationWarning period for this.
We tried, but because of the issue with byte-compiling, the warnings were
unconditionally suppressed for most users -- even the users who are diligent
enough to enable war
Nathaniel Smith added the comment:
> I wouldn't be OK with magic switching in the behaviour of ContextDecorator
> (that's not only semantically confusing, it's also going to make the
> contextlib function wrappers even slower than they already are).
I hear you on the
Nathaniel Smith added the comment:
@Yury: depends on what you mean by "100% reliable" :-). Like I said above, I'm
normally super against automagic detection of sync-vs-async functions because
of all the edge cases where it goes wrong, but in this specific case where
peopl
New submission from Nathaniel Smith:
The attached script looks innocent, but gives wildly incorrect results on all
versions of CPython I've tested.
It does two things:
- spawns a thread which just loops, doing nothing
- in the main thread, repeatedly increments a variable 'x
Nathaniel Smith added the comment:
Some thoughts based on discussion with Armin in #pypy:
It turns out if you simply delete the LocalsToFast and FastToLocals calls in
call_trampoline, then the test suite still passes. I'm pretty sure that pdb
relies on this as a way to set local vari
Nathaniel Smith added the comment:
It isn't obvious to me whether the write-through proxy idea is a good one on
net, but here's the rationale for why it might be.
Currently, the user-visible semantics of locals() and f_locals are a bit
complicated. AFAIK they aren't docu
Nathaniel Smith added the comment:
Interesting idea! I'm not sure I fully understand how it would work though.
What would you do for the frames that don't use the fast array, and where
locals() currently returns the "real" namespace?
How are you imagining that the trac
Nathaniel Smith added the comment:
> Folks that actually *wanted* the old behaviour would then need to do either
> "sys._getframe().f_locals" or "inspect.currentframe().f_locals".
So by making locals() and f_locals have different semantics, we'd be adding yet
an
Nathaniel Smith added the comment:
In terms of general design cleanliness, I think it would be better to make
`interrupt_main` work reliably than to have IDLE add workarounds for its
unreliability.
AFAICT the ideal, minimal redundancy solution would be:
- interrupt_main just calls raise
Nathaniel Smith added the comment:
Sorry, I meant bpo-21895.
--
___
Python tracker
<http://bugs.python.org/issue29926>
___
___
Python-bugs-list mailing list
Unsub
Nathaniel Smith added the comment:
> I like it because it categorically eliminates the "tracing or not?" global
> state dependence when it comes to manipulation of the return value of
> locals() - manipulating that will either always affect the original execution
>
Nathaniel Smith added the comment:
> A real Ctrl+C executes the registered control handlers for the process.
Right, but it's *extremely* unusual for a python 3 program to have a control
handler registered directly via SetConsoleCtrlHandler. This isn't an API that
the interp
1 - 100 of 488 matches
Mail list logo
