Nathaniel Smith added the comment:
New changeset 11a1493bc4198f1def5e572049485779cf54dc57 by Nathaniel J. Smith
(Christian Heimes) in branch 'master':
[bpo-28414] Make all hostnames in SSL module IDN A-labels (GH-5128)
https://github.com/python/cpyt
Nathaniel Smith added the comment:
New changeset 1c37e277190565f0e30fc9281caae4c899ac3b50 by Nathaniel J. Smith
(Miss Islington (bot)) in branch '3.7':
[bpo-28414] Make all hostnames in SSL module IDN A-labels (GH-5128) (GH-5843)
https://github.com/python/cpyt
Change by Nathaniel Smith :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Nathaniel Smith added the comment:
The help for 'n' says:
(Pdb) help n
n(ext)
Continue execution until the next line in the current function
is reached or it returns.
(And the docs [1] say essentially the same.)
It sounds like if that were true, then you would
Nathaniel Smith added the comment:
Do you mean, the fix is in 3.7 only?
--
status: pending -> open
___
Python tracker
<https://bugs.python.org/issu
Nathaniel Smith added the comment:
...huh, not sure why that shows me changing the status. I just typed something
in the text box, didn't touch any of the dropdowns...
--
status: open -> pending
___
Python tracker
<https://bugs
Nathaniel Smith added the comment:
If openssl says the behavior is undefined, then don't we have to first make it
defined before we can document it?
And if we're going to detect this case and guarantee some behavior, making it a
no-op like it is on regular sockets seems the
Nathaniel Smith added the comment:
I agree that backporting X509_VERIFY_PARAM_set1_host is unreasonable, at least
until the openssl ecosystem has moved forward a bit. But in earlier versions,
would it be easy to detect that do_handshake() hasn't been called and raise an
error?
The doc
Nathaniel Smith added the comment:
My point is that SSL_write(3ssl) says "WARNING: When calling SSL_write() with
num=0 bytes to be sent the behaviour is undefined."
Apparently on the particular openssl you're looking at, it gives
SSL_ERROR_SYSCALL with error code == 0 and le
Nathaniel Smith added the comment:
This is a duplicate of bpo-26692 and bpo-23530, and possibly others.
My impression is that len(os.sched_getaffinity(os.getpid())) is the Right
Guess. Currently sched_getaffinity isn't implemented on Windows, but bpo-23530
has some example code for h
Nathaniel Smith added the comment:
That stackoverflow thread points to the GNU coreutils 'nproc', which is an
interesting compendium of knowledge about this problem.
It looks like their algorithm is roughly:
1. Determine how many CPUs *could* this program access, by going down
Nathaniel Smith added the comment:
I can't find any evidence that NPROCS is used by other batch schedulers (I
looked at SLURM, Torque, and SGE). @M J Harvey, do you have any other examples
of systems that use it?
--
___
Python tracker
&
Nathaniel Smith added the comment:
> You mean duplicating "nproc"'s logic in Python?
Yeah.
> If someone wants to do the grunt work of implementing/testing it...
Well, that's true of any bug fix / improvement :-). The logic isn't terribly
complicated thoug
Nathaniel Smith added the comment:
Class decorators are also worth considering in cases where you find yourself
reaching for a metaclass.
--
nosy: +njs
___
Python tracker
<https://bugs.python.org/issue32
Nathaniel Smith added the comment:
You should also think about loop lifecycle: right now it's using the same loop
for all test cases, so callbacks can leak between tests. Twisted actually goes
a step further and explicitly checks for left over callbacks and fails the test
if any are
Nathaniel Smith added the comment:
PyOpenSSL supports renegotiation, and I've used it for writing exactly this
kind of test. Might be easier than wrapping a test harness around the 'openssl'
command.
For extra fun, openssl itself doesn't fully support renegotiation on
Nathaniel Smith added the comment:
> an OpenSSL cop-out
Perhaps, but they've been doing it this way for a decade+, and we're living in
a time when other libraries like BoringSSL are flat-out removing renegotiation
support, so good luck convincing them to fix things. Anyway, it
Nathaniel Smith added the comment:
Eric Wieser (added to CC) actually just opened a PR for this against NumPy:
https://github.com/numpy/numpy/pull/10741
I have weak and mixed feelings about the whole thing:
https://github.com/numpy/numpy/pull/10741#issuecomment-373637440
--
nosy
Nathaniel Smith added the comment:
Sorry, I misspoke -- I meant he opened a related PR. The PR is to add
as_integer_ratio to np.float16, np.float32, np.longdouble, not to add it to the
numpy integer types. There are similar issues though
Nathaniel Smith added the comment:
Whoa, wait, what?
I agree that the original post is not as diplomatic as it could be, but my
reaction to learning about this just now is also shock and confusion, so I
guess I can sympathize with the OP a bit...
The reason I'm surprised is that --
Nathaniel Smith added the comment:
@ncoghlan: The comparison I'm worried about is specifically this one: IIUC,
right now it's safe to run 'pip --version' in an arbitrary directory, but it's
not safe to run 'python -m pip --version' in an arbitrary directory
Nathaniel Smith added the comment:
Ah, yeah, I see:
~/t$ echo 'print("hi")' > re.py
~/t$ pip --version
pip 9.0.1 from
/home/njs/.user-python3.5-64bit/local/lib/python3.5/site-packages (python 3.5)
~/t$ python -m pip --version
hi
Traceback (most recent call last):
[
Nathaniel Smith added the comment:
Yeah, getaddrinfo isn't actually cancellable (because it's a blocking call that
has to be run in a thread), but it's side-effect-free so if the user requests
that it be cancelled then it's safe to return immediately while it keeps
runni
Nathaniel Smith added the comment:
The only time this should matter semantically is if someone is trying to
subclass SSLSocket and use multiple inheritance. This is something that people
*really* shouldn't do.
It also potentially makes it harder to backport ssl changes to 2.7.
I&
Nathaniel Smith added the comment:
Yes, a PR would be better... also, the two versions of the patch appear to be
identical?
CC'ing Antoine and Victor b/c they seem to have written this dangling_threads
checking stuff and I'm not sure I understand it :-).
As some extra background:
Nathaniel Smith added the comment:
@Victor: So to make sure I understand, the point of the check is to complain
about reference cycles involving thread objects, because when those happen in
the stdlib you consider them bugs?
--
___
Python tracker
New submission from Nathaniel Smith :
I thought there was already a bug for this, but it came up in conversation
again and I can't find one, so, here you go...
It looks like originally there was this bug for making control-C wake up the
asyncio event loop in Windows: https://github.com/p
Nathaniel Smith added the comment:
You're looking for 'Path.unlink'. Agreed though that the name makes it hard to
find. (I guess it makes sense to old-school Unix greybeards, but Python does
generally try to be useful for more people than just them...) There was some
discussio
Nathaniel Smith added the comment:
FYI, here's how trio handles such things – I'm sure it's not perfect, but might
be useful for inspiration:
https://trio.readthedocs.io/en/latest/reference-hazmat.html#instrument-api
--
nosy: +njs
___
Nathaniel Smith added the comment:
@adampl: The first step is for someone (possibly you :-)) to write a patch and
submit it as a PR against the Python master branch. Then once the actual change
is figured out, we can have the discussion about which releases to backport it
to
Change by Nathaniel Smith :
--
pull_requests: +6230
___
Python tracker
<https://bugs.python.org/issue26153>
___
___
Python-bugs-list mailing list
Unsubscribe:
Nathaniel Smith added the comment:
It does seem like it'd be unfortunate to end up in a situation like "sorry,
there's a bug in handling this python 2 code, so black won't be able to
reformat it until the next major python release". And I assume this issue is
Change by Nathaniel Smith :
--
nosy: +njs
priority: release blocker -> normal
___
Python tracker
<https://bugs.python.org/issue33363>
___
___
Python-bugs-lis
Nathaniel Smith added the comment:
Sorry, not sure why bpo decided to reset the priority there.
--
priority: normal -> release blocker
___
Python tracker
<https://bugs.python.org/issu
Nathaniel Smith added the comment:
That's a "form feed" or "page break" character:
https://en.wikipedia.org/wiki/Page_break
Quoting that page: "The form feed character is sometimes used in plain text
files of source code as a delimiter for a page break, or a
Nathaniel Smith added the comment:
Also it's a smaller win, but it might be worth considering whether we can avoid
the import of 'nt' on posix platforms and vice-versa.
--
___
Python tracker
<https://bugs.pyt
Nathaniel Smith added the comment:
I guess I should say that I'm still confused about why we're coming up with
such elaborate schemes here, instead of declaring that f_locals and locals()
shall return a dict proxy so that from the user's point of view, they Always
Just Work T
Nathaniel Smith added the comment:
@arigo: But CPython is already committed to supporting writes to locals() at
any moment, because at any moment you can set a trace function and in every
proposal trace functions can reliably write to locals. So I don't think this is
a serious obstacl
Nathaniel Smith added the comment:
@Nick:
> We're OK with the idea that installing a trace function might automatically
> turn off various compiler and interpreter managed optimisations
We're OK with assigning to locals() doing that too.
> What we're aiming to
Nathaniel Smith added the comment:
There's also aligned calloc, which no native APIs support but is still quite
useful.
--
___
Python tracker
<https://bugs.python.org/is
Nathaniel Smith added the comment:
> I'm not sure that it's a good idea to provide a "aligned malloc" fallback if
> such fallback would be inefficient. For example, we would have to
> overallocate the memory block not only for the requested alignement, but a
Nathaniel Smith added the comment:
> On the other hand, sane requests will have the exact multiple most of the
> time anyway.
The ways we've discussed using aligned allocation in numpy wouldn't follow this
requirement without special checking. Which isn't necessarily a
Nathaniel Smith added the comment:
Given the complexities here, and that the Track/Untrack functions are public
now, I do wonder if the actual aligned allocation routines should just be an
internal API (i.e., not exposed in Python.h
Nathaniel Smith added the comment:
> Can you elaborate why numpy wouldn't use this new API? I designed it with
> numpy in mind :-)
The reasons I had in mind are:
1) numpy hasn't actually come to a decision about whether to use aligned
allocation at all, or under what circu
Nathaniel Smith added the comment:
> But since no fast (kernel-zeroed) aligned_calloc() exists, I must use
> memset() anyway.
For large allocations, you'll probably be better off implementing your own
aligned allocator on top of calloc than implementing your own calloc on top of
Nathaniel Smith added the comment:
I believe https://github.com/python/cpython/pull/3010 is the fix you're looking
for.
--
nosy: +njs
___
Python tracker
<https://bugs.python.org/is
Nathaniel Smith added the comment:
See also PEP 565.
--
___
Python tracker
<https://bugs.python.org/issue24294>
___
___
Python-bugs-list mailing list
Unsub
Nathaniel Smith added the comment:
My impression was that progress on PEP 543 is temporarily stalled and not going
to be finished for 3.7. Is that wrong?
There's going to need to be a significant amount of shimming to implement the
PEP 543 interfaces on top of ssl no matter what y
New submission from Nathaniel Smith :
I have a test case that sets up a blocking SSLSocket, and eventually calls
unwrap() to do a proper SSL shutdown.
Every once in a while, the test blows up, because unwrap() unexpectedly raises
SSLWantWriteError. This is very unexpected for a blocking
Nathaniel Smith added the comment:
Oh darn, I restarted the 311618077 build on Travis and apparently that makes it
delete the log. Well, it was the same traceback as the one that's left, but
triggered by a different test.
--
___
Python tr
Nathaniel Smith added the comment:
There's no timeout. The man page claims SSL_ERROR_WANT_WRITE can't happen on a
blocking socket, but who knows...
Re: EINTR, this is all happening in a child thread. On Linux, this would mean
that it almost certainly isn't receiving any signal
Nathaniel Smith added the comment:
Another reason to consider making it possible to disable renegotiation is
HTTP/2. RFC 7540 says:
A deployment of HTTP/2 over TLS 1.2 MUST disable renegotiation. An
endpoint MUST treat a TLS renegotiation as a connection error
(Section 5.4.1) of
Nathaniel Smith added the comment:
It looks like openssl master has SSL_OP_NO_RENEGOTIATION:
https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_options.html
Before that, I guess one could use SSL_CTX_sess_{connect,accept}_renegotiate to
detect when a renegotiation has occurred and then
Change by Nathaniel Smith :
--
keywords: +patch
pull_requests: +4689
stage: -> patch review
___
Python tracker
<https://bugs.python.org/issue29137>
___
___
Py
Change by Nathaniel Smith :
--
keywords: +patch
pull_requests: +4691
stage: -> patch review
___
Python tracker
<https://bugs.python.org/issue30050>
___
___
Py
Change by Nathaniel Smith :
--
keywords: +patch
pull_requests: +4692
stage: needs patch -> patch review
___
Python tracker
<https://bugs.python.org/issu
Nathaniel Smith added the comment:
PR 4793 provides everything that Jinja2 and Trio actually need right now. It
doesn't provide any way to annotate tracebacks with extra data, but I'm not
entirely sure if that's necessary or what it would look like, so I figured I'd
at le
Nathaniel Smith added the comment:
Serhiy: I don't know what "specify a custom handler" means in this context. Can
you elaborate? The fd buffer overflow happens in a very delicate context where
we definitely cannot call python code or even safely touch PyOb
Nathaniel Smith added the comment:
Yeah, I agree with Antoine and Victor that a callback would be overkill, and it
would be extremely difficult to implement since at the point where the error
occurs, we don't have and can't ta
Nathaniel Smith added the comment:
Update!
I've been experimenting with this some more, and here's a more detailed
proposal, that I'd ideally like to get into 3.7. I don't *think* this is big
enough to need a PEP? I dunno, thoughts on that welcome.
Motivation: It&#x
Nathaniel Smith added the comment:
> 1. How will trio handle situations like:
>
> c = coro()
> await ...
> nursery.start_soon(c)
>
> ?
[...]
> Maybe creating a coroutine and not immediately passing it to 'start_soon' or
> similar API is an
Nathaniel Smith added the comment:
> In any case, in my opinion, it doesn't matter. `sys.set_coroutine_wrapper`
> controls a single thread-local setting,
> `sys.set_unawaited_coroutine_tracking` *also* controls a single thread-local
> setting. Both of them have the exact
Nathaniel Smith added the comment:
I think the earliest open bug about this is bpo-21327.
--
nosy: +njs
___
Python tracker
<https://bugs.python.org/issue32
Nathaniel Smith added the comment:
I did a little digging. It's more complicated than just "on Linux, these show
up in the socket type".
Background: SOCK_NONBLOCK and SOCK_CLOEXEC are Linux-isms. The standard way to
control blocking-ness and cloexec-ness is via ioctl/f
Nathaniel Smith added the comment:
Re-reading my above post, I realized I want to emphasize a little more how odd
the current relationship is between socket.type and SOCK_CLOEXEC. Right now,
the way it works is:
Python *always* adds SOCK_CLOEXEC to the type that it passes to the operating
New submission from Nathaniel Smith :
Suppose you're writing a library that allows users to make or accept SSL/TLS
connections. You use the 'ssl' module, because that's convenient. You need to
let your users configure your SSL/TLS connections, and there really isn't
Nathaniel Smith added the comment:
I think we already hold onto the ALPN list internally.
A possibly stickier issue is retrieving certificates, keys, trust db
configuration.
--
___
Python tracker
<https://bugs.python.org/issue32
Nathaniel Smith added the comment:
Yeah, I'm not entirely sure whether fixing this is actually doable or
worthwhile, but figured I should at least make an issue to discuss :-).
The problem is, in the motivating use case of wanting to be able to reliably
convert an SSLContext into some
Nathaniel Smith added the comment:
Ouch, yes, that's a tricky bug. This is definitely caused by the way that
asyncio internally converts signals into messages along a pipe (well, socket,
but same thing), and then after a fork-without-exec the child keeps writing
into that pipe.
Nathaniel Smith added the comment:
It might be possible to create ProcessPoolExecutor and get it to spawn all the
workers *before* you start the asyncio loop. It looks like ProcessPoolExecutor
delays spawning workers until the first piece of work is submitted, but at that
point it spawns all
Nathaniel Smith added the comment:
Ping -- anyone up for reviewing PR 4793?
https://github.com/python/cpython/pull/4793
It's pretty straightforward, and I figure better to ping now and beat the
end-of-month rush :-)
--
___
Python tracker
&
Nathaniel Smith added the comment:
Ping -- anyone up for reviewing PR 4789?
https://github.com/python/cpython/pull/4789
It's pretty straightforward, and I figure better to ping now and beat the
end-of-month rush :-). Also, it has an autoconf refresh in it, so it's likely
to develo
Change by Nathaniel Smith :
--
nosy: +njs
___
Python tracker
<https://bugs.python.org/issue32443>
___
___
Python-bugs-list mailing list
Unsubscribe:
Nathaniel Smith added the comment:
Yes, they're intentionally retained as no-ops, so that it remains possible to
load old extensions that were compiled against an fpe build and refer to those
symbols. Is there a problem?
--
___
Python tr
Nathaniel Smith added the comment:
It looks like Python's tracking the "running" state of async generators wrong:
we should have ag_running set to True when we enter asend/athrow/aclose and
False when we exit, but instead it's being toggled back and forth on each
*inne
Nathaniel Smith added the comment:
My thoughts: https://bugs.python.org/issue32526#msg309783
--
___
Python tracker
<https://bugs.python.org/issue30773>
___
___
Nathaniel Smith added the comment:
Wow, yeah, that is a tricky one.
Didn't Ned say, though, that at this point we should be treating 3.7 like an
already-released bugfix-only branch?
--
___
Python tracker
<https://bugs.python.org/is
Nathaniel Smith added the comment:
Fair enough. And I can't think of any specific way that fixing this is likely
to break anyone, just it's subtle enough that I don't necessarily trust my
intuition :-).
--
___
Python
Nathaniel Smith added the comment:
Note that in my experience, socket systems treat the buffer sizes as more
like... rough guidelines. Especially Windows and Linux. Which doesn't mean
they're not useful to expose somehow, but you can't assume that just because
you set the b
Nathaniel Smith added the comment:
The idea here is *not* to avoid using a thread pool in general. When the data
is on disk, using a thread pool is (a) unavoidable, because of how operating
system kernels are written, and (b) basically fine anyway, because the overhead
added by threads is
Nathaniel Smith added the comment:
Huh, that's weird. My first thought was some kind of race condition, but...
raise_signal uses raise(), which on Windows should be invoking the signal
handler synchronously, so the warning should definitely be printed before
raise_signal() returns.
Nathaniel Smith added the comment:
This is a duplicate of bpo-16487, which has more discussion about how the API
might work.
--
nosy: +njs
resolution: -> duplicate
stage: -> resolved
status: open -> closed
___
Python tracke
Nathaniel Smith added the comment:
I think the first thing is to add async "modes" to compile: in particular
"async-exec" and "async-single". These would be like the current "exec" and
"single" modes respectively, except that they act like
Nathaniel Smith added the comment:
Here's another case where this bug bites us:
https://github.com/python-trio/trio/issues/673
At startup, Trio checks if SIGINT is currently being handled by Python's
default SIGINT handler, and if so it substitutes its own SIGINT handler (which
Nathaniel Smith added the comment:
Part of the issue here is the one discussed in bpo-30773 / bpo-32526: async
generators allow themselves to be re-entered while another asend/athrow/aclose
call is in progress, and it causes weird and confusing results. So at a
minimum, trying to call
New submission from Nathaniel Smith :
Hey Ned, we just noticed that since a few days ago the trio testsuite is
failing on 3.7-dev (but not 3.7.0), in a test checking an obscure feature in
the ssl module: https://travis-ci.org/python-trio/trio/builds/431291929
And I just reproduced the issue
Nathaniel Smith added the comment:
(And Christian, if you know of any risky-sounding recent changes in
SSLObject.unwrap, lmk :-))
--
___
Python tracker
<https://bugs.python.org/issue34
Nathaniel Smith added the comment:
Git bisect says:
1229664f30dd5fd4da32174a19258f8312464d45 is the first bad commit
commit 1229664f30dd5fd4da32174a19258f8312464d45
Author: Miss Islington (bot) <31488909+miss-isling...@users.noreply.github.com>
Date: Mon Sep 17 12:12:13 2018 -0700
Nathaniel Smith added the comment:
The test doesn't involve any threads, so it does seem strange that this PR
changed its behavior.
I haven't checked against master carefully, but the original observation was
that our Travis "3.7-dev" and "3.8-dev" tests s
Nathaniel Smith added the comment:
The ssl module's unwrap() method is intended to do a clean shutdown of TLS
encryption on a socket (or memory BIO or whatever). The idea is that it sends a
"close notify" frame (i.e., tells the peer that we're shutting down), and then
i
Nathaniel Smith added the comment:
Oh, here it is:
https://github.com/python/cpython/commit/1229664f30dd5fd4da32174a19258f8312464d45#diff-e1cc5bf74055e388cda128367a814c8fR2587
-if (err < 0) {
+if (err.ssl < 0) {
Before in this function, 'err' was the return code f
Change by Nathaniel Smith :
--
keywords: +patch
pull_requests: +8881
stage: -> patch review
___
Python tracker
<https://bugs.python.org/issue34759>
___
___
Py
Nathaniel Smith added the comment:
PR posted. Also seems to affect 3.6, so adding that to the tags.
--
keywords: +3.6regression
versions: +Python 3.6
___
Python tracker
<https://bugs.python.org/issue34
Nathaniel Smith added the comment:
Whoops, yep, I forgot it doesn't auto-close.
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.pyth
Nathaniel Smith added the comment:
I didn't know this, but apparently the SSLKEYLOGFILE envvar is a de-facto
standard: chrome, firefox, and libcurl all check for this envvar, and if found
they log TLS secrets to the file in a specific format.
Reports of projects supporting this:
-
Nathaniel Smith added the comment:
FWIW, I hadn't noticed this change in py3, but it would never have occurred to
me that it's controversial... I thought everyone who'd studied the issue agreed
that round-half-to-even was the best default :-). Numpy has always done
round-to
Change by Nathaniel Smith :
--
nosy: -njs
___
Python tracker
<https://bugs.python.org/issue32956>
___
___
Python-bugs-list mailing list
Unsubscribe:
Nathaniel Smith added the comment:
As a point of information, this did in fact break pylint/astroid:
https://github.com/PyCQA/astroid/issues/617
--
nosy: +njs
___
Python tracker
<https://bugs.python.org/issue32
Nathaniel Smith added the comment:
Would it make sense to implement a "chaos" mode (that e.g. testing tools could
enable unconditionally), that disables the small integer and small string
caches?
--
nosy: +njs
___
Python track
Nathaniel Smith added the comment:
Yeah, something like that. Or sys.enable_chaos_mode(), that pytest or whoever
could call before running tests.
--
___
Python tracker
<https://bugs.python.org/issue34
301 - 400 of 488 matches
Mail list logo
