New submission from Nathaniel Smith :
Asyncio needs a way to schedule work from other threads; and it also needs a
way to scheduler work from code that can run at arbitrary times in the same
thread, such as signal handlers or object finalizers ("reentrant contexts").
Nathaniel Smith added the comment:
> What would make it not reentrant-safe?
Probably the most obvious example of a non-reentrant-safe operation is taking a
lock. It's very natural to write code like:
def call_soon(...):
with self._call_soon_lock:
...
but now imagine tha
Nathaniel Smith added the comment:
FYI Christian, your "typical scenario for HTTP" doesn't make sense to me... you
can't send HTTP Connection Upgrade in the middle of a regular request/response
cycle. I feel like the typical scenario ought to be more like:
* client
* s
Nathaniel Smith added the comment:
Also, python-config is inconsistent with distutils. It should link to libpython
only in the cases where distutils does. (IIRC it's supposed to depend on
whether python was built with --enable-shared.)
--
nosy:
Changes by Nathaniel Smith :
--
nosy: +njs
___
Python tracker
<http://bugs.python.org/issue21176>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from Nathaniel Smith:
Numpy would like to switch to using the CPython allocator interface in order to
take advantage of the new tracemalloc infrastructure in 3.4. But, numpy relies
on the availability of calloc(), and the CPython allocator API does not expose
calloc().
https
Nathaniel Smith added the comment:
@Charles-François: I think your worries about calloc and overcommit are
unjustified. First, calloc and malloc+memset actually behave the same way here
-- with a large allocation and overcommit enabled, malloc and calloc will both
go ahead and return the
Nathaniel Smith added the comment:
On my laptop (x86-64, Linux 3.13, 12 GB RAM):
$ python3 -c "[i for i in range(9)]"
zsh: killed python3 -c "[i for i in range(9)]"
$ dmesg | tail -n 2
[404714.401901] Out of memory: Kill process 10752 (python3) score 687
Nathaniel Smith added the comment:
Right, python3 -c 'b"x" * (2 ** 48)' does give an instant MemoryError for me.
So I was wrong about it being the VM limit indeed.
The documentation on this is terrible! But, if I'm reading this right:
http://lxr.free-electrons.com/
Nathaniel Smith added the comment:
> It would be interesting to see some NumPy benchmarks (Nathaniel?).
What is it you want to see? NumPy already uses calloc; we benchmarked it when
we added it and it made a huge difference to various realistic workloads :-).
What NumPy gets out of this is
Nathaniel Smith added the comment:
A simple solution would be to change the name of the struct, so that
non-updated libraries will get a compile error instead of a runtime crash.
--
___
Python tracker
<http://bugs.python.org/issue21
New submission from Nathaniel Smith:
There's a small copy-paste error in the docs for collections.abc.AsyncGenerator
-- it's called collections.abc.Generator:
"class collections.abc.Generator
ABC for asynchronous generator classes that implement the protocol defined
in
New submission from Nathaniel Smith:
The string "isasync" does not appear to occur on this page:
https://docs.python.org/3.6/library/inspect.html
--
assignee: docs@python
components: Documentation
messages: 281662
nosy: docs@python, njs, yselivanov
priority: normal
severi
New submission from Nathaniel Smith:
It turns out that CPython built with --with-fpectl has a different ABI than
CPython built without --with-fpectl (which is the default). Specifically, if
you have an extension module built against a --with-fpectl CPython, and it uses
the PyFPE_{START,END
Nathaniel Smith added the comment:
> At that point, does it actually make sense to provide the shims? Or should we
> instead just add the deprecation warnings and say "Don't use the
> --with-fpectl option, as it doesn't work properly, and breaks ABI
> compatibility
New submission from Nathaniel Smith:
The documentation for select.epoll.poll doesn't document the return value at
all, which is somewhat important information :-)
I think it's a list of (fd, eventmask) tuples?
https://docs.python.org/3.7/library/select.html#select.
New submission from Nathaniel Smith:
When calling kevent(), selectors.KqueueSelector.select sets the "maxevents"
argument to len(self._fd_to_key). So if there are no fds registered, it passes
maxevents=0.
It turns out that the kevent() API has a special case behavior for maxeve
New submission from Nathaniel Smith:
If SelectSelector.select() is called when there are no fds registered, then it
ends up calling select.select([], [], [], timeout).
On sensible operating systems, this is equivalent to time.sleep(timeout). On
Windows, it raises an error. Asyncio manages to
Nathaniel Smith added the comment:
> Please don't do that. In Python, we have a long tradition of trying to
> provide thin wrappers to OS functions: os and select modules are good example.
I don't find this argument terribly convincing... Python also has a long
history of pap
Nathaniel Smith added the comment:
Side note, in case anyone else finds themselves looking for the MSDN magazine
article referenced above: it appears to be in the June 2007 issue, titled
"CONCURRENCY: Synchronization Primitives New To Windows Vista".
For the moment at least, this
New submission from Nathaniel Smith:
Example 1:
---
def f():
try:
raise KeyError
except Exception:
yield
gen = f()
gen.send(None)
gen.throw(ValueError)
-
Output:
Traceback (most recent call last):
File "", line 1, in
File "
Nathaniel Smith added the comment:
I disagree with the stated reason for closing this, because in general,
implicit context chaining doesn't care about where the exception was
instantiated, only where it was raised. For example:
-
err = ValueError()
try:
raise KeyError
e
Nathaniel Smith added the comment:
I read the patch but I don't understand the logic behind it :-). Why should the
value of tstate->exc_type affect whether we save/restore exc_info? Won't this
mean that things are still broken for code like:
-
def f():
try:
r
Nathaniel Smith added the comment:
(Issue 29587 is a duplicate of this one, but it has some more information on
where the !throwflag check came from and why it might be possible to remove it
now.)
--
___
Python tracker
<http://bugs.python.
New submission from Nathaniel Smith:
The attached script sets up a stack of generators calling each other via 'yield
from': f yields from g yield from h. Then the generator at the bottom of the
stack yields.
Before they yield, sys._getframe shows the expected stack (or if
Changes by Nathaniel Smith :
--
nosy: +njs
___
Python tracker
<http://bugs.python.org/issue18861>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from Nathaniel Smith:
The following code prints "KeyError()", which is obviously wrong and rather
baffling. Just passing an exception object around *as an object* should not
trigger implicit exception chaining!
If you replace the async functions with regular functio
New submission from Nathaniel Smith:
So he asked me to file a bug reminding him to write more unit tests.
Interesting cases:
async def f():
return (x for x in await g())
async def f():
return (await x for x in g())
(and ditto for [], {} comprehensions)
--
components
Nathaniel Smith added the comment:
We just ran into this in pip -- https://github.com/pypa/pip/pull/3836
I'd really recommend dropping the current "grovel through the binary doing a
regex search" strategy -- it's incredibly error prone, and AFAICT doesn't
really give
Nathaniel Smith added the comment:
> The purpose of the function was to determine the minimum libc compatibility
> requirements of the executable.
For what it's worth, I didn't know this, the pip authors obviously didn't know
this, and after rereading the docs just no
New submission from Nathaniel Smith:
Following on from the discussion starting here:
http://thread.gmane.org/gmane.comp.python.devel/150438/focus=150604
Here's a patch to improve __class__ assignment.
1) We remove the HEAPTYPE check from object_set_class, and move it to
same_slots_
Nathaniel Smith added the comment:
Here's a slightly improved patch spurred by a parenthetical comment of
Antoine's on the mailing list :-).
The only change is that it adds a check in subclass_dealloc to correct the
reference counting in the weird case that someone converts a HEAPT
Nathaniel Smith added the comment:
It's not terribly difficult to write a crude-but-effective aligned allocator on
top of raw malloc:
def aligned_malloc(size, alignment):
assert alignment < 255
raw_pointer = (uint8*) malloc(size + alignment)
shift = alignment - (raw
Nathaniel Smith added the comment:
Re: msg232219: If you go down the route of adding both aligned_malloc and
aligned_free to the Allocator structure, I think you might as well support it
for all domains? For the PyMem and PyObject domains you can just literally set
the default functions to be
Nathaniel Smith added the comment:
I hereby invoke the one month ping rule! Patch, be pinged!
--
___
Python tracker
<http://bugs.python.org/issue22986>
___
___
New submission from Nathaniel Smith:
DeprecationWarning and PendingDeprecationWarning are invisible by default. The
rationale for this is that they are only useful to people who are writing code,
so they should not be shown to end-users who are merely running code.
If someone is typing stuff
Nathaniel Smith added the comment:
I also filed a similar bug with ipython:
https://github.com/ipython/ipython/issues/8478
--
___
Python tracker
<http://bugs.python.org/issue24
New submission from Nathaniel Smith:
(Noticed while fixing the IPython equivalent of issue 24294)
The obvious way to deprecate a module is to issue a DeprecationWarning inside
the main body of the module, i.e.
# thirdpartymodule.py
import warnings
warnings.warn("{} is deprecated&qu
Nathaniel Smith added the comment:
*cough* You know, there's more to life than Python-X.Y.tar.gz :-). Not that I
know how PendingDeprecationWarning is used in the wild. I've been thinking
maybe we (numpy) should start using it for stuff that we want to discourage
people from using (
Nathaniel Smith added the comment:
Okay, that sounds reasonable to me.
--
___
Python tracker
<http://bugs.python.org/issue24294>
___
___
Python-bugs-list mailin
Nathaniel Smith added the comment:
There isn't really any magic in how warnings work. Basically someone calls
warnings.warn(...), which is a regular Python function, and it gathers up some
information about the specific warning message and calling context, checks a
global var
Nathaniel Smith added the comment:
A nice new API is certainly a nice thing, but I feel that changing the stack
walking code should be considered as fixing a regression introduced in 3.3.
Indeed, searching github for code that actually uses the stacklevel= argument:
https://github.com
Nathaniel Smith added the comment:
Recording this here so it doesn't get lost: Marc-Andre Lemberg suggested on
python-ideas that for the builtin REPL, this should be enabled iff
sys.stdin.isatty(). (I guess he is worried about 'cat script.py | python'.) I'm
not really sur
Nathaniel Smith added the comment:
For whatever it's worth as a non-core-developer, the patch looks good to me.
--
___
Python tracker
<http://bugs.python.org/is
Nathaniel Smith added the comment:
Ping.
I know this is pretty trivial and everyone's focused on 3.5-related stuff, but
it would be nice to get this finalized soon b/c the sooner CPython commits to
some standard behavior here, the sooner all the other (faster-moving) python
REPLs
Nathaniel Smith added the comment:
For 3.4/3.5 purposes, I propose a simpler algorithm: first, define a function
which takes a module name and returns true if it is part of the internal
warning machinery and false otherwise. This is easy because we know what import
machinery we ship.
Then
Nathaniel Smith added the comment:
Yeah, yours is probably better in fact, I was just trying to make the semantics
as obvious as explicit as possible for a comment :-)
--
___
Python tracker
<http://bugs.python.org/issue24
Nathaniel Smith added the comment:
I don't see how any of those suggestions help for writing an automated test.
Spawning a shell is irrelevant; the problem is to get a tty, which is much
harder. There only way I can see that might work for an automated test is to use
https://docs.pytho
Nathaniel Smith added the comment:
If pty is going to work at all then it should work regardless of whether the
tests themselves are being run under a tty, yes.
I personally would not want to merge a test based on making isatty lie, because
the point of tests is to increase confidence that
Nathaniel Smith added the comment:
> I still don't quite understand: what is the user-perceived result of this
> change? Module authors issuing a DeprecationWarning can now use stacklevel=2
> instead of stacklevel=10?
Exactly. There are a lot of deprecated modules in the
Nathaniel Smith added the comment:
You're right, "impossible" is a slight exaggeration :-). As an alternative,
every package could indeed carry around a table containing the details of
importlib's call stack in every version of Python.
(I also haven't checked whether
Nathaniel Smith added the comment:
Well, yeah, that indeed sucks.
Not sure what the best solution is. Some options:
1) "Don't do that then"
2) Explicitly add a "__class__" property to every immutable type, that
unconditionally errors out on assignment.
3) Add a hack
Nathaniel Smith added the comment:
Python goes to great lengths to make __class__ assignment work in general (I
was surprised too). Historically the exception has been that instances of
statically allocated C extension type objects could not have their __class__
reassigned. Semantically, this
Nathaniel Smith added the comment:
Wow, Mark, why so hostile? What's so wrong with my proposed less-intrusive
patch that you feel the need to call for reversion while condescendingly
pointing me to a non-solution? Of course I know about six. There was a whole
python-dev thread abou
Nathaniel Smith added the comment:
I need to get to bed so I'll finish up tomorrow, but FYI I have a working patch
-- I just want to add some __bases__ assignment test cases to make Larry happy.
(Apparently there are no test cases for __bases__ assignment at all
currently... :-(.)
B
Nathaniel Smith added the comment:
Actually, I fail at grep, and actually there are already tons of good tests for
__bases__ assignment in test_descr.py. So, patch attached, and analysis follows.
Background on __class__ assignment
--
While it's cert
Nathaniel Smith added the comment:
Mark Shannon wrote:
> So, just make sure that you insert the new object into sys.modules *before*
> doing any imports or calls to code that could import your module and it will
> all work fine.
The problem with doing this is that you're now
Nathaniel Smith added the comment:
On further thought, here's a slightly improved version of the patch I posted
above.
The difference is that the first version allowed through attempted __class__
assignments where either the old or new class was a subclass of ModuleType; the
new version
Nathaniel Smith added the comment:
Doh, apparently I did once know about issue23726 and then forgot. Good catch.
Technically we could add a __class__ field to ModuleType directly, but I think
then the ModuleType __class__ setter would basically need to be an exact
reimplementation of
Nathaniel Smith added the comment:
Thanks Raymond. Hi Guido. Sorry about the mess.
My overview of the situation so far, and of the patch currently attached to
this bug, is here (and a bit in the next post):
https://bugs.python.org/issue24912#msg249438
>From where I sit this all looks lik
Nathaniel Smith added the comment:
Guido van Rossum wrote:
> But first, why is it so important to assign the __class__ of a module? It
> seems somebody is trying to make modules into what they weren't meant to be.
Because you told me to do it this way on python-dev
Nathaniel Smith added the comment:
> I do understand your predicament. Can you live with just a special case for
> modules? __class__ assignment is full of non-portable special cases already.
Not only can I live with it, but -- unless I misunderstand your meaning -- this
is exact
Nathaniel Smith added the comment:
Pull request version of patch is here:
https://bitbucket.org/larry/cpython350/pull-requests/9/fix-issue-24912-disallow-reassigning-the/diff
(identical to issue24912-v2.patch except for the addition of a NEWS entry
Nathaniel Smith added the comment:
Adding Eric Snow to nosy because it seems like there may be some natural
overlap between this and the per-subinterpreter GIL ideas he brought up on
python-ideas back in June.
--
nosy: +eric.snow
___
Python tracker
Nathaniel Smith added the comment:
Since I know there's a lot of text here for people to process, here's an
attempted TL;DR (with inspiration from Serhiy's msg249495):
There were three parts to the original change:
1) Bug fixes for typeobject.c
2) Enabling __class__ assign
Nathaniel Smith added the comment:
There's already a PR on bitbucket that I made, in case that's helpful. The only
difference from the v2 patch are that I rephrased some of the comment in a more
neutral way (based on feedback from Mark posted on the PR itself), and added a
NEWS
Nathaniel Smith added the comment:
Sorry, you were too quick for me :-)
--
___
Python tracker
<http://bugs.python.org/issue24912>
___
___
Python-bugs-list mailin
Nathaniel Smith added the comment:
I didn't want to get further into the weeds of debating basic design points on
the tracker while blocking rc3, but, for the record -- the proposal in
msg249504 that one could keep the namespace of an old and new module object
effectively in sync by def
Nathaniel Smith added the comment:
Hooray! Thanks Larry.
Would it make sense to do a 3.4.x backport, or is that closed now with 3.5
being imminent?
--
___
Python tracker
<http://bugs.python.org/issue24
Nathaniel Smith added the comment:
Some limited code search statistics posted upthread (msg28) suggest that
~100% of real-world code is using stacklevel=2 unconditionally and thus getting
incorrect results on 3.3 and 3.4.
--
___
Python tracker
Nathaniel Smith added the comment:
Also just checked searchcode.com, which allows more fine-grained queries, and
it reports ~6500 hits for "stacklevel=2" and exactly 0 for "stacklevel=8".
Huh. So the official word is that requiring stacklevel=8 on 3.4 is not a bug,
rath
Nathaniel Smith added the comment:
ok
--
___
Python tracker
<http://bugs.python.org/issue24305>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by Nathaniel Smith :
--
nosy: +njs
___
Python tracker
<http://bugs.python.org/issue4709>
___
___
Python-bugs-list mailing list
Unsubscribe:
Nathaniel Smith added the comment:
> We are working on the Fortran issue in a few different ways, one of which is
> your work on MinGW compatibility so that gfortran can be used. But that isn't
> the only solution to this whole problem, and it's not even preferable for a
&g
Nathaniel Smith added the comment:
Hi Steve- okay, thanks for clarifying! I think you already know this, but for
the general record: the reason for the apparent fixation on this solution is
that after a lot of struggle it's emerged as basically the only contender for
scipy-developme
Nathaniel Smith added the comment:
Nothing's changed in python 2.7. Basically: (a) no numpy ints have ever
serialized in py3. (b) in py2, either np.int32 *xor* np.int64 will serialize
correctly, and which one it is depends on sizeof(long) in the C compiler used
to build Python. (This fo
Nathaniel Smith added the comment:
Name should be _PyThreadState_UncheckedGet
--
nosy: +njs
___
Python tracker
<http://bugs.python.org/issue26154>
___
___
Pytho
Nathaniel Smith added the comment:
Hi Victor,
This is really great, thanks for working on it!
What do you think about the richer api I proposed here?
http://bugs.python.org/issue18835#msg232221
--
___
Python tracker
<http://bugs.python.
Nathaniel Smith added the comment:
> Note: I don't understand what are domain, tag (address of the memory block?)
> or quantity (size of the memory block?).
Yes, the idea is basically the same as now, except with an extra const char *
specifying the "domain", so that on
Nathaniel Smith added the comment:
I think we're talking past each other :-).
> If I change tracemalloc, it's not to fullfit numpy requirements, it must
> remain very generic
Nothing about what I'm saying is relevant to numpy -- the patch attached to
this bug report
Nathaniel Smith added the comment:
There are other leakable resources besides heap and GPU memory -- shmem
segments and file descriptors are two that I thought of earlier, but there are
probably others too. (Note that not all file descriptors are associated with a
python file object.) I guess
Nathaniel Smith added the comment:
High-level questions:
- How do you anticipate the integers naming domains will be allocated? Is it
like port numbers, and you'll maintain a registry somewhere ("domains 0-100 are
reserved for the interpreter, pycuda has reserved 100-110, ..."
Nathaniel Smith added the comment:
Yes, it probably would be a good idea to disallow assigning __class__ for
immutable types.
--
___
Python tracker
<http://bugs.python.org/issue23
Nathaniel Smith added the comment:
> Would it be possible to upgrade the "manylinux" compiler (take a more recent
> GCC version)?
No, it's possible :-(. 4.8.2 is the very most modern version of GCC you can use
if you want to build binaries to run on CentOS/RHEL 5. (And
Nathaniel Smith added the comment:
@rgbecker: Are you able to pull out the config.log generated by running
python's ./configure script, and post that somewhere?
--
___
Python tracker
<http://bugs.python.org/is
Nathaniel Smith added the comment:
Yeah, the config.log there clearly shows that configure is using gcc 4.8.2 on
the failed builds. Very odd.
Stepping back for a moment, is there any point in continuing to debug this?
Given Benjamin's comment up-thread:
> I mainly converted them to s
Nathaniel Smith added the comment:
> 3.6 currently compiles correctly with gcc4.2, the default on Mac OS X 10.5
> (Leopard).
So to summarize our current understanding: gcc 4.3 added support for C99
inline, which explains why gcc 4.2 works and gcc 4.8 doesn't.
WTF
Nathaniel Smith added the comment:
+1 to the change for generators. This is actually in the PEP 533 draft:
https://www.python.org/dev/peps/pep-0533/#modifications-to-basic-iterator-types
For coroutines, doesn't this overlap with the existing "coroutine not awaited
401 - 488 of 488 matches
Mail list logo
