[issue13343] Lambda keyword-only argument not updating co_freevars
New submission from Joshua Landau : When setting defaults to keyword-only arguments in lambdas which are inside non-global scopes, cPython doesn't push the name to it's closure's co_freevars. EXAMPLE: global_variable = None (lambda: (lambda *, keyword_only=global_variable: None))() Because the inner lambda hasn't told the outer lambda to put global_variable in co_freevars, it fails to create the default to keyword_only. This only happens if the inner function is a lambda and you are setting a keyword_only variable. It does not cause a crash if global_variable is local to the outer lambda, as the opcode LOAD_FAST is still created properly (as opposed to LOAD_NAME). It does not crash if global_variable is used elsewhere in the outer function as co_freevars will be updated with it, allowing LOAD_NAME to retrieve it. I've never done a bug here before and I'm unsure what to say, so please be nice and correct me if I'm doing it wrong. -- components: Interpreter Core messages: 147026 nosy: Joshua.Landau priority: normal severity: normal status: open title: Lambda keyword-only argument not updating co_freevars type: crash versions: Python 3.2 ___ Python tracker <http://bugs.python.org/issue13343> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13343] Lambda keyword-only argument not updating co_freevars
Joshua Landau added the comment: Glad to help :) It's made my day. I get to boast at school now! -- ___ Python tracker <http://bugs.python.org/issue13343> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13658] Extra clause in class grammar documentation
New submission from Joshua Landau :
Inside the grammar for classes[1], the documentation states that the
inheritance list can be of type:
"(" [argument_list [","] | comprehension] ")"
The "comprehension" part seems to be superfluous, especially as it is valid
grammar without the clause.
The 2.7 docs state just "[expression list]", so either the addition should be
justified or the extra clause removed.
[1]
http://docs.python.org/py3k/reference/compound_stmts.html#grammar-token-classdef
Please see
http://mail.python.org/pipermail/python-list/2011-December/1284989.html for
discussion.
--
assignee: docs@python
components: Documentation
messages: 150169
nosy: Joshua.Landau, docs@python
priority: normal
severity: normal
status: open
title: Extra clause in class grammar documentation
versions: Python 3.2
___
Python tracker
<http://bugs.python.org/issue13658>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: http://www.python.org/dev/peps/pep-0448/ is out; see what you think. See http://mail.python.org/pipermail/python-ideas/2013-July/021872.html for all the juicy discussion so far. -- nosy: +Joshua.Landau ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15924] 404 link on Python about page
New submission from Joshua Landau: http://www.python.org/about/ section "Python plays well with others", last paragraph, link "extension modules" links to http://www.python.org/doc/ext/intro.html, a 404 page. http://www.python.org/doc/ext/ redirects to http://docs.python.org/extending/ -- components: None messages: 170345 nosy: Joshua.Landau priority: normal severity: normal status: open title: 404 link on Python about page ___ Python tracker <http://bugs.python.org/issue15924> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16010] Some Unicode in identifiers improperly rejected
New submission from Joshua Landau:
"a¹ = None" is not valid, even though unicodedata.normalize("NFKC", "¹") == "1".
One would expect "a¹ = None" and "a1 = None" to be equivalent in this case, as
with "aⁱ = None" and "ai = None".
I am not sure how many other characters exhibit the same problem.
References:
http://docs.python.org/py3k/reference/lexical_analysis.html#identifiers
http://mail.python.org/pipermail/python-list/2012-September/631420.html
"¹" === "\u00b9"
"ⁱ" === "\u2071"
--
components: Interpreter Core
messages: 171082
nosy: Joshua.Landau
priority: normal
severity: normal
status: open
title: Some Unicode in identifiers improperly rejected
versions: Python 3.2
___
Python tracker
<http://bugs.python.org/issue16010>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21547] '!s' formatting documentation bug
New submission from Joshua Landau: In the docs for 2.x about the formatting syntax: https://docs.python.org/2/library/string.html#format-string-syntax it says "Two conversion flags are currently supported: '!s' which calls str() on the value, and '!r' which calls repr()." but for unicode formatters, '!s' calls unicode() instead. See http://stackoverflow.com/questions/23773816/why-python-str-format-doesnt-call-str for the question that found this. -- assignee: docs@python components: Documentation messages: 218863 nosy: Joshua.Landau, docs@python priority: normal severity: normal status: open title: '!s' formatting documentation bug versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue21547> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21587] Tabs in source
New submission from Joshua Landau: There are tabs in the source: http://hg.python.org/cpython/file/5c8d71516235/Include/listobject.h#l49 I don't really know, but this seems like a release blocker to me ;). -- messages: 219183 nosy: Joshua.Landau priority: normal severity: normal status: open title: Tabs in source ___ Python tracker <http://bugs.python.org/issue21587> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21593] Clarify re.search documentation first match
New submission from Joshua Landau: The documentation for re.search does not state that it returns the first match. This should be added, or a clarification added if this is implementation-defined. https://docs.python.org/3/library/re.html#re.search --- See also http://stackoverflow.com/questions/23906400/is-regular-expression-search-guaranteed-to-return-first-match -- assignee: docs@python components: Documentation messages: 219270 nosy: Joshua.Landau, docs@python priority: normal severity: normal status: open title: Clarify re.search documentation first match versions: Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5 ___ Python tracker <http://bugs.python.org/issue21593> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21642] "_ if 1else _" does not compile
New submission from Joshua Landau:
By the docs,
Except at the beginning of a logical line or in
string literals, the whitespace characters space,
tab and formfeed can be used interchangeably to
separate tokens. Whitespace is needed between two
tokens only if their concatenation could otherwise
be interpreted as a different token
(e.g., ab is one token, but a b is two tokens).
"_ if 1else _" should compile equivalently to "_ if 1 else _".
The tokenize module does this correctly:
import io
import tokenize
def print_tokens(string):
tokens = tokenize.tokenize(io.BytesIO(string.encode("utf8")).readline)
for token in tokens:
print("{:12}{}".format(tokenize.tok_name[token.type], token.string))
print_tokens("_ if 1else _")
#>>> ENCODINGutf-8
#>>> NAME_
#>>> NAMEif
#>>> NUMBER 1
#>>> NAMEelse
#>>> NAME_
#>>> ENDMARKER
but it fails when compiled with, say, "compile", "eval" or "ast.parse".
import ast
compile("_ if 1else _", "", "eval")
#>>> Traceback (most recent call last):
#>>> File "", line 32, in
#>>> File "", line 1
#>>> _ if 1else _
#>>> ^
#>>> SyntaxError: invalid token
eval("_ if 1else _")
#>>> Traceback (most recent call last):
#>>> File "", line 40, in
#>>> File "", line 1
#>>> _ if 1else _
#>>> ^
#>>> SyntaxError: invalid token
ast.parse("_ if 1else _")
#>>> Traceback (most recent call last):
#>>> File "", line 48, in
#>>> File "/usr/lib/python3.4/ast.py", line 35, in parse
#>>> return compile(source, filename, mode, PyCF_ONLY_AST)
#>>> File "", line 1
#>>> _ if 1else _
#>>> ^
#>>> SyntaxError: invalid token
Further, some other forms work:
1 if 0b1else 0
#>>> 1
1 if 1jelse 0
#>>> 1
See
http://stackoverflow.com/questions/23998026/why-isnt-this-a-syntax-error-in-python
particularly,
http://stackoverflow.com/a/23998128/1763356
for details.
--
messages: 219614
nosy: Joshua.Landau
priority: normal
severity: normal
status: open
title: "_ if 1else _" does not compile
type: compile error
versions: Python 3.4
___
Python tracker
<http://bugs.python.org/issue21642>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21642] "_ if 1else _" does not compile
Joshua Landau added the comment: Here's a minimal example of the difference: 1e #>>> ... etc ... #>>> SyntaxError: invalid token 1t #>>> ... etc ... #>>> SyntaxError: invalid syntax -- ___ Python tracker <http://bugs.python.org/issue21642> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22451] filtertuple, filterstring and filterunicode don't have optimization for PyBool_Type
New submission from Joshua Landau: All code referred to is from bltinmodule.c, Python 2.7.8: https://github.com/python/cpython/blob/2.7/Python/bltinmodule.c filter implements and optimization for PyBool_Type, making it equivalent to PyNone: # Line 303 if (func == (PyObject *)&PyBool_Type || func == Py_None) The specializations for tuples, byte strings and unicode don't have this: # Lines 2776, 2827, 2956, 2976 if (func == Py_None) This is a damper against recommending `filter(bool, ...)`. --- Python 3 of course does not have these specializations, so has no bug. -- components: Library (Lib) messages: 227199 nosy: Joshua.Landau priority: normal severity: normal status: open title: filtertuple, filterstring and filterunicode don't have optimization for PyBool_Type type: performance versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue22451> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22451] filtertuple, filterstring and filterunicode don't have optimization for PyBool_Type
Joshua Landau added the comment: It's solely a speed thing. I think it was an oversight that the optimisation was only applied to lists. I didn't expect the optimisation to break when applied to tuples. -- ___ Python tracker <http://bugs.python.org/issue22451> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22451] filtertuple, filterstring and filterunicode don't have optimization for PyBool_Type
Joshua Landau added the comment: That sounds OK to me. It's a bit of a non-issue once you know about it. -- ___ Python tracker <http://bugs.python.org/issue22451> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment:
The problem seems to be that with the removal of
-else if (TYPE(ch) == STAR) {
-vararg = ast_for_expr(c, CHILD(n, i+1));
-if (!vararg)
-return NULL;
-i++;
-}
-else if (TYPE(ch) == DOUBLESTAR) {
-kwarg = ast_for_expr(c, CHILD(n, i+1));
-if (!kwarg)
-return NULL;
-i++;
-}
the code will ignore any subnodes that aren't of type "argument". However, the
grammar still says
arglist: (argument ',')* (argument [','] | '*' test [',' '**' test] | '**' test)
so this is incorrect.
Here's an example of what you might get
inner(
"a",argument comma
*"bcd", star test comma
"e",argument comma
f=6,argument comma
**{"g": 7}, doublestar test comma
h=8,argument comma
**{"i":9} doublestar test
)
--
___
Python tracker
<http://bugs.python.org/issue2292>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: This causes a segmentation fault if any keyword arguments come after a **-unpack. Minimal demo: f(**x, x=x) -- ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment:
Just change
if (!PyUnicode_Compare(tmp, key)) {
when iterating over prior keyword arguments to
if (tmp && !PyUnicode_Compare(tmp, key)) {
since tmp (the argument's name) can now be NULL.
--
___
Python tracker
<http://bugs.python.org/issue2292>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment:
I take it back; that just causes
>>> f(**{}, c=2)
XXX lineno: 1, opcode: 105
Traceback (most recent call last):
File "", line 1, in
SystemError: unknown opcode
--
___
Python tracker
<http://bugs.python.org/issue2292>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment:
I think I've got it working; I'm just working out how to make a patch and
adding a test or two. I think I'll also need to sign the contributor agreement.
While I'm at it, here are a few other deviations from the PEP:
- {*()} and {**{}} aren't supported
- [*[0] for i in [0]] gives a SystemError
- "return *(1, 2, 3)," fails whilst "*(1, 2, 3)," succeeds
--
___
Python tracker
<http://bugs.python.org/issue2292>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: This was a rather minor fix; I basically moved from STORE_SUBSCR to STORE_MAP and fixed a BUILD_MAP opcode. -- Added file: http://bugs.python.org/file37795/starunpack7.diff ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: Aye, I'd done so (see starunpack7.diff). It was the fuss to reapply it ontop of your newer diff and making sure I'd read at least *some* of the devguide before barging on. Anyhow, here's another small fix to deal with the [*[0] for i in [0]] problem. I'm not nearly confident I can touch the grammar, though, so the other problems are out of my reach. -- Added file: http://bugs.python.org/file37798/starunpack8.diff ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment:
I'm getting
>>> f(x=5, **{'x': 1}, **{'x': 3}, y=2)
Traceback (most recent call last):
File "", line 1, in
TypeError: f() got multiple values for keyword argument 'x'
Which, as I understand, is the correct result. I'm using starunpack8.diff right
now.
--
___
Python tracker
<http://bugs.python.org/issue2292>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment:
> The problem with using STORE_MAP is you create a new dict for each keyword
> argument in that situation.
You don't; if you look at the disassembly for producing a built-in dict
("dis.dis('{1:2, 2:3, 3:4}')") you'll see they use STORE_MAP too. STORE_MAP
seems to just be the map equivalent of LIST_APPEND.
I've done simple timings that show my version being faster...
Unfortunately, it points out there is definitely a memory leak. This reproduces:
def f(a):
pass
while True:
f(**{}, a=1)
This goes for both patches 8 and 9.
--
___
Python tracker
<http://bugs.python.org/issue2292>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment:
2 here as well:
15 LOAD_CONST 2 ('w')
18 LOAD_CONST 3 (1)
21 BUILD_MAP1
24 LOAD_CONST 4 (2)
27 LOAD_CONST 5 ('x')
30 STORE_MAP
31 BUILD_MAP1
34 LOAD_CONST 6 (3)
37 LOAD_CONST 7 ('y')
40 STORE_MAP
41 LOAD_CONST 8 (4)
44 LOAD_CONST 9 ('z')
47 STORE_MAP
48 LOAD_CONST 10 (5)
51 LOAD_CONST 11 ('r')
54 STORE_MAP
55 BUILD_MAP_UNPACK 2
--
___
Python tracker
<http://bugs.python.org/issue2292>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: Some of the tests seemed to be failing simply because they were incorrect. This fixes that. -- Added file: http://bugs.python.org/file37806/starunpack12.diff ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: I think I've fixed the memory leaks (plural). There were also a host of other problems with the _UNPACK opcodes in ceval. Here are the things I remember fixing, although I think I did slightly more: - Not throwing an error when PyDict_New or PyDict_Update fails. - Not doing Py_DECREF on stack items being popped. - Not checking if intersection is non-NULL. - Not doing Py_DECREF on intersection. Now the primary problem is giving good errors; I don't know how to make them look like they came from the function call. I'm not sure I want to, either, since these opcodes are used elsewhere. I do need to check something about this (what requirements are there on how you leave the stack when you "goto error"?), but that's an easy fix if my current guess isn't right. -- Added file: http://bugs.python.org/file37811/starunpack13.diff ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: > The _UNPACK opcodes are new in this changelist. Yup, but they're used in the other unpacking syntax too: (*(1, 2, 3), *(4, 5, 6)) -- ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: Good catch. CALL_FUNCTION seems to split its opcode into two to give it a positional-keyword pair so this seems fine. I'd hope we can do the same thing; personally I would do: BUILD_MAP_UNPACK( position_of_function_in_stack_or_0 << 8 | number_to_pack ) This way if building for a function we can do the check *and* give good errors that match the ones raised from CALL_FUNCTION. When the top 8 bits are 0, we don't do checks. What do you think? Would dual-usage be too confusing? -- ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: Functions are already limited to 255 arguments, so I don't think so. -- ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: According to the standard, int can be only 16 bits long so that only leaves 255/255. However, if the offset is on top of the dictionary count, this is easily enough to clear the limits for the maximum function size (worst case is a merge of 255 dicts with an offset of 1 or a merge of 2 dicts with an offset of 254). -- ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: We wouldn't want to replace STORE_MAP since that's used in dictionary comprehensions, but replacing BUILD_MAP with BUILD_MAP(n) sounds like a great idea. -- ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment:
I've looked at BUILD_MAP(n). It seems to work and has speed improvements but:
- I was wrong about the 16-bit int thing. It turns out CPython is happily
treating them as 32 bit as long as they are prefixed by an EXTENDED_ARG bytecode
https://docs.python.org/3/library/dis.html#opcode-EXTENDED_ARG
This could be used by BUILD_MAP rather than falling back to BUILD_MAP_UNPACK.
- It's probably best to not include it here, since it's a disjoint issue. This
patch wouldn't really be affected by its absence.
Also, if we limit BUILD_MAP_MERGE to 255 dictionaries, this will also apply to
the {**a, **b, **c, ...} syntax, although I really can't see it being a problem.
--
___
Python tracker
<http://bugs.python.org/issue2292>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: Why would that simplify things? -- ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: I phrased that badly. Whilst I can see minor simplifications to BUILD_MAP_UNPACK, the only way to add more information to CALL_FUNCTION_XXX would be through EXTENDED_ARG. This seems like it would outweigh any benefits, and the tiny duplication of error checking removed would be far dwarfed by the unpacking code in CALL_FUNCTION_XXX. -- ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment:
I imagine it like (in the map unpacking code)
func_offset = (oparg >> 8) & 0xFF;
num_maps = oparg & 0xFF;
// later
if (func_offset) {
// do checks
if (repeated_argument) {
raise_error_from_function(PEEK(func_offset + num_maps));
}
}
This code should be relatively quick, in an already-slow opcode, and rather
short.
If adding to CALL_FUNCTION_XXX, you would have to add an EXTENDED_ARG opcode
(because CALL_FUNCTION_XXX already uses the bottom 16 bits), add checks for the
top bits in the opcode, duplicate the (large) dictionary merging function. This
doesn't seem like it saves much work.
--
___
Python tracker
<http://bugs.python.org/issue2292>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment:
We wouldn't actually need to raise it "from" somewhere else; the line numbering
and frame are already correct. The only difficulty is that the traceback
currently says
# func(a=1, **{'a': 1})
TypeError: func() got multiple values for keyword argument 'arg'
To do this from the UNPACK opcode would require knowing where the function is
in order to print its name. (We also need to know whether to do the check at
all, so we'd be hijacking some bits the UNPACK opcode anyway.)
--
___
Python tracker
<http://bugs.python.org/issue2292>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: The stack will have the function, then any number of positional arguments, then optionally an *args, then any number (>= 2) of maps to unpack. To get to the function, you need to know the sum count of all of these. -- ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: The function object that's on the stack. -- ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: Just before any arguments to the function. -- ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment:
No, that happens in CALL_FUNCTION_KW:
>>> import dis
>>> dis.dis("f(x=1, **{'x': 1})")
1 0 LOAD_NAME0 (f)
3 LOAD_CONST 0 ('x')
6 LOAD_CONST 1 (1)
9 LOAD_CONST 1 (1)
12 LOAD_CONST 0 ('x')
15 BUILD_MAP1
18 CALL_FUNCTION_KW 256 (0 positional, 1 keyword pair)
21 RETURN_VALUE
There's no call to BUILD_MAP_UNPACK at all. Namely, it's raised from
update_keyword_args (in turn from ext_do_call).
--- Tangential note: ---
In fact, it seems the only reason we keep the mess of unpacking in two places
rather than just using BUILD_TUPLE_UNPACK and BUILD_MAP_UNPACK unconditionally
is that CALL_FUNCTION_XXX looks to be slightly more efficient by only dealing
with the case of a single unpack at the end. I think I see how to make the
_UNPACKs fast enough for this case, though, so maybe we could remove it and
unify a few things. I'd need to write it up, though.
--
___
Python tracker
<http://bugs.python.org/issue2292>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23316] Incorrect evaluation order of function arguments with *args
New submission from Joshua Landau: It is claimed that all expressions are evaluated left-to-right, including in functions¹. However, f(*a(), b=b()) will evaluate b() before a(). ¹ https://docs.python.org/3/reference/expressions.html#evaluation-order -- components: Interpreter Core messages: 234672 nosy: Joshua.Landau priority: normal severity: normal status: open title: Incorrect evaluation order of function arguments with *args type: behavior versions: Python 3.4, Python 3.5 ___ Python tracker <http://bugs.python.org/issue23316> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: Amazing, thanks. I also just uncovered http://bugs.python.org/issue23316; we'll need to support a patch for that. In fact, bad evaluation order is why I haven't yet gotten down my unification strategy. I wouldn't worry about extra opcodes when using *args or **kwargs, though; what matters is mostly avoiding extra copies. I guess a few `timeit`s will show whether I'm right or totally off-base. Most of what's needed for the error stuff is already implemented; one just needs to set the top bit flag (currently just 1<<8) to "1 + arg_count_on_stack()", which is a trivial change. I'll push a patch for that after I'm done fiddling with the unification idea. -- ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: If we're supporting f(**x for x in y) surely we should also support f(x: y for x, y in z) I personally don't like this idea. -- ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment:
Quick-fix for Guido's bug attached. I'm not familiar with this part of the
code, yet, so take this tentatively. I just changed
while (containers > 1) {
to
while (containers) {
---
@Guido
My comments were assuming `f(**x for x in y)` meant `f({**x for x in y})`.
I see your reasoning, but I don't like how your version has
(x for y in z for x in y) == (*y for y in z)
f(x for y in z for x in y) != f(*y for y in z)
This seems like a tripping point. I've never wanted to unpack a 2D iterable
into an argument list, so personally I'm not convinced by the value-add either.
--
Added file: http://bugs.python.org/file37866/starunpack19.diff
___
Python tracker
<http://bugs.python.org/issue2292>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: Update for the error messages fix. I've put aside the idea of unifying things for now because there are a couple of interdependencies I wasn't expecting and I absolutely don't want the fast-path for f(x) to get slower. -- Added file: http://bugs.python.org/file37867/starunpack20.diff ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: Special-cased `(*i for i in x)` to use YIELD_FROM instead of looping. Speed improved, albeit still only half as fast as chain.from_iterable. Fixed error message check in test_syntax and removed semicolons. -- Added file: http://bugs.python.org/file37928/starunpack30.diff ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: I don't know the etiquette rules for the issue tracker, but I'd really appreciate having something to debug -- it's working for me, you see. -- ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21593] Clarify re.search documentation first match
Changes by Joshua Landau : -- versions: +Python 3.6 ___ Python tracker <http://bugs.python.org/issue21593> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26843] tokenize does not include Other_ID_Start or Other_ID_Continue in identifier
New submission from Joshua Landau: This is effectively a continuation of https://bugs.python.org/issue9712. The line in Lib/tokenize.py Name = r'\w+' must be changed to a regular expression that accepts Other_ID_Start at the start and Other_ID_Continue elsewhere. Hence tokenize does not accept '℘·'. See the reference here: https://docs.python.org/3.5/reference/lexical_analysis.html#identifiers I'm unsure whether unicode normalization (aka the `xid` properties) needs to be dealt with too. Credit to toriningen from http://stackoverflow.com/a/29586366/1763356. -- components: Library (Lib) messages: 264145 nosy: Joshua.Landau priority: normal severity: normal status: open title: tokenize does not include Other_ID_Start or Other_ID_Continue in identifier type: behavior versions: Python 3.5 ___ Python tracker <http://bugs.python.org/issue26843> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26843] tokenize does not include Other_ID_Start or Other_ID_Continue in identifier
Joshua Landau added the comment: Sorry, I'd stumbled on my old comment on the closed issue and completely forgot about the *last* time I did the same thing. -- ___ Python tracker <http://bugs.python.org/issue26843> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9712] tokenize yield an ERRORTOKEN if the identifier starts with a non-ascii char
Joshua Landau added the comment: This doesn't seem to be a complete fix; the regex used does not include Other_ID_Start or Other_ID_Continue from https://docs.python.org/3.5/reference/lexical_analysis.html#identifiers Hence tokenize does not accept '℘·'. Credit to modchan from http://stackoverflow.com/a/29586366/1763356. -- nosy: +Joshua.Landau versions: +Python 3.4 ___ Python tracker <http://bugs.python.org/issue9712> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2292] Missing *-unpacking generalizations
Joshua Landau added the comment: There is a change as part of this to make dict building more like list and set building, which both have this behaviour. The same changes have likely occurred before whenever BUILD_LIST and BUILD_SET were introduced, and this behaviour seems particularly undefined. That said, I did overlook the difference. Hopefully there's agreement that it doesn't matter. -- ___ Python tracker <http://bugs.python.org/issue2292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21593] Clarify re.search documentation first match
Joshua Landau added the comment: This should also be applied to regex.search's docstring. https://docs.python.org/3.5/library/re.html#re.regex.search -- resolution: fixed -> status: closed -> open ___ Python tracker <http://bugs.python.org/issue21593> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24194] tokenize yield an ERRORTOKEN if an identifier uses Other_ID_Start or Other_ID_Continue
New submission from Joshua Landau:
This is valid:
℘· = 1
print(℘·)
#>>> 1
But this gives an error token:
from io import BytesIO
from tokenize import tokenize
stream = BytesIO("℘·".encode("utf-8"))
print(*tokenize(stream.read), sep="\n")
#>>> TokenInfo(type=56 (ENCODING), string='utf-8', start=(0, 0), end=(0,
0), line='')
#>>> TokenInfo(type=53 (ERRORTOKEN), string='℘', start=(1, 0), end=(1, 1),
line='℘·')
#>>> TokenInfo(type=53 (ERRORTOKEN), string='·', start=(1, 1), end=(1, 2),
line='℘·')
#>>> TokenInfo(type=0 (ENDMARKER), string='', start=(2, 0), end=(2, 0),
line='')
This is a continuation of http://bugs.python.org/issue9712. I'm not able to
reopen the issue, so I thought I should report it anew.
It is tokenize that is wrong - Other_ID_Start and Other_ID_Continue are
documented to be valid:
https://docs.python.org/3.5/reference/lexical_analysis.html#identifiers
--
components: Library (Lib)
messages: 243188
nosy: Joshua.Landau
priority: normal
severity: normal
status: open
title: tokenize yield an ERRORTOKEN if an identifier uses Other_ID_Start or
Other_ID_Continue
type: behavior
versions: Python 3.4
___
Python tracker
<http://bugs.python.org/issue24194>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
