[Python-Dev] Re: pth file encoding

2021-03-17 Thread Michał Górny
On Wed, 2021-03-17 at 13:55 +0900, Inada Naoki wrote:
> OK. setuptools doesn't specify encoding at all. So locale-specific
> encoding is used.
> We can not fix it in short term.

How about writing paths as bytestrings in the long term?  I think this
should eliminate the necessity of knowing the correct encoding for
the filesystem.

-- 
Best regards,
Michał Górny


___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/EKG7ELEWSG6ZPFYOVTCNVJCGV5W7S7J3/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: pth file encoding

2021-03-17 Thread Paul Moore
On Wed, 17 Mar 2021 at 08:13, Michał Górny  wrote:
>
> On Wed, 2021-03-17 at 13:55 +0900, Inada Naoki wrote:
> > OK. setuptools doesn't specify encoding at all. So locale-specific
> > encoding is used.
> > We can not fix it in short term.
>
> How about writing paths as bytestrings in the long term?  I think this
> should eliminate the necessity of knowing the correct encoding for
> the filesystem.

If I have a path in my Python program that is "a£b" (a unicode string)
and I want to write it to a .pth file, what encoding should I use to
"write it as a bytestring"? I don't understand what you;re trying to
suggest here.
Paul
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/YBE6D37V73OXZYNEW36JO24ZBD7EKAJQ/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: pth file encoding

2021-03-17 Thread Inada Naoki
On Wed, Mar 17, 2021 at 5:33 PM Paul Moore  wrote:
>
> On Wed, 17 Mar 2021 at 08:13, Michał Górny  wrote:
> >
> > On Wed, 2021-03-17 at 13:55 +0900, Inada Naoki wrote:
> > > OK. setuptools doesn't specify encoding at all. So locale-specific
> > > encoding is used.
> > > We can not fix it in short term.
> >
> > How about writing paths as bytestrings in the long term?  I think this
> > should eliminate the necessity of knowing the correct encoding for
> > the filesystem.
>
> If I have a path in my Python program that is "a£b" (a unicode string)
> and I want to write it to a .pth file, what encoding should I use to
> "write it as a bytestring"? I don't understand what you;re trying to
> suggest here.
> Paul

On Windows, it must be UTF-8. For example, we use `chcp 65001` in
`activate.bat` to support unicode path.
On Unix, raw path is bytestring. So paths can be written as-is. Python
decode it with fsencoding.

So I think this is the ideal solution. But this solution requires
platform-specific code in the site.py.
I don't think pth files are important enough for this complexity.

Sub-optimal idea is using UTF-8. It is the best encoding for Windows.
And most Unix systems use UTF-8 too.

Regards,

-- 
Inada Naoki  
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/NWBYQHLUIIWU2U2MX4KZXJH4PBTNJYAW/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: pth file encoding

2021-03-17 Thread Antoine Pitrou
On Tue, 16 Mar 2021 11:44:13 +0900
Inada Naoki  wrote:
> Hi, all.
> 
> I found .pth file is decoded by the default (i.e. locale-specific) encoding.
> https://github.com/python/cpython/blob/0269ce87c9347542c54a653dd78b9f60bb9fa822/Lib/site.py#L173
> 
> pth files contain:
> 
> * import statements
> * paths
> 
> For import statement, UTF-8 is the default Python code encoding.
> For paths, fsencoding is the right encoding. It is UTF-8 on Windows
> (excpet PYTHONLEGACYWINDOWSFSENCODING is set), and locale-specific
> encoding in Linux.
> 
> What encoding should we use?
> 
> * UTF-8
> * sys.getfilesystemencoding()
> * Keep status-quo.

You could add special markup to specify utf8 encoding:

# -*- encoding: utf8 -*-

If no markup is present, use locale encoding.  If markup is present,
use utf8 encoding.  Bail out if markup specifies something else than
utf8.

Then update all pth-producing tools to write utf8-encoded pth files
(at least on the Python versions that support the encoding markup).
In 15 years, you can switch to utf8 by default.

Regards

Antoine.


___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/J2IM4IQ3L3XEN6XBRFSDLQ2S2FORN3PP/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: pth file encoding

2021-03-17 Thread Paul Moore
On Wed, 17 Mar 2021 at 08:52, Inada Naoki  wrote:
> On Windows, it must be UTF-8. For example, we use `chcp 65001` in
> `activate.bat` to support unicode path.
> On Unix, raw path is bytestring. So paths can be written as-is. Python
> decode it with fsencoding.

Remember that .pth files contain executable code as well as paths, so
fsencoding is not correct for a .pth file as a whole.

> So I think this is the ideal solution. But this solution requires
> platform-specific code in the site.py.
> I don't think pth files are important enough for this complexity.

.pth files are pretty important in the packaging community. I'd
strongly support making their format and behaviour more precisely
defined.

> Sub-optimal idea is using UTF-8. It is the best encoding for Windows.
> And most Unix systems use UTF-8 too.

+1. IMO, UTF-8 is the only reasonable choice here.

The problem is with the transition - we need to find a way to deal
with existing `.pth` files, and with people using older version of
tools (like setuptools and pipx) that write `.pth` files (so we can't
assume, for example, that Python 3.12 will never see a .pth file using
the old-style encoding).

It's worth noting that using the default encoding is the *correct* way
of writing .pth files at the moment (as that's how site.py reads them
- see https://github.com/python/cpython/blob/master/Lib/site.py#L173)
so this is technically a file format change - tools writing .pth files
will *have* to include version-specific code if they want to support
multiple versions of Python. We need to be very clear about this -
it's not just a case of "tools need to specify the encoding".

Paul
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/MIZLKDTX2EXEHFKKHO33FRSO7EH62DGW/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: pth file encoding

2021-03-17 Thread Paul Moore
On Wed, 17 Mar 2021 at 09:26, Paul Moore  wrote:
> The problem is with the transition - we need to find a way to deal
> with existing `.pth` files, and with people using older version of
> tools (like setuptools and pipx) that write `.pth` files (so we can't
> assume, for example, that Python 3.12 will never see a .pth file using
> the old-style encoding).

Hmm, I just checked and pipx uses UTF-8 when writing .pth files. See
https://github.com/pipxproject/pipx/blob/master/src/pipx/venv.py#L176
(and lol, it was my mistake, I wrote that code -
https://github.com/pipxproject/pipx/pull/168). I'm inclined to report
that as a bug, even though it appears no-one has complained about it.
But that seems counter-productive given the context here.

Paul
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/V5AJK4WZY2JCGZVFI5KY3QD4DYVSSIBB/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: pth file encoding

2021-03-17 Thread Steve Dower

On 3/17/2021 8:00 AM, Michał Górny wrote:

How about writing paths as bytestrings in the long term?  I think this
should eliminate the necessity of knowing the correct encoding for
the filesystem.


That's what we're trying to do, the problem is that they start as 
strings, and so we need to convert them to a bytestring.


That conversion is the encoding ;)

And yeah, for reading, I'd use a UTF-8 reader that falls back to locale 
on failure (and restarts reading the file). But for writing, we need the 
tools that create these files (including Notepad!) to use the encoding 
we want.


Cheers,
Steve

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/MVD67FOAJRCNR2XXLJ4JDVFPYGZWYLDP/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: pth file encoding

2021-03-17 Thread Stefan Ring
On Wed, Mar 17, 2021 at 6:37 PM Steve Dower  wrote:
>
> On 3/17/2021 8:00 AM, Michał Górny wrote:
> > How about writing paths as bytestrings in the long term?  I think this
> > should eliminate the necessity of knowing the correct encoding for
> > the filesystem.
>
> That's what we're trying to do, the problem is that they start as
> strings, and so we need to convert them to a bytestring.
>
> That conversion is the encoding ;)
>
> And yeah, for reading, I'd use a UTF-8 reader that falls back to locale
> on failure (and restarts reading the file). But for writing, we need the
> tools that create these files (including Notepad!) to use the encoding
> we want.

A somewhat radical idea carrying this to the extreme would be to use
UTF-16 (LE) on Windows. After all, this _is_ the native file system
encoding, and Notepad will happily read and write it.
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/WRAW4UI3X3WYMQ3FMIERDKTVD6WKD5S2/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: pth file encoding

2021-03-17 Thread Steve Dower

On 3/17/2021 6:08 PM, Stefan Ring wrote:

A somewhat radical idea carrying this to the extreme would be to use
UTF-16 (LE) on Windows. After all, this _is_ the native file system
encoding, and Notepad will happily read and write it.


I'm not opposed to detecting a BOM by default (when no other encoding is 
specified), but that won't help most UTF-8 files which these days come 
with no marker at all.


I wouldn't change the default file encoding for writing though (except 
to unmarked UTF-8, and only with the compatibility approach Inada is 
working on). Everyone has basically come around to the idea that UTF-8 
is the only needed encoding, and I'm sure if it had existed when Windows 
decided to support a universal character set, it would have been chosen. 
But with what we have now, UTF-16-LE is not a good choice for anything 
apart from compatibility with Windows.


Cheers,
Steve

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/LTEJSNOH6EHESXSMXSW352JFG2SF7ZMX/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: pth file encoding

2021-03-17 Thread Ivan Pozdeev via Python-Dev

On 17.03.2021 20:30, Steve Dower wrote:

On 3/17/2021 8:00 AM, Michał Górny wrote:

How about writing paths as bytestrings in the long term?  I think this
should eliminate the necessity of knowing the correct encoding for
the filesystem.


That's what we're trying to do, the problem is that they start as strings, and 
so we need to convert them to a bytestring.

That conversion is the encoding ;)

And yeah, for reading, I'd use a UTF-8 reader that falls back to locale on failure (and restarts reading the file). But for writing, we 
need the tools that create these files (including Notepad!) to use the encoding we want.




I don't see a problem with using a file encoding specification like in Python 
source files.
Since site.py is under our control, we can introduce it easily.

We can opt to allow only UTF-8 here -- then we wait out a transitional period and disallow anything else than UTF-8 (then the specification 
can be removed, too).



Cheers,
Steve

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/MVD67FOAJRCNR2XXLJ4JDVFPYGZWYLDP/
Code of Conduct: http://python.org/psf/codeofconduct/


--
Regards,
Ivan

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/WZJ5EIP47AQV6X4MBN7427O4TNN5F4WY/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: pth file encoding

2021-03-17 Thread Steve Dower

On 3/17/2021 7:34 PM, Ivan Pozdeev via Python-Dev wrote:

On 17.03.2021 20:30, Steve Dower wrote:

On 3/17/2021 8:00 AM, Michał Górny wrote:

How about writing paths as bytestrings in the long term?  I think this
should eliminate the necessity of knowing the correct encoding for
the filesystem.


That's what we're trying to do, the problem is that they start as 
strings, and so we need to convert them to a bytestring.


That conversion is the encoding ;)

And yeah, for reading, I'd use a UTF-8 reader that falls back to 
locale on failure (and restarts reading the file). But for writing, we 
need the tools that create these files (including Notepad!) to use the 
encoding we want.




I don't see a problem with using a file encoding specification like in 
Python source files.

Since site.py is under our control, we can introduce it easily.

We can opt to allow only UTF-8 here -- then we wait out a transitional 
period and disallow anything else than UTF-8 (then the specification can 
be removed, too).


The only thing we can introduce *easily* is an error when the 
(exclusively third-party) tools that create them aren't up to date. 
Getting everyone to specify the encoding we want is a much bigger 
problem with a much slower solution.


This particular file is probably the worst case scenario, but preferring 
UTF-8 and handling existing files with a fallback is the best we can do 
(especially since an assumption of UTF-8 can be invalidated on a 
particular file, whereas most locale encodings cannot). Once we openly 
document that it should be UTF-8, tools will have a chance to catch up, 
and eventually the fallback will become harmless.


Cheers,
Steve
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/5B53GCQNYXFBYAHSJKI6I34XAV6S67HN/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Non-monotonically increasing line numbers in dis.findlinestarts() output

2021-03-17 Thread Skip Montanaro
Consider this little session from the tip of the spear:

>>> sys.version
'3.10.0a6+ (heads/master:0ab152c6b5, Mar 15 2021, 17:24:38) [GCC 10.2.0]'
>>> def while2(a):
... while a >= 0:
... a -= 1
... return a
...
>>> dis.dis(while2)
  2   0 LOAD_FAST0 (a)
  2 LOAD_CONST   1 (0)
  4 COMPARE_OP   5 (>=)
  6 POP_JUMP_IF_FALSE   24

  3 >>8 LOAD_FAST0 (a)
 10 LOAD_CONST   2 (1)
 12 INPLACE_SUBTRACT
 14 STORE_FAST   0 (a)

  2  16 LOAD_FAST0 (a)
 18 LOAD_CONST   1 (0)
 20 COMPARE_OP   5 (>=)
 22 POP_JUMP_IF_TRUE 8

  4 >>   24 LOAD_FAST0 (a)
 26 RETURN_VALUE
>>> list(dis.findlinestarts(while2.__code__))
[(0, 2), (8, 3), (16, 2), (24, 4)]

I get that somewhere along the way the compiler has been optimized to
duplicate the loop's test so as to eliminate some jumps (perhaps). It's not
clear to me that the line number should be "2" in the duplicated test
though. Shouldn't it be "3"?

I stumbled on this while trying to generate a line number table in my side
project register VM. As I understand it, the line number delta in the
output table is supposed to always be >= 0. In my code I'm using
dis.findlinestarts() to determine the line numbers for each block. Perhaps
I should be modifying its results. OTOH, maybe it's a bug. (If that's the
consensus, I will create an issue.)

Skip
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/V76BSGEHGHPM3D4TXYTGBXCZ5UCCBOVV/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: pth file encoding

2021-03-17 Thread Ivan Pozdeev via Python-Dev


On 17.03.2021 23:04, Steve Dower wrote:

On 3/17/2021 7:34 PM, Ivan Pozdeev via Python-Dev wrote:

On 17.03.2021 20:30, Steve Dower wrote:

On 3/17/2021 8:00 AM, Michał Górny wrote:

How about writing paths as bytestrings in the long term?  I think this
should eliminate the necessity of knowing the correct encoding for
the filesystem.


That's what we're trying to do, the problem is that they start as strings, and 
so we need to convert them to a bytestring.

That conversion is the encoding ;)

And yeah, for reading, I'd use a UTF-8 reader that falls back to locale on failure (and restarts reading the file). But for writing, we 
need the tools that create these files (including Notepad!) to use the encoding we want.




I don't see a problem with using a file encoding specification like in Python 
source files.
Since site.py is under our control, we can introduce it easily.

We can opt to allow only UTF-8 here -- then we wait out a transitional period and disallow anything else than UTF-8 (then the 
specification can be removed, too).


The only thing we can introduce *easily* is an error when the (exclusively third-party) tools that create them aren't up to date. Getting 
everyone to specify the encoding we want is a much bigger problem with a much slower solution.


I don't see a problem with either.
If we want to standardize something, we have to encourage, then ultimately 
enforce compliance, this way or another.



This particular file is probably the worst case scenario, but preferring UTF-8 and handling existing files with a fallback is the best we 
can do (especially since an assumption of UTF-8 can be invalidated on a particular file, whereas most locale encodings cannot). Once we 
openly document that it should be UTF-8, tools will have a chance to catch up, and eventually the fallback will become harmless.


Cheers,
Steve


--
Regards,
Ivan

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/LN3MHC7O7NHBCCROZGZJOZ5DY76KFLJP/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Non-monotonically increasing line numbers in dis.findlinestarts() output

2021-03-17 Thread MRAB

On 2021-03-17 22:10, Skip Montanaro wrote:

Consider this little session from the tip of the spear:

 >>> sys.version
'3.10.0a6+ (heads/master:0ab152c6b5, Mar 15 2021, 17:24:38) [GCC 10.2.0]'
 >>> def while2(a):
...     while a >= 0:
...         a -= 1
...     return a
...
 >>> dis.dis(while2)
   2           0 LOAD_FAST                0 (a)
               2 LOAD_CONST               1 (0)
               4 COMPARE_OP               5 (>=)
               6 POP_JUMP_IF_FALSE       24

   3     >>    8 LOAD_FAST                0 (a)
              10 LOAD_CONST               2 (1)
              12 INPLACE_SUBTRACT
              14 STORE_FAST               0 (a)

   2          16 LOAD_FAST                0 (a)
              18 LOAD_CONST               1 (0)
              20 COMPARE_OP               5 (>=)
              22 POP_JUMP_IF_TRUE         8

   4     >>   24 LOAD_FAST                0 (a)
              26 RETURN_VALUE
 >>> list(dis.findlinestarts(while2.__code__))
[(0, 2), (8, 3), (16, 2), (24, 4)]

I get that somewhere along the way the compiler has been optimized to 
duplicate the loop's testso as to eliminate some jumps (perhaps). It's 
not clear to me that the line number should be "2" in the duplicated 
test though.Shouldn't it be "3"?


The condition is on line 2. The fact that it has been duplicated n the 
bytecode is irrelevant; it's still on line 2 in the source.


Line 3 has the decrement and line 4 has the return.

I stumbled on this while trying to generate a line number table in my 
side project register VM. As I understand it, the line number delta in 
the output table is supposed to always be >= 0. In my code I'm using 
dis.findlinestarts() to determine the line numbers for each block. 
Perhaps I should be modifying its results. OTOH, maybe it's a bug. (If 
that's the consensus, I will create an issue.)



___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/X52UUKO22IHCKCW7N6GNBFAHBRZTMMMO/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Non-monotonically increasing line numbers in dis.findlinestarts() output

2021-03-17 Thread Ned Batchelder

On 3/17/21 6:41 PM, MRAB wrote:

On 2021-03-17 22:10, Skip Montanaro wrote:

I stumbled on this while trying to generate a line number table in my 
side project register VM. As I understand it, the line number delta 
in the output table is supposed to always be >= 0. In my code I'm 
using dis.findlinestarts() to determine the line numbers for each 
block. Perhaps I should be modifying its results. OTOH, maybe it's a 
bug. (If that's the consensus, I will create an issue.)


co_lnotab has had negative deltas since 3.6.

--Ned.
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/EK2O6SE6J6LROGNHKPQYTQNJH5VFE2M4/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Non-monotonically increasing line numbers in dis.findlinestarts() output

2021-03-17 Thread Skip Montanaro
> co_lnotab has had negative deltas since 3.6.

Thanks. I'm probably misreading Objects/lnotab_notes.txt.

Skip
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/ATKSFCDGEXW5G6LADOM2F4SAUVLARHGP/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] aiter/anext review request

2021-03-17 Thread Joshua Bronson
Dear python-dev,

New here (but not to Python). 👋 Brett Cannon recommended I start a thread here 
(thanks, Brett!).

In December, two colleagues and I submitted 
https://github.com/python/cpython/pull/23847, "Add aiter and anext to 
builtins", which would fix https://bugs.python.org/issue31861.

Would any core developers who may be reading this be willing and able to 
provide a code review?

We would love to try to address any review feedback before having to fix 
(another round of) merge conflicts. (And ideally maybe even get this landed in 
time for the 3.10 feature freeze in early May?)

Thanks and hope this finds you well.
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/5XUVPB5H4PFUGTC5F7KAN4STKAEOFBQM/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Non-monotonically increasing line numbers in dis.findlinestarts() output

2021-03-17 Thread Victor Stinner
Hi Skip,

I modified co_lnotab format when I worked on the FAT Python
optimization project which modified the AST to emit more efficient
bytecode. My optimizer moved instructions, the line number could
become non-monotonic (..., line 2, line 3, line 2, line 4, ...), and
so lnotab (..., +1, -1, +2, ...) could be negative.

Example with loop unrolling:

def func1():
for i in ("a", "b"):
print(i)

is optimized:

def func2():
i = "a" # line 1
print(i) # line 2 (+1)

i = "b" # line 1 (-1)
print(i) # line 2 (+1)

I'm happy to see that Python 3.10 now also implements faster bytecode
which rely on this change ;-)

It seems like loop unrolling is not implemented in Python 3.10 yet. I
guess that Python/ast_opt.c has no strategy yet to decide if an
optimization should be applied or not depending on the code size. Loop
unrolling can make code way bigger.

Victor

On Thu, Mar 18, 2021 at 12:12 AM Ned Batchelder  wrote:
>
> On 3/17/21 6:41 PM, MRAB wrote:
> > On 2021-03-17 22:10, Skip Montanaro wrote:
> >
> >> I stumbled on this while trying to generate a line number table in my
> >> side project register VM. As I understand it, the line number delta
> >> in the output table is supposed to always be >= 0. In my code I'm
> >> using dis.findlinestarts() to determine the line numbers for each
> >> block. Perhaps I should be modifying its results. OTOH, maybe it's a
> >> bug. (If that's the consensus, I will create an issue.)
>
> co_lnotab has had negative deltas since 3.6.
>
> --Ned.
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/[email protected]/message/EK2O6SE6J6LROGNHKPQYTQNJH5VFE2M4/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/VQAT4BHRJN4DKBLJ27MG6PWWVQJOUTFY/
Code of Conduct: http://python.org/psf/codeofconduct/