[Python-Dev] Re: What's the rule for issuing line tracing numbers when using a tracing function?
Em qui., 16 de set. de 2021 às 11:49, Fabio Zadrozny
escreveu:
> Hi all,
>
> I have a weird case where I thought line events would be issued and yet
> they aren't even though they're in the instructions in the bytecode (both
> in 3.9 and 3.10).
>
> i.e.:
>
> Given the code:
>
> def check_backtrack(x): # line 1
> if not (x == 'a' # line 2
> or x == 'c'): # line 3
> pass # line 4
>
> it has dis.dis such as:
>
> 2 0 LOAD_FAST0 (x)
> 2 LOAD_CONST 1 ('a')
> 4 COMPARE_OP 2 (==)
> 6 POP_JUMP_IF_TRUE12 (to 24)
>
> 3 8 LOAD_FAST0 (x)
> 10 LOAD_CONST 2 ('c')
> 12 COMPARE_OP 2 (==)
>
> 2 14 POP_JUMP_IF_TRUE10 (to 20)
>
> 4 16 LOAD_CONST 0 (None)
> 18 RETURN_VALUE
>
> 2 >> 20 LOAD_CONST 0 (None)
> 22 RETURN_VALUE
> >> 24 LOAD_CONST 0 (None)
> 26 RETURN_VALUE
>
> So, by just following the instructions/line numbers, I'd say that when the
> instruction:
>
> 2 14 POP_JUMP_IF_TRUE10 (to 20)
>
> is executed, a line event would take place, yet, this isn't true, but if
> that offset is changed to include more instructions then such a line event
> is issued.
>
> i.e.: something as:
>
> def tracer(frame, event, arg):
> print(frame, event)
> return tracer
>
> import sys
> sys.settrace(tracer)
> check_backtrack('f')
>
> prints:
>
> 1 call
> 2 line
> 3 line
> 4 line
> 4 return
>
> when I expected it to print:
>
> 1 call
> 2 line
> 3 line
> 2 line |<-- this is not being issued
> 4 line
> 4 return
>
> So, I have some questions related to this:
>
> Does anyone know why this happens?
> What's the rule to identify this?
> Why is that line number assigned to that instruction (i.e.: it seems a bit
> odd that this is set up like that in the first place)?
>
> Thanks,
>
> Fabio
>
> p.s.: I'm asking because in a debugger which changes bytecode I want to
> keep the same semantics and it appears that if I add more bytecode at that
> instruction offset, those semantics aren't kept (but I don't really know
> what are the semantics to keep here since it seems like that instruction
> should issue a line event even though it doesn't).
>
Answering my own question after investigating:
It boils down to the way that ceval.c does prediction of bytecodes which
makes it miss the line.
i.e.: the compare is something as:
TARGET(COMPARE_OP): {
assert(oparg <= Py_GE);
PyObject *right = POP();
PyObject *left = TOP();
PyObject *res = PyObject_RichCompare(left, right, oparg);
SET_TOP(res);
Py_DECREF(left);
Py_DECREF(right);
if (res == NULL)
goto error;
PREDICT(POP_JUMP_IF_FALSE);
PREDICT(POP_JUMP_IF_TRUE);
DISPATCH();
}
Given that, PREDICT makes the "POP_JUMP_IF_FALSE" / "POP_JUMP_IF_TRUE"
line be ignored and its line becomes merged with the "COMPARE_OP" in the
tracing and when the bytecode is manipulated this is no longer true, so
those spurious line events start to be generated in the tracing.
So, in the end it boils down the the eval loop not respecting what's
written in the bytecode under some conditions -- in this particular case
it's probably good as the jump line seems to be off and I'd say it's a bug
in the bytecode generation which is then fixed by a bug in the PREDICT
ignoring that line that's off, so, the bugs nullify each other, but then,
my rewriting of bytecode makes the PREDICT fail and then the issue of
having the line off in the bytecode becomes apparent while tracing.
As a note, this is the 2nd time I've been bitten by PREDICT (
https://mail.python.org/archives/list/[email protected]/message/HYTVMG4YTU3QLRGEE4LMLGFANQPO4PD3/)...
Does someone know if with the introduction of the new
optimizations/quickening the PREDICT will still be used in 3.11? If there
are 2 interpretation modes now, the PREDICT probably makes it hard to have
both do the same things since the eval loop doesn't really match what the
bytecode says due to the PREDICT.
Thanks,
Fabio
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/W6YZXQAG4GMMXJ6GXLHAYFP4CCMMVEZF/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: python3.10rc2 compilation on android/termux/clang12.0.1 fails
I looked a bit more into this and managed to run python 3.10.0rc2 on
android in termux. No additional patches were needed, but the build
process isn't straight forward.
Some packages don't build on-device and have to be built on a computer.
Python is one of them
(https://github.com/termux/termux-packages/issues/4157. The issue is
closed because making all packages build on-device is no longer a goal
of termux)
Follow https://github.com/termux/termux-packages/wiki/Build-environment
to set up your termux build environment. You can use Docker, Vagrant,
Ubuntu or Arch Linux. I used docker.
Next build and install python3.10.0rc2 *on the build machine*. You need
python3.10 installed to cross-compile python3.10.
After that go back to the termux packages and change
packages/python/build.sh. All you need to change is version number, url
and hash. Here's a patch anyways:
diff --git a/packages/python/build.sh b/packages/python/build.sh
index c36bff5e5..d9fd86d02 100644
--- a/packages/python/build.sh
+++ b/packages/python/build.sh
@@ -2,10 +2,11 @@ TERMUX_PKG_HOMEPAGE=https://python.org/
TERMUX_PKG_DESCRIPTION="Python 3 programming language intended to
enable clear programs"
TERMUX_PKG_LICENSE="PythonPL"
TERMUX_PKG_MAINTAINER="@termux"
-_MAJOR_VERSION=3.9
-TERMUX_PKG_VERSION=${_MAJOR_VERSION}.7
-TERMUX_PKG_SRCURL=https://www.python.org/ftp/python/${TERMUX_PKG_VERSION}/Python-${TERMUX_PKG_VERSION}.tar.xz
-TERMUX_PKG_SHA256=f8145616e68c00041d1a6399b76387390388f8359581abc24432bb969b5e3c57
+_MAJOR_VERSION=3.10
+_MINOR_VERSION=.0
+TERMUX_PKG_VERSION=${_MAJOR_VERSION}${_MINOR_VERSION}rc2
+TERMUX_PKG_SRCURL=https://www.python.org/ftp/python/${_MAJOR_VERSION}${_MINOR_VERSION}/Python-${TERMUX_PKG_VERSION}.tar.xz
+TERMUX_PKG_SHA256=e75b56088548b7b9ad1f2571e6f5a2315e4808cb6b5fbe8288502afc802b2f24
TERMUX_PKG_DEPENDS="gdbm, libandroid-support, libbz2, libcrypt,
libffi, liblzma, libsqlite, ncurses, ncurses-ui-libs, openssl, readline,
zlib"
TERMUX_PKG_RECOMMENDS="clang, make, pkg-config"
TERMUX_PKG_SUGGESTS="python-tkinter"
Finally just run "./build-package.sh -i -f python" and send
"output/python*.deb" to your phone, where you can install it using dpkg -i.
Adrian
On 9/16/21 14:27, Adrian Freund wrote:
As you are using termux it might be worth checking out the build
arguments and patches termux uses to build their own version of python
(Currently 3.9.7):
https://github.com/termux/termux-packages/tree/master/packages/python
I'm not sure if this will be enough to build python3.10 or if
additional patches and build arguments are needed though.
Adrian
On 9/15/21 19:06, Sandeep Gupta wrote:
I am trying to compile Python3.10rc2 on rather unusual platform
(termux on android). The
gcc version is listed below:
~ $ g++ -v
clang version 12.0.1
Target: aarch64-unknown-linux-android24
Thread model: posix
InstalledDir: /data/data/com.termux/files/usr/bin
I get following warnings and errors:
Python/pytime.c:398:10: warning: implicit conversion from 'long' to
'double' changes value from 9223372036854775807 to
9223372036854775808 [-Wimplicit-const-int-float-conver
sion]
if (!_Py_InIntegralTypeRange(_PyTime_t, d)) {
Python/bootstrap_hash.c:141:17: error: implicit declaration of
function 'getrandom' is invalid in C99
[-Werror,-Wimplicit-function-declaration]
n = getrandom(dest, n, flags);
^
Python/bootstrap_hash.c:145:17: error: implicit declaration of
function 'getrandom' is invalid in C99
[-Werror,-Wimplicit-function-declaration]
n = getrandom(dest, n, flags);
Not sure if this is limitation of the platform or Python codebase
needs fixes.
Thanks
-S
___
Python-Dev mailing list [email protected]
To unsubscribe send an email [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived
athttps://mail.python.org/archives/list/[email protected]/message/KEURSMCLUVI7VPKM6M2VUV4JIW6FP66Z/
Code of Conduct:http://python.org/psf/codeofconduct/
___
Python-Dev mailing list [email protected]
To unsubscribe send an email [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived
athttps://mail.python.org/archives/list/[email protected]/message/WFKYODUXY3UHENII7U47XQPUIZQFJDG4/
Code of Conduct:http://python.org/psf/codeofconduct/___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/2V5AVBB4QDN2NSEUFBQJ4X2M2FC6IMUA/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: What's the rule for issuing line tracing numbers when using a tracing function?
> Does someone know if with the introduction of the new optimizations/quickening the PREDICT will still be used in 3.11? If there are 2 interpretation modes now, the PREDICT probably makes it hard to have both do the same things since the eval loop doesn't really match what the bytecode says due to the PREDICT. For your use case (tracing), the behavior should be the same as 3.10. PEP 659 and its current implementation doesn't run specialized bytecode when tracing and falls back to normal bytecode. The behavior of normal bytecode hasn't changed either (other than what's currently listed in the changelog). -KJ ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/QHNPOVCQNCXN33QVCKSQUTPQJZAJCCR7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2021-09-10 - 2021-09-17) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open7381 ( +5) closed 49587 (+63) total 56968 (+68) Open issues with patches: 2938 Issues opened (37) == #45170: tarfile missing cross-directory checking https://bugs.python.org/issue45170 opened by xiongpanju #45171: stacklevel handling in logging module is inconsistent https://bugs.python.org/issue45171 opened by joukewitteveen #45172: netbsd CAN protocol flags addition https://bugs.python.org/issue45172 opened by devnexen #45173: Remove configparser deprecations https://bugs.python.org/issue45173 opened by hugovk #45174: DragonflyBSD fix nis module build https://bugs.python.org/issue45174 opened by devnexen #45176: Many regtest failures on Windows with non-ASCII account name https://bugs.python.org/issue45176 opened by minghua #45178: Support for linking unnamed temporary files into filesystem on https://bugs.python.org/issue45178 opened by WGH #45183: Unexpected exception with zip importer https://bugs.python.org/issue45183 opened by ronaldoussoren #45184: Add `pop` function to remove context manager from (Async)ExitS https://bugs.python.org/issue45184 opened by andreash #45189: Drop the "list_frozen" command from _test_embed. https://bugs.python.org/issue45189 opened by eric.snow #45191: Error.__traceback__.tb_lineno is wrong https://bugs.python.org/issue45191 opened by nahco314 #45192: The tempfile._infer_return_type function cannot infer the type https://bugs.python.org/issue45192 opened by rekyungmin #45193: IDLE Show completions pop-up not working on Ubuntu Linux https://bugs.python.org/issue45193 opened by taleinat #45197: IDLE should suppress ValueError for list.remove() https://bugs.python.org/issue45197 opened by rhettinger #45198: __set_name__ documentation not clear about its usage with non- https://bugs.python.org/issue45198 opened by xitop #45199: IDLE: document search (find) and replace better https://bugs.python.org/issue45199 opened by terry.reedy #45200: test_multiprocessing_fork failws with timeout https://bugs.python.org/issue45200 opened by terry.reedy #45201: API function PySignal_SetWakeupFd is not exported and unusable https://bugs.python.org/issue45201 opened by cy #45203: Improve specialization stats for LOAD_METHOD and BINARY_SUBSC https://bugs.python.org/issue45203 opened by Mark.Shannon #45206: test_contextlib_async logs "Task was destroyed but it is pendi https://bugs.python.org/issue45206 opened by vstinner #45209: multiprocessing tests log: UserWarning: resource_tracker: Ther https://bugs.python.org/issue45209 opened by vstinner #45210: tp_dealloc docs should mention error indicator may be set https://bugs.python.org/issue45210 opened by ezyang #45211: Useful (expensive) information is discarded in getpath.c. https://bugs.python.org/issue45211 opened by eric.snow #45213: Frozen modules are looked up using a linear search. https://bugs.python.org/issue45213 opened by eric.snow #45214: implement LOAD_NONE opcode https://bugs.python.org/issue45214 opened by iritkatriel #45215: Add docs for Mock name and parent args and deprecation warning https://bugs.python.org/issue45215 opened by andrei.avk #45216: Remove redundand information from difflib docstrings https://bugs.python.org/issue45216 opened by serhiy.storchaka #45218: cmath.log has an invalid signature https://bugs.python.org/issue45218 opened by mark.dickinson #45221: Linker flags starting with -h breaks setup.py (regression) https://bugs.python.org/issue45221 opened by ux #45222: test_venv: test_with_pip fails on version 3.8.12 with: pip._ve https://bugs.python.org/issue45222 opened by sxt1001 #45223: test_spawn_doesnt_hang (test.test_pty.PtyTest) fails when stdi https://bugs.python.org/issue45223 opened by Alexander Kanavin #45226: Misleading error message when pathlib.Path.symlink_to() fails https://bugs.python.org/issue45226 opened by dfntlymaybe #45229: Always use unittest for collecting tests in regrtests https://bugs.python.org/issue45229 opened by serhiy.storchaka #45231: make regen-all changes files on Linux https://bugs.python.org/issue45231 opened by vstinner #45233: Allow split key dictionaries with values owned by other object https://bugs.python.org/issue45233 opened by Mark.Shannon #45234: copy_file raises FileNotFoundError when src is a directory https://bugs.python.org/issue45234 opened by Alex Grund #45235: argparse does not preserve namespace with subparser defaults https://bugs.python.org/issue45235 opened by ALSchwalm Most recent 15 issues with no replies (15) == #45235: argparse does not preserve namespace with subparser defaults https://bugs.python.org/issue45235 #45226: Misleading error message when pathlib.Path.symlink_to() fail
