On Mon, Aug 26, 2024 at 02:04:18PM +0200, Roland Hieber wrote: > * Split off the "Running QEMU" section into a stand-alone section > * Mention /usr/lib/debug instead of the old ./debug/ folders > * Describe `ptxdist gdb` and debugging on the target directly > * Keep GDB usage short; refer to third-party documentation > * Use literal blocks instead of code blocks without highlighting > * Make use of the |ptxdistPlatformDir| template variable > > Signed-off-by: Roland Hieber <[email protected]> > --- > PATCH v2: > * somehow I forgot to stage & amend various changes before sending v1, > please disregard the old one > > > doc/daily_work.inc | 226 ++++++++++++++++++++++++++++++++++----------- > 1 file changed, 174 insertions(+), 52 deletions(-) > > diff --git a/doc/daily_work.inc b/doc/daily_work.inc > index b98adf75197d..65b0bad25d97 100644 > --- a/doc/daily_work.inc > +++ b/doc/daily_work.inc > @@ -285,15 +285,11 @@ nonexisting file, we can limit the output to all > ``open`` system calls: > The binary may fail due to a missing ``/etc/foo.conf``. This could be a > hint on what is going wrong (it might not be the final solution). > > -Debugging with CPU emulation > ----------------------------- > - > -If we do not need some target related feature to run our application, we > -can also debug it through a simple CPU emulation. Thanks to QEMU we can > -run ELF binaries for other architectures than our build host is. > +Running an Application Made for a Different Architecture > +-------------------------------------------------------- > > -Running an Application made for a different Architecture > -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > +Thanks to QEMU we can run ELF binaries for different architectures than our > +build host. > > PTXdist creates a fully working root filesystem with all run-time > components in ``root/``. Lets assume we made a PTXdist based project for > @@ -318,64 +314,134 @@ host’s operating system. Using QEMU in this way let us > simply check our > programs. There are also QEMU environments for other architectures > available. > > -Debugging an Application made for a different Architecture > -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > - > -Debugging our application is also possible with QEMU. All we need are a > -root filesystem with debug symbols available, QEMU and an architecture > -aware debugger. > - > -The root filesystem with debug symbols will be provided by PTXdist, the > -architecture aware debugger comes with the OSELAS.Toolchain. Two > -consoles are required for this debug session in this example. We start > -the QEMU in the first console as: > - > -.. code-block:: text > - > - $ cd ptxdistPlatformDir/root > - ptxdistPlatformDir/root$ qemu-<architecture> -g 1234 -cpu <cpu-core> -L > . usr/bin/myapp > +Debugging an Application Made for a Different Architecture > +---------------------------------------------------------- > + > +Debugging an application is possible in three ways: > + > +#. :ref:`Directly running GDB on the target hardware <gdb_on_target>`: > + This approach has the advantage that the target system is > "self-contained", > + but it is necessary to install the debug symbols in the target's root file > + system, which can significantly improve the memory footprint. Furthermore,
"improve" sounds wrong here. Maybe you meant "increase"? > + it is often not possible for GDB to access the application's source code > + unless further steps are taken (e.g. making the source code available in > the > + correct paths over an NFS share or from a USB drive). > + > +#. :ref:`Running the application in a QEMU emulating the target architecture > <gdb_via_qemu>`: > + This approach doesn't require any additional hardware and can therefore be > + done completely on the build host, but it is limited to applications > + that do not depend on specific behaviour of the target hardware, like > + using external interfaces or architecture-specific features that are not > + available via QEMU (e.g. accessing GPS hardware over UART, or graphics > + acceleration). > + > +#. :ref:`Running gdbserver on the target hardware, and connecting to it with > GDB > + from the build host <ptxdist_gdb>`: > + This is the simplest use case when a target hardware with network > interfaces > + is available, and PTXdist includes native support for it. > + In this case, the target file system only has to include the *gdbserver* > + binary, but does not need to contain any further debug information or > source > + code, which are supplied by the build host. > + > +In any case, we need the respective binaries including their debug > information, > +and a debugger for the target architecture. > +The root filesystem with debug symbols is provided by PTXdist in > +``|ptxdistPlatformDir|/root/`` on the build host, and the architecture-aware > debugger > +comes with the OSELAS.Toolchain. > + > +.. note:: When PTXdist builds a target root file system, the binaries that > are > + installed into it during the *targetinstall* stage of each package are > + always stripped of their debug symbols. Instead, the debug symbols are > + installed under ``|ptxdistPlatformDir|/root/usr/lib/debug/`` in separate > files named > + after the *build ID* of the binary (which is a checksum over the contents > of > + the binary, and can change when a binary is rebuilt). > + > + When GDB loads a binary, it will look for and automatically load matching > + debug symbols in ``usr/lib/debug`` relative to its *sysroot*, which is > + usually ``/``, but can be configured at runtime. > + > +.. note:: Information on using GDB is not part of this manual. > + There are countless cheatsheets available on the internet, but in most > + cases you will get around sufficiently with the ``break > <location|function>``, > + ``run``, ``next``, ``step``, ``finish``, ``continue``, ``print > <statement|address>``, > + ``list <location|function>`` and ``quit`` commands. > + Current GDB versions also include a well-structured online help with the > + ``help`` command, which can show you how to use each command. > + > +.. _gdb_on_target: > + > +Running GDB on the Target Hardware > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +In this case, GDB and the debug information need to be available on the > target hardware. > +GDB can be enabled in the *menuconfig* with the ``PTXCONF_GDB`` symbol. > +Additionally, the debug symbols are installed into the target root file > system > +when the ``PTXCONF_DEBUG_PACKAGES_INSTALL`` option is enabled. > +(If an image has been built before, i.e. the *targetinstall* stages have > +already been executed, a ``ptxdist clean root`` followed by a ``ptxdist go`` > or > +``ptxdist images`` might be needed to install all debug symbols.) > + > +Due to the size of the resulting root file system, the target root file > system > +is often supplied via :ref:`NFSROOT <nfsroot>`. > + > +On the target, GDB can then be used normally (with the caveat that the > source files are > +not available unless supplied manually):: > + > + root@target $ gdb /usr/bin/systemctl > + Reading symbols from /usr/bin/systemctl... > + Reading symbols from > /usr/lib/debug/.build-id/fb/3b770e637b51276dcdc41a1e8b37df8a6d6686.debug... > + (gdb) break main > + Breakpoint 1 at 0x6c90: file ../systemd-256.1/src/systemctl/systemctl.c, > line 1372. > + (gdb) run > + Starting program: /usr/bin/systemctl > + [Thread debugging using libthread_db enabled] > + Using host libthread_db library "/lib/libthread_db.so.1". > + > + Breakpoint 1, main (argc=1, argv=0xbee9bd64) > + at ../systemd-256.1/src/systemctl/systemctl.c:1372 > + 1372 in ../systemd-256.1/src/systemctl/systemctl.c > + (gdb) backtrace > + #0 main (argc=1, argv=0xbee9bd64) > + at ../systemd-256.1/src/systemctl/systemctl.c:1372 > + (gdb) list > + 1372 ../systemd-256.1/src/systemctl/systemctl.c: No such file or > directory. > + (gdb) quit > + > +.. _gdb_via_qemu: > + > +Debugging with CPU Emulation > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +If we do not need target-specific features to run our application, we can > also > +debug it through CPU emulation. > +Two consoles are required for this debug session in this example. We start > +the QEMU in the first console as:: > > -.. note:: PTXdist always builds a root filesystem ``root/``. > - It contains all components without debug > - information (all binaries are in the same size as used later on on the > - real target). In addition, each directory that contains binaries also > - contains a ``.debug/`` directory. It contains a file with only the debug > - symbols for each binary. These files are ignored while running > - applications but GDB knows about it and will automatically load the debug > - files. > + $ cd |ptxdistPlatformDir|/root > + |ptxdistPlatformDir|/root$ qemu-<architecture> -g 1234 -cpu <cpu-core> > -L . usr/bin/myapp > > The added *-g 1234* parameter lets QEMU wait for a GDB connection to run > the application. > > In the second console we start GDB with the correct architecture > support. This GDB comes with the same OSELAS.Toolchain that was also > -used to build the project: > - > -.. code-block:: text > +used to build the project:: > > - $ ./selected_toolchain/<target>-gdb --tui > platform-<platformname>/root/usr/bin/myapp > - > -This will run a *curses* based GDB. Not so easy to handle (we must enter > -all the commands and cannot click with a mouse!), but very fast to take > -a quick look at our application. > + $ ./selected_toolchain/<target>-gdb > |ptxdistPlatformDir|/root/usr/bin/myapp > > At first we tell GDB where to look for debug symbols. The correct > -directory here is ``root/``. > - > -.. code-block:: text > +directory here is ``|ptxdistPlatformDir|/root/``:: > > - (gdb) set solib-absolute-prefix platform-<platformname>/root > + (gdb) set solib-absolute-prefix |ptxdistPlatformDir|/root This part of the documentation is pretty old and a bit outdated. Can you change it to use 'ptxdist gdb'? It should possible to use it in this case and then manually setting 'solib-absolute-prefix' is not needed. Michael > > -Next we connect this GDB to the waiting QEMU: > - > -.. code-block:: text > +Next we connect this GDB to the waiting QEMU:: > > (gdb) target remote localhost:1234 > Remote debugging using localhost:1234 > [New Thread 1] > 0x40096a7c in _start () from root/lib/ld.so.1 > > -As our application is already started, we can’t use the GDB command > +As our application is already started via QEMU, we can’t use the GDB command > ``start`` to run it until it reaches ``main()``. We set a breakpoint > instead at ``main()`` and *continue* the application: > > @@ -387,10 +453,6 @@ instead at ``main()`` and *continue* the application: > Continuing. > Breakpoint 1, main (argc=1, argv=0x4007f03c) at myapp.c:644 > > -The top part of the running gdbtui console will always show us the > -current source line. Due to the ``root/`` directory usage all > -debug information for GDB is available. > - > Now we can step through our application by using the commands *step*, > *next*, *stepi*, *nexti*, *until* and so on. > > @@ -403,6 +465,66 @@ Now we can step through our application by using the > commands *step*, > debug symbols kept, it will be also possible for GDB to debug C library > functions our application calls (so it might worth the disk space). > > +.. _ptxdist_gdb: > + > +Remote-Debugging with *gdbserver* > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +When debugging with *gdbserver*, the debugging session is split into two > parts: > +the server runs on the target and is remote-controlled by a GDB instance > +running on the build host over a network connection. > +You need to enable the ``PTXCONF_GDBSERVER`` symbol in PTXdists's > *menuconfig* > +which makes *gdbserver* available on your target. > + > +Like for the approach above, you will need two terminals. > +In the first terminal you connect to your running target (e.g. via SSH or > +serial console), start *gdbserver* and tell it the binary to debug and where > +to listen for connections:: > + > + root@target $ gdbserver 192.168.1.11:1234 /usr/bin/systemctl > + Process /usr/bin/systemctl created; pid = 477 > + Listening on port 1234 > + > +In the above example, 192.168.1.11 is the outside-facing IP address of the > +target, and 1234 is the port on which to listen. > +gdbserver has now created a process, and waits for connections from a GDB. > + > +In the other terminal, you then start ``ptxdist gdb``, which will set up GDB > +correctly so that it can find all necessary debug symbols in PTXdist's > +``|ptxdistPlatformDir|/root/`` folder, and the sources in > +``|ptxdistPlatformDir|/sysroot-target/``. > +You then connect to the gdbserver instance running on the target using GDB's > +*target remote* command:: > + > + user@buildhost $ ptxdist gdb > + (gdb) target remote 192.168.9.10:1234 > + Remote debugging using 192.168.9.10:1234 > + Reading symbols from > …/my-bsp/|ptxdistPlatformDir|/root/usr/bin/systemctl... > + Reading symbols from > …/my-bsp/|ptxdistPlatformDir|/root/usr/lib/debug/.build-id/fb/3b770e637b51276dcdc41a1e8b37df8a6d6686.debug... > + Reading symbols from > …/my-bsp/|ptxdistPlatformDir|/root/lib/ld-linux-armhf.so.3... > + Reading symbols from > …/my-bsp/|ptxdistPlatformDir|/root/usr/lib/debug/.build-id/0d/a8d3736dcd31fbf43ea0cb79d1f9699c6695ff.debug... > + 0xb6f29624 in _start () from > …/my-bsp/|ptxdistPlatformDir|/root/lib/ld-linux-armhf.so.3 > + (gdb) break main > + Breakpoint 1 at 0x4e3c90: file > ../systemd-256.1/src/systemctl/systemctl.c, line 1372. > + (gdb) continue > + Continuing. > + > + Breakpoint 1, main (argc=1, argv=0xbe829d84) at > ../systemd-256.1/src/systemctl/systemctl.c:1372 > + 1372 DEFINE_MAIN_FUNCTION_WITH_POSITIVE_FAILURE(run); > + (gdb) list > + 1367 > + 1368 /* Note that we return r here, not 0, so that we can > implement the LSB-like return codes */ > + 1369 return r; > + 1370 } > + 1371 > + 1372 DEFINE_MAIN_FUNCTION_WITH_POSITIVE_FAILURE(run); > + 1373 #endif > + (gdb) > + > +You can now use GDB as usual, and when the debugged process terminates (or > you > +quit the GDB session), *gdbserver* will terminate as well, and close the > +connection. > + > Migration between Releases > -------------------------- > > -- > 2.39.2 > > > -- Pengutronix e.K. | | Steuerwalder Str. 21 | http://www.pengutronix.de/ | 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
