zrthxn created this revision. Herald added a project: All. zrthxn requested review of this revision. Herald added a project: LLDB. Herald added a subscriber: lldb-commits.
Repository: rG LLVM Github Monorepo https://reviews.llvm.org/D121942 Files: lldb/docs/intel_pt.rst lldb/source/Plugins/Trace/intel-pt/README.md
Index: lldb/source/Plugins/Trace/intel-pt/README.md =================================================================== --- lldb/source/Plugins/Trace/intel-pt/README.md +++ /dev/null @@ -1,90 +0,0 @@ -## Instructions to build LLDB with Intel® PT Support - -### Before you get started - -1. Confirm that your processor is an Intel® x86_64 based processor. - -2. Check for the existance of this particular file on your Linux system - ```bash - cat /sys/bus/event_source/devices/intel_pt/type - ``` - The output of this should be a number greater than ...? - -### Build Instructions - -1. Clone [LibIPT library](https://github.com/intel/libipt). The Intel® Processor Trace Decoder Library is Intel's® reference implementation for decoding Intel® PT. - ```bash - git clone g...@github.com:intel/libipt.git - ``` - -2. Clone [the LLVM Project](https://github.com/llvm/llvm-project) in its entirety from Github or Phabricator. - ```bash - git clone g...@github.com:llvm/llvm-project.git - ``` - -3. Create build directories for both of these projects, outside the repositories. - ```bash - mkdir lldb-build libipt-build - ``` - -4. Start by building the libipt library first. LibIPT uses CMake and Make so *make* sure you have those. From here we will use `<lldb-build>` and `<libipt-build>` to represent the full paths of these directories. Change this in the following commands to the full path for your system. - ```bash - cmake -S libipt -B <libipt-build> - ``` - Run make in the LibIPT build directory - ```bash - cd <libipt-build> - make - cd .. - ``` - This will generate a few files in the `<libipt-build>/lib` and `<libipt-build>/libipt/include` directories. - -5. Now it is time to build LLDB and link the LibIPT build and header files into it. Start by configuring all the build files for LLDB from LLVM. - ```bash - cmake \ - -B lldb-build \ - -G Ninja \ - -DLLVM_ENABLE_PROJECTS="clang;libcxx;lldb;libcxxabi" \ - -DLLDB_BUILD_INTEL_PT=ON \ - -DLIBIPT_INCLUDE_PATH="<libipt-build>/libipt/include" \ - -DLIBIPT_LIBRARY_PATH="<libipt-build>/lib" \ - -S llvm-project/llvm - ``` - If this step goes right, you should see no errors and **no warnings for unused variables.** - We have now configured the LLDB build to be built with Ninja. Make sure you have `ninja` installed. - -6. Build LLDB. This will take a while. - ```bash - cd <lldb-build> - ninja lldb lldb-server - ``` - -7. When the build completes after a few decades, test it by running the version you just built. - ```bash - ./bin/lldb <program> - ``` - Inside LLDB, set a breakpoint and start the process runnning - ```lldb - (lldb) break main - (lldb) run - ``` - Now, if everything went well, when you run the following command, you should see no errors. - ```lldb - (lldb) process trace start - ``` - You should also be able to see the instructions for any line. - ```lldb - (lldb) next - (lldb) thread trace dump instructions - ``` - -Congratulations, you can now trace away! - -### References - -Some details about how Intel® Processor Trace works are in [this great blog post](https://engineering.fb.com/2021/04/27/developer-tools/reverse-debugging/). -Other things about how LLDB works with this are included in [the RFC document](https://docs.google.com/document/d/1cOVTGp1sL_HBXjP9eB7qjVtDNr5xnuZvUUtv43G5eVI/edit#) for this. - -- https://easyperf.net/blog/2019/08/23/Intel-Processor-Trace -- https://easyperf.net/blog/2019/08/30/Intel-PT-part2#appendix-how-to-build-gdb-with-intel-pt-support -- https://reviews.llvm.org/D91679 \ No newline at end of file Index: lldb/docs/intel_pt.rst =================================================================== --- /dev/null +++ lldb/docs/intel_pt.rst @@ -0,0 +1,210 @@ +Tracing with Intel Processor Trace (Intel PT) +============================================= + +.. contents:: + :local: + +Intel PT is a technology available in modern Intel CPUs that allows efficient +tracing of all the instructions executed by a process. LLDB can collect traces and dump +them using its symbolication stack. You can read more here https://easyperf.net/blog/2019/08/23/Intel-Processor-Trace. + +Prerequisites +------------- + +Confirm that your CPU supports Intel PT (see https://www.intel.com/content/www/us/en/support/articles/000056730/processors.html) +and that your operating system is Linux. + +Check for the existance of this particular file on your Linux system +:: + + $ cat /sys/bus/event_source/devices/intel_pt/type + +The output should be a number. Otherwise, try upgrading your kernel. + + +Build Instructions +------------------ + +Clone and build the low level Intel PT decoder library [LibIPT library](https://github.com/intel/libipt). +:: + + $ git clone g...@github.com:intel/libipt.git + $ mkdir libipt-build + $ cmake -S libipt -B libipt-build + $ cd libipt-build + $ make + +This will generate a few files in the `<libipt-build>/lib` and `<libipt-build>/libipt/include` directories. + +Configure and build LLDB with Intel PT support +:: + + $ cmake \ + -DLLDB_BUILD_INTEL_PT=ON \ + -DLIBIPT_INCLUDE_PATH="<libipt-build>/libipt/include" \ + -DLIBIPT_LIBRARY_PATH="<libipt-build>/lib" \ + ... other common configuration parameters + +If this step goes correctly, you should see no errors and **no warnings for unused variables.** +:: + + $ cd lldb-build + $ ninja lldb lldb-server # if using Ninja + +How to Use +---------- + +When you are debugging a process, you can turn on intel-pt tracing, which will ârecordâ all the instructions +that the process will execute. After turning it on, you can continue debugging, and at any breakpoint, +you can inspect the instruction list. + +For example: +:: + lldb <target> + > b main + > run + > process trace start # start tracing on all threads, including future ones + # keep debugging until you hit a breakpoint + + > thread trace dump instructions + # this should output something like + + thread #2: tid = 2861133, total instructions = 5305673 + libc.so.6`__GI___libc_read + 45 at read.c:25:1 + [4962255] 0x00007fffeb64c63d subq $0x10, %rsp + [4962256] 0x00007fffeb64c641 movq %rdi, -0x18(%rbp) + libc.so.6`__GI___libc_read + 53 [inlined] __libc_read at read.c:26:10 + [4962257] 0x00007fffeb64c645 callq 0x7fffeb66b640 ; __libc_enable_asynccancel + libc.so.6`__libc_enable_asynccancel + [4962258] 0x00007fffeb66b640 movl %fs:0x308, %eax + libc.so.6`__libc_enable_asynccancel + 8 + [4962259] 0x00007fffeb66b648 movl %eax, %r11d + + # you can keep pressing ENTER to see more and more instructions + +The number between brackets is the instruction index, and by default the current thread will be picked. + +Configuring the trace size +-------------------------- + +The CPU stores the instruction list in a compressed format in a ring buffer, which keeps the latest information. By default, LLDB uses a buffer of 4KB per thread, but you can change it by running. The size must be a power of 2 and at least 4KB. +:: + thread trace start all -s <size_in_bytes> + +For reference, a 1MB trace buffer can easily store around 5M instructions. + +Printing more instructions +-------------------------- + +If you want to dump more instructions at a time, you can run +:: + thread trace dump instructions -c <count> + +Printing the instructions of another thread +------------------------------------------- + +By default the current thread will be picked when dumping instructions, but you can do +:: + thread trace dump instructions <#thread index> + #e.g. + thread trace dump instructions 8 + +to select another thread. + +Crash Analysis +-------------- + +What if you are debugging + tracing a process that crashes? Then you can just do +:: + thread trace dump instructions + +To inspect how it crashed! There's nothing special that you need to do. For example + + * thread #1, name = 'a.out', stop reason = signal SIGFPE: integer divide by zero + frame #0: 0x00000000004009f1 a.out`main at main.cpp:8:14 + 6 int x; + 7 cin >> x; + -> 8 cout << 12 / x << endl; + 9 return 0; + 10 } + (lldb) thread trace dump instructions -c 5 + thread #1: tid = 604302, total instructions = 8388 + libstdc++.so.6`std::istream::operator>>(int&) + 181 + [8383] 0x00007ffff7b41665 popq %rbp + [8384] 0x00007ffff7b41666 retq + a.out`main + 66 at main.cpp:8:14 + [8385] 0x00000000004009e8 movl -0x4(%rbp), %ecx + [8386] 0x00000000004009eb movl $0xc, %eax + [8387] 0x00000000004009f0 cltd + +.. note:: + at this moment, we are not including the failed instruction in the trace, but in the future we might do it for readability. + + +Offline Trace Analysis +---------------------- + +It's also possible to record a trace using a custom Intel PT collector and decode + symbolicate the trace using LLDB. For that, the command trace load is useful. +In order to use trace load, you need to first create a JSON file with the definition of the trace session. For example +:: + { + "trace": { + "type": "intel-pt", + "pt_cpu": { + "vendor": "intel", + "family": 6, + "model": 79, + "stepping": 1 + } + }, + "processes": [ + { + "pid": 815455, + "triple": "x86_64-*-linux", + "threads": [ + { + "tid": 815455, + "traceFile": "trace.file" # raw thread-specific trace from the AUX buffer + } + ], + "modules": [ # this are all the shared libraries + the main executable + { + "file": "a.out", # optional if it's the same as systemPath + "systemPath": "a.out", + "loadAddress": "0x0000000000400000", + }, + { + "file": "libfoo.so", + "systemPath": "/usr/lib/libfoo.so", + "loadAddress": "0x00007ffff7bd9000", + }, + { + "systemPath": "libbar.so", + "loadAddress": "0x00007ffff79d7000", + } + ] + } + ] + } + +You can see the full schema by typing +:: + trace schema intel-pt + +The JSON file mainly contains all the shared libraries that were part of the traced process, along with their memory load address. +If the analysis is done on the same computer where the traces were obtained, it's enough to use the âsystemPathâ field. +If the analysis is done on a different machines, these files need to be copied over and the âfileâ field should point to the +location of the file relative to the JSON file. +Once you have the JSON file and the module files in place, you can simple run +:: + lldb + > trace load /path/to/json + > thread trace dump instructions <optional thread index> + +Then it's like in the live session case + +References +---------- + +Original RFC document [the RFC document](https://docs.google.com/document/d/1cOVTGp1sL_HBXjP9eB7qjVtDNr5xnuZvUUtv43G5eVI) for this. +Some details about how Meta is using Intel Processor Trace can be found in this [blog post](https://engineering.fb.com/2021/04/27/developer-tools/reverse-debugging/).
_______________________________________________ lldb-commits mailing list lldb-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
