Re: [lldb-dev] [RFC] Fast Conditional Breakpoints (FCB)
> On Aug 16, 2019, at 11:13 AM, Ismail Bennani via lldb-dev > wrote: > > Hi Pavel, > > Thanks for all your feedbacks. > > I’ve been following the discussion closely and find your approach quite > interesting. > > As Jim explained, I’m also trying to have a conditional breakpoint, that is > able to stop a specific thread (name or id) when the condition expression > evaluates to true. > > I feel like stacking up options with your approach would imply doing more > context switches. > But it’s definitely a better fallback mechanism than the current one. I’ll > try to make a prototype to see the performance difference for both approaches. > > >> On Aug 15, 2019, at 10:10 AM, Pavel Labath wrote: >> >> Hello Ismail, and wellcome to LLDB. You have a very interesting (and not >> entirely trivial) project, and I wish you the best of luck in your work. I >> think this will be a very useful addition to lldb. >> >> It sounds like you have researched the problem very well, and the overall >> direction looks good to me. However, I do have some ideas suggestions about >> possible tweaks/improvements that I would like to hear your thoughts on. >> Please find my comments inline. >> >> On 14/08/2019 22:52, Ismail Bennani via lldb-dev wrote: >>> Hi everyone, >>> I’m Ismail, a compiler engineer intern at Apple. As a part of my internship, >>> I'm adding Fast Conditional Breakpoints to LLDB, using code patching. >>> Currently, the expressions that power conditional breakpoints are lowered >>> to LLVM IR and LLDB knows how to interpret a subset of it. If that fails, >>> the debugger JIT-compiles the expression (compiled once, and re-run on each >>> breakpoint hit). In both cases LLDB must collect all program state used in >>> the condition and pass it to the expression. >>> The goal of my internship project is to make conditional breakpoints faster >>> by: >>> 1. Compiling the expression ahead-of-time, when setting the breakpoint and >>> inject into the inferior memory only once. >>> 2. Re-route the inferior execution flow to run the expression and check >>> whether >>> it needs to stop, in-process. >>> This saves the cost of having to do the context switch between debugger and >>> the inferior program (about 10 times) to compile and evaluate the condition. >>> This feature is described on the [LLDB Project >>> page](https://lldb.llvm.org/status/projects.html#use-the-jit-to-speed-up-conditional-breakpoint-evaluation). >>> The goal would be to have it working for most languages and architectures >>> supported by LLDB, however my original implementation will be for C-based >>> languages targeting x86_64. It will be extended to AArch64 afterwards. >>> Note the way my prototype is implemented makes it fully extensible for other >>> languages and architectures. >>> ## High Level Design >>> Every time a breakpoint that holds a condition is hit, multiple context >>> switches are needed in order to compile and evaluate the condition. >>> First, the breakpoint is hit and the control is given to the debugger. >>> That's where LLDB wraps the condition expression into a UserExpression that >>> will get compiled and injected into the program memory. Another round-trip >>> between the inferior and the LLDB is needed to run the compiled expression >>> and extract the expression results that will tell LLDB to stop or not. >>> To get rid of those context switches, we will evaluate the condition inside >>> the program, and only stop when the condition is true. LLDB will achieve >>> this >>> by inserting a jump from the breakpoint address to a code section that will >>> be allocated into the program memory. It will save the thread state, run the >>> condition expression, restore the thread state and then execute the copied >>> instruction(s) before jumping back to the regular program flow. >>> Then we only trap and return control to LLDB when the condition is true. >>> ## Implementation Details >>> To be able to evaluate a breakpoint condition without interacting with the >>> debugger, LLDB changes the inferior program execution flow by overwriting >>> the instruction at which the breakpoint was set with a branching >>> instruction. >>> The original instruction(s) are copied to a memory stub allocated in the >>> inferior program memory called the __Fast Conditional Breakpoint >>> Trampoline__ >>> or __FCBT__. The FCBT will allow us the re-route the program execution flow >>> to >>> check the condition in-process while preserving the original program >>> behavior. >>> This part is critical to setup Fast Conditional Breakpoints. >>> ``` >>> Inferior Binary Trampoline >>> |.| +-+ >>> |.| | | >>> |.| +->+ Save RegisterContext | >>> |.| | |
Re: [lldb-dev] [RFC] Fast Conditional Breakpoints (FCB)
> On Aug 19, 2019, at 2:30 PM, Frédéric Riss wrote: > > > >> On Aug 16, 2019, at 11:13 AM, Ismail Bennani via lldb-dev >> wrote: >> >> Hi Pavel, >> >> Thanks for all your feedbacks. >> >> I’ve been following the discussion closely and find your approach quite >> interesting. >> >> As Jim explained, I’m also trying to have a conditional breakpoint, that is >> able to stop a specific thread (name or id) when the condition expression >> evaluates to true. >> >> I feel like stacking up options with your approach would imply doing more >> context switches. >> But it’s definitely a better fallback mechanism than the current one. I’ll >> try to make a prototype to see the performance difference for both >> approaches. >> >> >>> On Aug 15, 2019, at 10:10 AM, Pavel Labath wrote: >>> >>> Hello Ismail, and wellcome to LLDB. You have a very interesting (and not >>> entirely trivial) project, and I wish you the best of luck in your work. I >>> think this will be a very useful addition to lldb. >>> >>> It sounds like you have researched the problem very well, and the overall >>> direction looks good to me. However, I do have some ideas suggestions about >>> possible tweaks/improvements that I would like to hear your thoughts on. >>> Please find my comments inline. >>> >>> On 14/08/2019 22:52, Ismail Bennani via lldb-dev wrote: Hi everyone, I’m Ismail, a compiler engineer intern at Apple. As a part of my internship, I'm adding Fast Conditional Breakpoints to LLDB, using code patching. Currently, the expressions that power conditional breakpoints are lowered to LLVM IR and LLDB knows how to interpret a subset of it. If that fails, the debugger JIT-compiles the expression (compiled once, and re-run on each breakpoint hit). In both cases LLDB must collect all program state used in the condition and pass it to the expression. The goal of my internship project is to make conditional breakpoints faster by: 1. Compiling the expression ahead-of-time, when setting the breakpoint and inject into the inferior memory only once. 2. Re-route the inferior execution flow to run the expression and check whether it needs to stop, in-process. This saves the cost of having to do the context switch between debugger and the inferior program (about 10 times) to compile and evaluate the condition. This feature is described on the [LLDB Project page](https://lldb.llvm.org/status/projects.html#use-the-jit-to-speed-up-conditional-breakpoint-evaluation). The goal would be to have it working for most languages and architectures supported by LLDB, however my original implementation will be for C-based languages targeting x86_64. It will be extended to AArch64 afterwards. Note the way my prototype is implemented makes it fully extensible for other languages and architectures. ## High Level Design Every time a breakpoint that holds a condition is hit, multiple context switches are needed in order to compile and evaluate the condition. First, the breakpoint is hit and the control is given to the debugger. That's where LLDB wraps the condition expression into a UserExpression that will get compiled and injected into the program memory. Another round-trip between the inferior and the LLDB is needed to run the compiled expression and extract the expression results that will tell LLDB to stop or not. To get rid of those context switches, we will evaluate the condition inside the program, and only stop when the condition is true. LLDB will achieve this by inserting a jump from the breakpoint address to a code section that will be allocated into the program memory. It will save the thread state, run the condition expression, restore the thread state and then execute the copied instruction(s) before jumping back to the regular program flow. Then we only trap and return control to LLDB when the condition is true. ## Implementation Details To be able to evaluate a breakpoint condition without interacting with the debugger, LLDB changes the inferior program execution flow by overwriting the instruction at which the breakpoint was set with a branching instruction. The original instruction(s) are copied to a memory stub allocated in the inferior program memory called the __Fast Conditional Breakpoint Trampoline__ or __FCBT__. The FCBT will allow us the re-route the program execution flow to check the condition in-process while preserving the original program behavior. This part is critical to setup Fast Conditional Breakpoints. ``` Inferior Binary Trampoline |.| +-+ |.|
Re: [lldb-dev] [patch] char8_t support (plus dlang UTF8/16/32)
Hi James,
Thanks for working on this. I've opened a code review for your patch:
https://reviews.llvm.org/D66447
I've had to make some modification for it to compile and added a test.
Cheers,
Jonas
On Sun, Aug 18, 2019 at 6:34 PM James Blachly via lldb-dev
wrote:
>
> Dear LLDB developers:
>
> I have added support for C++20 char8_t, as well as support for dlang's
> char/wchar/dchar types. As I am not a professional developer, and the
> submission-review-merge process for LLVM projects seems somewhat byzantine, I
> wanted to offer this up on the list in the hopes that others find it useful
> and someone will be able to integrate it.
>
> kind regards
> James
>
>
> Using an example program that defines each of the unicode types as a single
> character as well as string, we see a major improvement.
>
> BEFORE:
> (lldb) frame v
> error: need to add support for DW_TAG_base_type 'char8_t' encoded with DW_ATE
> = 0x10, bit_size = 8
> (void) c8 =
>
> (char16_t) c16 = U+ u'\0'
> (char32_t) c32 = U+0x7fff U'翿'
> (void [11]) str8 = ([0] = , [1] = determine byte size.>, [2] = , [3] = to determine byte size.>, [4] = , [5] =
> , [6] = , [7]
> = , [8] = ,
> [9] = , [10] = size.>)
> (void *) str8ptr = 0x7fffe3d9
> (char16_t [12]) str16 = u"Hello UTF16"
> (char16_t *) str16ptr = 0x7fffe3b0 u"Hello UTF16"
> (char32_t [12]) str32 = U"Hello UTF32"
> (char32_t *) str32ptr = 0x7fffe370 U"Hello UTF32"
>
> AFTER:
> (lldb) frame v
> (char8_t) c8 = 0x00 u8'\0'
> (char16_t) c16 = U+ u'\0'
> (char32_t) c32 = U+0x7fff U'翿'
> (char8_t [11]) str8 = u8"Hello UTF8"
> (char8_t *) str8ptr = 0x7fffe3c9 u8"Hello UTF8"
> (char16_t [12]) str16 = u"Hello UTF16"
> (char16_t *) str16ptr = 0x7fffe3a0 u"Hello UTF16"
> (char32_t [12]) str32 = U"Hello UTF32"
> (char32_t *) str32ptr = 0x7fffe360 U"Hello UTF32”
>
>
>
>
> diff --git a/include/lldb/lldb-enumerations.h
> b/include/lldb/lldb-enumerations.h
> index f9830c04b..e7189dc9d 100644
> --- a/include/lldb/lldb-enumerations.h
> +++ b/include/lldb/lldb-enumerations.h
> @@ -167,6 +167,7 @@ enum Format {
>eFormatOctal,
>eFormatOSType, // OS character codes encoded into an integer 'PICT' 'text'
> // etc...
> + eFormatUnicode8,
>eFormatUnicode16,
>eFormatUnicode32,
>eFormatUnsigned,
> diff --git a/source/Plugins/Language/CPlusPlus/CPlusPlusLanguage.cpp
> b/source/Plugins/Language/CPlusPlus/CPlusPlusLanguage.cpp
> index 0b3c31816..15e0a82bd 100644
> --- a/source/Plugins/Language/CPlusPlus/CPlusPlusLanguage.cpp
> +++ b/source/Plugins/Language/CPlusPlus/CPlusPlusLanguage.cpp
> @@ -853,6 +853,14 @@ static void
> LoadSystemFormatters(lldb::TypeCategoryImplSP cpp_category_sp) {
>
>// FIXME because of a bug in the FormattersContainer we need to add a
> summary
>// for both X* and const X* ()
> + AddCXXSummary(
> + cpp_category_sp, lldb_private::formatters::Char8StringSummaryProvider,
> + "char8_t * summary provider", ConstString("char8_t *"), string_flags);
> + AddCXXSummary(cpp_category_sp,
> +lldb_private::formatters::Char8StringSummaryProvider,
> +"char8_t [] summary provider",
> +ConstString("char8_t \\[[0-9]+\\]"), string_array_flags,
> true);
> +
>AddCXXSummary(
>cpp_category_sp, lldb_private::formatters::Char16StringSummaryProvider,
>"char16_t * summary provider", ConstString("char16_t *"),
> string_flags);
> @@ -890,6 +898,9 @@ static void LoadSystemFormatters(lldb::TypeCategoryImplSP
> cpp_category_sp) {
>.SetHideItemNames(true)
>.SetShowMembersOneLiner(false);
>
> + AddCXXSummary(
> + cpp_category_sp, lldb_private::formatters::Char8SummaryProvider,
> + "char8_t summary provider", ConstString("char8_t"), widechar_flags);
>AddCXXSummary(
>cpp_category_sp, lldb_private::formatters::Char16SummaryProvider,
>"char16_t summary provider", ConstString("char16_t"), widechar_flags);
> diff --git a/source/Plugins/Language/CPlusPlus/CxxStringTypes.cpp
> b/source/Plugins/Language/CPlusPlus/CxxStringTypes.cpp
> index 959079070..3ea7589d8 100644
> --- a/source/Plugins/Language/CPlusPlus/CxxStringTypes.cpp
> +++ b/source/Plugins/Language/CPlusPlus/CxxStringTypes.cpp
> @@ -32,6 +32,31 @@ using namespace lldb;
> using namespace lldb_private;
> using namespace lldb_private::formatters;
>
> +bool lldb_private::formatters::Char8StringSummaryProvider(
> +ValueObject &valobj, Stream &stream, const TypeSummaryOptions &) {
> + ProcessSP process_sp = valobj.GetProcessSP();
> + if (!process_sp)
> +return false;
> +
> + lldb::addr_t valobj_addr = GetArrayAddressOrPointerValue(valobj);
> + if (valobj_addr == 0 || valobj_addr == LLDB_INVALID_ADDRESS)
> +return false;
> +
> + StringPrinter::ReadStringAndDumpToStreamOptions options(valobj);
> + options.SetLocation(valobj_addr);
> + options.SetProcessSP(process_sp);
> + options.SetStream(&stream);
> + option
