Re: [lldb-dev] Linux issues where I am not getting breakpoints...

[Pavel Labath via lldb-dev]

Are you sure this is not just an artifact of stdio buffering? I tried the
same experiment, but I placed a real log statement, and I could see that
all the LoadModuleAtAddress calls happen between the $T and $c packets in
the gdb-remote packet sequence.

The module loading should be synchronous, so I think the problem lies
elsewhere.

What is the nature of the breakpoint that is not getting hit? Can you
provide a repro case? The only bug like this that I am aware of is that we
fail to hit breakpoints in global constructors in shared libraries, but
that hasn't worked even in 3.8..



On 10 April 2017 at 22:51, Greg Clayton via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> I have added some logging to a program that is not hitting breakpoints
> with LLDB top of tree SVN. An older lldb 3.8 hits the breakpoint just fine.
> I placed some logging in LLDB:
>
> ModuleSP DynamicLoader::LoadModuleAtAddress(const FileSpec &file,
> addr_t link_map_addr,
> addr_t base_addr,
> bool base_addr_is_offset) {
>   printf("%s: lma = 0x%16.16llx, ba = 0x%16.16llx, baio = %i\n",
> file.GetPath().c_str(), link_map_addr, base_addr, base_addr_is_offset);
>
>
> This is called by DynamicLoaderPOSIXDYLD::LoadAllCurrentModules().
>
> My problem is I see:
>
> [vdso]: lma = 0x, ba = 0x77ffa000, baio = 0
> linux-vdso.so.1: lma = 0x77ffe6e0, ba = 0x77ffa000, baio =
> 1
> /tmp/liba.so: lma = 0x77ff66a8, ba = 0x77e3, baio = 1
> 8 locations added to breakpoint 1
> /tmp/libb.so: lma = 0x77e2f000, ba = 0x77d43000, baio = 1
> [==] Running 14 tests from 1 test case.
> [--] Global test environment set-up.
> [--] 14 tests from MyTest
> [ RUN  ] MyTest.Test1
> [   OK ] MyTest.Test1 (0 ms)
> /tmp/libc.so: lma = 0x77e2f000, ba = 0x77d43000, baio = 1
> /tmp/libd.so: lma = 0x77e2f000, ba = 0x77d43000, baio = 1
>
>
> Note that I see program output _during_ the messages that are showing that
> shared libraries are being loaded? I would assume we are loading shared
> libraries synchronously, but the log seems to indicated otherwise.
>
> If anyone knows anything on this subject please let me know...
>
> Greg Clayton
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Linux issues where I am not getting breakpoints...

[Greg Clayton via lldb-dev]

> On Apr 11, 2017, at 5:33 AM, Pavel Labath  wrote:
> 
> Are you sure this is not just an artifact of stdio buffering? I tried the 
> same experiment, but I placed a real log statement, and I could see that all 
> the LoadModuleAtAddress calls happen between the $T and $c packets in the 
> gdb-remote packet sequence. 
> 
> The module loading should be synchronous, so I think the problem lies 
> elsewhere.
> 
> What is the nature of the breakpoint that is not getting hit? Can you provide 
> a repro case? The only bug like this that I am aware of is that we fail to 
> hit breakpoints in global constructors in shared libraries, but that hasn't 
> worked even in 3.8..

I unfortunately can't attach a repro case. I will be able to track this down, 
just need some pointers. I did notice that I wasn't able to hit breakpoints in 
global constructors though... Do we know why? On Mac, we get notified of shared 
libraries as they load so we never miss anything. Why are we not able to get 
the same thing with linux?
> 
> 
> On 10 April 2017 at 22:51, Greg Clayton via lldb-dev  > wrote:
> I have added some logging to a program that is not hitting breakpoints with 
> LLDB top of tree SVN. An older lldb 3.8 hits the breakpoint just fine. I 
> placed some logging in LLDB:
> 
> ModuleSP DynamicLoader::LoadModuleAtAddress(const FileSpec &file,
> addr_t link_map_addr,
> addr_t base_addr,
> bool base_addr_is_offset) {
>   printf("%s: lma = 0x%16.16llx, ba = 0x%16.16llx, baio = %i\n", 
> file.GetPath().c_str(), link_map_addr, base_addr, base_addr_is_offset);
> 
> 
> This is called by DynamicLoaderPOSIXDYLD::LoadAllCurrentModules().
> 
> My problem is I see:
> 
> [vdso]: lma = 0x, ba = 0x77ffa000, baio = 0
> linux-vdso.so.1: lma = 0x77ffe6e0, ba = 0x77ffa000, baio = 1
> /tmp/liba.so: lma = 0x77ff66a8, ba = 0x77e3, baio = 1
> 8 locations added to breakpoint 1
> /tmp/libb.so: lma = 0x77e2f000, ba = 0x77d43000, baio = 1
> [==] Running 14 tests from 1 test case.
> [--] Global test environment set-up.
> [--] 14 tests from MyTest
> [ RUN  ] MyTest.Test1
> [   OK ] MyTest.Test1 (0 ms)
> /tmp/libc.so: lma = 0x77e2f000, ba = 0x77d43000, baio = 1
> /tmp/libd.so: lma = 0x77e2f000, ba = 0x77d43000, baio = 1
> 
> 
> Note that I see program output _during_ the messages that are showing that 
> shared libraries are being loaded? I would assume we are loading shared 
> libraries synchronously, but the log seems to indicated otherwise.
> 
> If anyone knows anything on this subject please let me know...
> 
> Greg Clayton
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org 
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev 
> 
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Linux issues where I am not getting breakpoints...

[Tamas Berghammer via lldb-dev]

See https://bugs.llvm.org/show_bug.cgi?id=25806 for details about why we
can't set breakpoint in the static initializer (it is an LLDB bug).

For your investigation a few pointers/guesses (assuming it is not some
stdout displaying issue what I consider unlikely based on your description):
* Do your application calls dlopen? It can explain why you see stdout
before some library load events and also I can imagine more issue in that
code path.
* Are you sure LoadModuleAtAddress called from LoadAllCurrentModules in all
4 cases? It can be called from RefreshModules as well what is used when we
get notified about a new library and I expect it to be more likely based on
the output (for the second 2 line).
* I suggest to stop at libc.so'_start and see what libraries are loaded
there (I expect it to be after the first 2 log and before the stdout).
Verify that both shared-library-event breakpoint is resolved at this time
(by "breakpoint list -i") and also set a manual breakpoint there to see
when it triggers. I expect you will hit that breakpoint just after your log
lines are displayed. If that is the case you should get a stacktrace and
see the callstack causing the event to be triggered.
* The libraries you should look out in the log is libc.so and
ld-linux-x86-64.so
(or similar). The dynamic loader integration should work only after these 2
libraries are loaded.

On Tue, Apr 11, 2017 at 3:56 PM Greg Clayton via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> On Apr 11, 2017, at 5:33 AM, Pavel Labath  wrote:
>
> Are you sure this is not just an artifact of stdio buffering? I tried the
> same experiment, but I placed a real log statement, and I could see that
> all the LoadModuleAtAddress calls happen between the $T and $c packets in
> the gdb-remote packet sequence.
>
> The module loading should be synchronous, so I think the problem lies
> elsewhere.
>
> What is the nature of the breakpoint that is not getting hit? Can you
> provide a repro case? The only bug like this that I am aware of is that we
> fail to hit breakpoints in global constructors in shared libraries, but
> that hasn't worked even in 3.8..
>
>
> I unfortunately can't attach a repro case. I will be able to track this
> down, just need some pointers. I did notice that I wasn't able to hit
> breakpoints in global constructors though... Do we know why? On Mac, we get
> notified of shared libraries as they load so we never miss anything. Why
> are we not able to get the same thing with linux?
>
>
>
> On 10 April 2017 at 22:51, Greg Clayton via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
>
> I have added some logging to a program that is not hitting breakpoints
> with LLDB top of tree SVN. An older lldb 3.8 hits the breakpoint just fine.
> I placed some logging in LLDB:
>
> ModuleSP DynamicLoader::LoadModuleAtAddress(const FileSpec &file,
> addr_t link_map_addr,
> addr_t base_addr,
> bool base_addr_is_offset) {
>   printf("%s: lma = 0x%16.16llx, ba = 0x%16.16llx, baio = %i\n",
> file.GetPath().c_str(), link_map_addr, base_addr, base_addr_is_offset);
>
>
> This is called by DynamicLoaderPOSIXDYLD::LoadAllCurrentModules().
>
> My problem is I see:
>
> [vdso]: lma = 0x, ba = 0x77ffa000, baio = 0
> linux-vdso.so.1: lma = 0x77ffe6e0, ba = 0x77ffa000, baio =
> 1
> /tmp/liba.so: lma = 0x77ff66a8, ba = 0x77e3, baio = 1
> 8 locations added to breakpoint 1
> /tmp/libb.so: lma = 0x77e2f000, ba = 0x77d43000, baio = 1
> [==] Running 14 tests from 1 test case.
> [--] Global test environment set-up.
> [--] 14 tests from MyTest
> [ RUN  ] MyTest.Test1
> [   OK ] MyTest.Test1 (0 ms)
> /tmp/libc.so: lma = 0x77e2f000, ba = 0x77d43000, baio = 1
> /tmp/libd.so: lma = 0x77e2f000, ba = 0x77d43000, baio = 1
>
>
> Note that I see program output _during_ the messages that are showing that
> shared libraries are being loaded? I would assume we are loading shared
> libraries synchronously, but the log seems to indicated otherwise.
>
> If anyone knows anything on this subject please let me know...
>
> Greg Clayton
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
>
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Linux issues where I am not getting breakpoints...

[Pavel Labath via lldb-dev]

On 11 April 2017 at 15:56, Greg Clayton  wrote:

>
> On Apr 11, 2017, at 5:33 AM, Pavel Labath  wrote:
>
> Are you sure this is not just an artifact of stdio buffering? I tried the
> same experiment, but I placed a real log statement, and I could see that
> all the LoadModuleAtAddress calls happen between the $T and $c packets in
> the gdb-remote packet sequence.
>
> The module loading should be synchronous, so I think the problem lies
> elsewhere.
>
> What is the nature of the breakpoint that is not getting hit? Can you
> provide a repro case? The only bug like this that I am aware of is that we
> fail to hit breakpoints in global constructors in shared libraries, but
> that hasn't worked even in 3.8..
>
>
> I unfortunately can't attach a repro case. I will be able to track this
> down, just need some pointers. I did notice that I wasn't able to hit
> breakpoints in global constructors though... Do we know why? On Mac, we get
> notified of shared libraries as they load so we never miss anything. Why
> are we not able to get the same thing with linux?
>
>
It looks like we are intercepting the library load too late, but I haven't
investigated yet how to fix it. It's definitely possible (this works fine
in gdb), but I don't know how, as the dynamic linker is still a big unknown
to me. FWIW, I think I'll be messing with the dynamic loader plugin
soon(ish), so I'll try to fix this then.

pl
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev