[lldb-dev] Proposal: Using LLD in tests
Hello all, currently we have a couple of tests, in-tree or under review, which are very close to being host-independent. The only part they are missing is the ability to link a intermediate object file: - the ppc64 test in https://reviews.llvm.org/D44437 needs a linker to resolve relocations in the debug info (*) - the PDB tests under lit/SymbolFile/PDB need a linker to produce the program database. I think it would be great if everyone were able to run these tests and verify they don't regress them before they actually push a patch. Apart from that, I have started looking at writing some non-execution debug info (**) tests as a part of adding DWARF v5 accelerator table support to lldb (both to test the new implementation, and to make sure I don't regress existing ones). Ideally I'd like to make sure that everyone is able to run them, regardless of their primary (or only) development platform. For this, I also need a linker capable of running everywhere (*) To achieve these goals, I'd like to propose that we add LLD as a (optional, but strongly recommended) dependency for running tests and start using it in the tests I mention. Doing this would optional in the sense that the tests would be marked "REQUIRED: lld", and simply skipped if lld is not available (so the tests would still be green). I say "strongly recommended" because not having lld checked out should not be an excuse for breaking the test, and the patch author should pro-actively revert a patch which breaks such tests and investigate. I hope this proposal is not too controversial. LLD is already required on windows to run dotest tests. Also, all monorepo users likely have it already, or it is very easy for them to enable it. For non-monorepo users it should be a matter of checking out one extra repository. Please Let me know what you think. pavel (*) our ELF parser has very limited support for applying debug info relocations -- it only works for x86, and only a couple of relocations are currently implemented. It would be possible to remove the linker dependency by implementing these (essentially, doing the link ourselves -- this is what llvm does), but given the large number of architectures and relocation types, combined with the long term goal of reusing the llvm's ELF parser, this does not seem like a worthwhile goal. Also, it does not help the windows situation, as in the PDB model it's the linker who produces the pdb's. (**) I'll write a separate email about this, but what I'm essentially thinking of is producing a stand-alone module (either from .yaml, .s, .ll, or .c), hitting it with various FindXXX methods, and dumping the results. ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
[lldb-dev] Advice on architectures with multiple address spaces
Hi lldb developers, I've been researching using lldb + gdbserver stub that is based on Harvard architecture with multiple address spaces (one program, multiple data). The commonly adopted approach is that everything is mapped to a single "virtual" address space. The stub reads/writes from/to the right memory based on the "virtual" addresses. But I'd like to use real addresses with address space id instead. So, I've started looking at what has to be changed. I've enhanced read/write commands (e.g. memory read --as ...) and RSP protocol (new packet) so that the stub can read/write properly. That wasn't that complicated. Now I've hit an issue with expressions (LLVMUserExpression.cpp) and local variables (DWARFExpressions.cpp). There is a lot of memory read/write functions that take just an address argument. Is the only way to go to patch all these calls? Has anybody solved it differently? Thanks for any advice. -- Zdenek ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Proposal: Using LLD in tests
The last I knew LLD doesn't work on mach-o very well, so be sure to not require LLD for linking any Darwin executables. > On Apr 19, 2018, at 6:42 AM, Pavel Labath via lldb-dev > wrote: > > Hello all, > > currently we have a couple of tests, in-tree or under review, which are > very close to being host-independent. The only part they are missing is the > ability to link a intermediate object file: > - the ppc64 test in https://reviews.llvm.org/D44437 needs a linker to > resolve relocations in the debug info (*) > - the PDB tests under lit/SymbolFile/PDB need a linker to produce the > program database. > > I think it would be great if everyone were able to run these tests and > verify they don't regress them before they actually push a patch. > > Apart from that, I have started looking at writing some non-execution debug > info (**) tests as a part of adding DWARF v5 accelerator table support to > lldb (both to test the new implementation, and to make sure I don't regress > existing ones). Ideally I'd like to make sure that everyone is able to run > them, regardless of their primary (or only) development platform. For this, > I also need a linker capable of running everywhere (*) > > To achieve these goals, I'd like to propose that we add LLD as a (optional, > but strongly recommended) dependency for running tests and start using it > in the tests I mention. Doing this would optional in the sense that the > tests would be marked "REQUIRED: lld", and simply skipped if lld is not > available (so the tests would still be green). I say "strongly recommended" > because not having lld checked out should not be an excuse for breaking the > test, and the patch author should pro-actively revert a patch which breaks > such tests and investigate. > > I hope this proposal is not too controversial. LLD is already required on > windows to run dotest tests. Also, all monorepo users likely have it > already, or it is very easy for them to enable it. For non-monorepo users > it should be a matter of checking out one extra repository. Please Let me > know what you think. > > pavel > > (*) our ELF parser has very limited support for applying debug info > relocations -- it only works for x86, and only a couple of relocations are > currently implemented. It would be possible to remove the linker dependency > by implementing these (essentially, doing the link ourselves -- this is > what llvm does), but given the large number of architectures and relocation > types, combined with the long term goal of reusing the llvm's ELF parser, > this does not seem like a worthwhile goal. Also, it does not help the > windows situation, as in the PDB model it's the linker who produces the > pdb's. > > (**) I'll write a separate email about this, but what I'm essentially > thinking of is producing a stand-alone module (either from .yaml, .s, .ll, > or .c), hitting it with various FindXXX methods, and dumping the results. > ___ > lldb-dev mailing list > lldb-dev@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Proposal: Using LLD in tests
Yes, I considered the lld+Mach-O situation. I understand that it does not work very well, but I don't know what exactly that means. However, I am not sure that we even need a linker for Mach-O. As I understand it, in a darwin world, the linker does not even touch the debug info, and lldb is already capable of reading the debug info from .o files. So at least for non dsym-case it seems to me we should be good. It may be possible we need a linker for the dSYM (i don't know exactly how that works), in which case we may just not be able to test dsym this way, but even that will be better than what we have now. PS: I am not proposing to do anything to the existing dotest tests (they need access to a running process anyway, so playing with the linker will not help). I am just trying to make new debug-info specific tests universally available. On Thu, 19 Apr 2018 at 17:24, Greg Clayton wrote: > The last I knew LLD doesn't work on mach-o very well, so be sure to not require LLD for linking any Darwin executables. > > On Apr 19, 2018, at 6:42 AM, Pavel Labath via lldb-dev < lldb-dev@lists.llvm.org> wrote: > > > > Hello all, > > > > currently we have a couple of tests, in-tree or under review, which are > > very close to being host-independent. The only part they are missing is the > > ability to link a intermediate object file: > > - the ppc64 test in https://reviews.llvm.org/D44437 needs a linker to > > resolve relocations in the debug info (*) > > - the PDB tests under lit/SymbolFile/PDB need a linker to produce the > > program database. > > > > I think it would be great if everyone were able to run these tests and > > verify they don't regress them before they actually push a patch. > > > > Apart from that, I have started looking at writing some non-execution debug > > info (**) tests as a part of adding DWARF v5 accelerator table support to > > lldb (both to test the new implementation, and to make sure I don't regress > > existing ones). Ideally I'd like to make sure that everyone is able to run > > them, regardless of their primary (or only) development platform. For this, > > I also need a linker capable of running everywhere (*) > > > > To achieve these goals, I'd like to propose that we add LLD as a (optional, > > but strongly recommended) dependency for running tests and start using it > > in the tests I mention. Doing this would optional in the sense that the > > tests would be marked "REQUIRED: lld", and simply skipped if lld is not > > available (so the tests would still be green). I say "strongly recommended" > > because not having lld checked out should not be an excuse for breaking the > > test, and the patch author should pro-actively revert a patch which breaks > > such tests and investigate. > > > > I hope this proposal is not too controversial. LLD is already required on > > windows to run dotest tests. Also, all monorepo users likely have it > > already, or it is very easy for them to enable it. For non-monorepo users > > it should be a matter of checking out one extra repository. Please Let me > > know what you think. > > > > pavel > > > > (*) our ELF parser has very limited support for applying debug info > > relocations -- it only works for x86, and only a couple of relocations are > > currently implemented. It would be possible to remove the linker dependency > > by implementing these (essentially, doing the link ourselves -- this is > > what llvm does), but given the large number of architectures and relocation > > types, combined with the long term goal of reusing the llvm's ELF parser, > > this does not seem like a worthwhile goal. Also, it does not help the > > windows situation, as in the PDB model it's the linker who produces the > > pdb's. > > > > (**) I'll write a separate email about this, but what I'm essentially > > thinking of is producing a stand-alone module (either from .yaml, .s, .ll, > > or .c), hitting it with various FindXXX methods, and dumping the results. > > ___ > > lldb-dev mailing list > > lldb-dev@lists.llvm.org > > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Advice on architectures with multiple address spaces
> On Apr 19, 2018, at 6:51 AM, Zdenek Prikryl via lldb-dev
> wrote:
>
> Hi lldb developers,
>
> I've been researching using lldb + gdbserver stub that is based on Harvard
> architecture with multiple address spaces (one program, multiple data). The
> commonly adopted approach is that everything is mapped to a single "virtual"
> address space. The stub reads/writes from/to the right memory based on the
> "virtual" addresses. But I'd like to use real addresses with address space id
> instead. So, I've started looking at what has to be changed.
>
> I've enhanced read/write commands (e.g. memory read --as ...) and RSP
> protocol (new packet) so that the stub can read/write properly. That wasn't
> that complicated.
It might be nice to add a new RSP protocol packet that asks for the address
space names/values:
qGetAddressSpaces
which would return something like:
1:text;2:data1,3:data2
or it would return not supported. If we get a valid return value from
qGetAddressSpaces, then it enables the use of the new packet you added above.
Else it defaults to using the old memory read functions.
>
> Now I've hit an issue with expressions (LLVMUserExpression.cpp) and local
> variables (DWARFExpressions.cpp). There is a lot of memory read/write
> functions that take just an address argument. Is the only way to go to patch
> all these calls? Has anybody solved it differently?
My quick take is that any APIs that take just a lldb::addr_t would need to take
something like:
struct SpaceAddress {
static constexpr uint32_t kNoSpace = 0;
lldb::addr_t addr;
uint32_t space;
};
We would need a default value for "space" (feel free to rename) that indicates
the default address space as most of our architectures would not need this
support. If we added a constructor like:
SpaceAddress(lldb::addr_t a) : addr(a), space(kNoSpace) {}
Then all usages of the APIs that used to take just a "lldb::addr_t" would
implicitly call this constructor and continue to act as needed. Then we would
need to allow lldb_private::Address objects to resolve to a SpaceAddress:
SpaceAddress lldb_private::Address::GetSpaceAddress(Target *target) const;
Since each lldb_private::Address has a section and each section knows its
address space. Then the tricky part is finding all locations in the expression
parser and converting those to track and use SpaceAddress. We would probably
need to modify the allocate memory packets in the RSP protocol to be able to
allocate memory in any address space as well.
I didn't spend much time think about correct names above, so feel free to
suggest alternate naming.
Best advice:
- make things "just work" to keep changes to a minimum and allowing
lldb::addr_t to implicitly convert to a SpaceAddress easily
- when modifying RSP, make sure to check for existence of new feature before
enabling it
- query for address space names so when we dump SpaceAddress we can show
something that means something to the user. This means we would need to query
the address space names from the current lldb_private::Process for display.
Submitting might go easier if we break it down into chunks:
1 - add SpaceAddress and modify all needed APIs to use it
2 - add ProcessGDBRemote changes that enable this support
It will be great to support this as a first class citizen within LLDB. You
might ask the Hexagon folks if they have done anything in case they already
support this is some shape or form.
Greg Clayton
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Proposal: Using LLD in tests
> On Apr 19, 2018, at 9:39 AM, Pavel Labath wrote: > > Yes, I considered the lld+Mach-O situation. I understand that it does not > work very well, but I don't know what exactly that means. > > However, I am not sure that we even need a linker for Mach-O. As I > understand it, in a darwin world, the linker does not even touch the debug > info, and lldb is already capable of reading the debug info from .o files. > So at least for non dsym-case it seems to me we should be good. It may be > possible we need a linker for the dSYM (i don't know exactly how that > works), in which case we may just not be able to test dsym this way, but > even that will be better than what we have now. The problem is the linker must produce a debug map in the symbol table that contains the addresses of all linked items. So using LLD won't work unless that is fully supported as LLDB won't be able to like the debug info on the fly, nor will dsymutil have a debug map to use in order to make a dSYM file. > > PS: I am not proposing to do anything to the existing dotest tests (they > need access to a running process anyway, so playing with the linker will > not help). I am just trying to make new debug-info specific tests > universally available. What I was trying to say is this is fine if we always compile/link as an ELF file with some triple that uses ELF for these tests. They just probably shouldn't be mach-o. Greg > > > On Thu, 19 Apr 2018 at 17:24, Greg Clayton wrote: > >> The last I knew LLD doesn't work on mach-o very well, so be sure to not > require LLD for linking any Darwin executables. > >>> On Apr 19, 2018, at 6:42 AM, Pavel Labath via lldb-dev < > lldb-dev@lists.llvm.org> wrote: >>> >>> Hello all, >>> >>> currently we have a couple of tests, in-tree or under review, which are >>> very close to being host-independent. The only part they are missing is > the >>> ability to link a intermediate object file: >>> - the ppc64 test in https://reviews.llvm.org/D44437 needs a linker to >>> resolve relocations in the debug info (*) >>> - the PDB tests under lit/SymbolFile/PDB need a linker to produce the >>> program database. >>> >>> I think it would be great if everyone were able to run these tests and >>> verify they don't regress them before they actually push a patch. >>> >>> Apart from that, I have started looking at writing some non-execution > debug >>> info (**) tests as a part of adding DWARF v5 accelerator table support > to >>> lldb (both to test the new implementation, and to make sure I don't > regress >>> existing ones). Ideally I'd like to make sure that everyone is able to > run >>> them, regardless of their primary (or only) development platform. For > this, >>> I also need a linker capable of running everywhere (*) >>> >>> To achieve these goals, I'd like to propose that we add LLD as a > (optional, >>> but strongly recommended) dependency for running tests and start using > it >>> in the tests I mention. Doing this would optional in the sense that the >>> tests would be marked "REQUIRED: lld", and simply skipped if lld is not >>> available (so the tests would still be green). I say "strongly > recommended" >>> because not having lld checked out should not be an excuse for breaking > the >>> test, and the patch author should pro-actively revert a patch which > breaks >>> such tests and investigate. >>> >>> I hope this proposal is not too controversial. LLD is already required > on >>> windows to run dotest tests. Also, all monorepo users likely have it >>> already, or it is very easy for them to enable it. For non-monorepo > users >>> it should be a matter of checking out one extra repository. Please Let > me >>> know what you think. >>> >>> pavel >>> >>> (*) our ELF parser has very limited support for applying debug info >>> relocations -- it only works for x86, and only a couple of relocations > are >>> currently implemented. It would be possible to remove the linker > dependency >>> by implementing these (essentially, doing the link ourselves -- this is >>> what llvm does), but given the large number of architectures and > relocation >>> types, combined with the long term goal of reusing the llvm's ELF > parser, >>> this does not seem like a worthwhile goal. Also, it does not help the >>> windows situation, as in the PDB model it's the linker who produces the >>> pdb's. >>> >>> (**) I'll write a separate email about this, but what I'm essentially >>> thinking of is producing a stand-alone module (either from .yaml, .s, > .ll, >>> or .c), hitting it with various FindXXX methods, and dumping the > results. >>> ___ >>> lldb-dev mailing list >>> lldb-dev@lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Proposal: Using LLD in tests
> > the PDB tests under lit/SymbolFile/PDB need a linker to produce the program > database With this proposal, would we preserve any coverage for MSVC produced debug information? On Thu, Apr 19, 2018 at 9:47 AM, Greg Clayton via lldb-dev < lldb-dev@lists.llvm.org> wrote: > > > > On Apr 19, 2018, at 9:39 AM, Pavel Labath wrote: > > > > Yes, I considered the lld+Mach-O situation. I understand that it does not > > work very well, but I don't know what exactly that means. > > > > However, I am not sure that we even need a linker for Mach-O. As I > > understand it, in a darwin world, the linker does not even touch the > debug > > info, and lldb is already capable of reading the debug info from .o > files. > > So at least for non dsym-case it seems to me we should be good. It may be > > possible we need a linker for the dSYM (i don't know exactly how that > > works), in which case we may just not be able to test dsym this way, but > > even that will be better than what we have now. > > The problem is the linker must produce a debug map in the symbol table > that contains the addresses of all linked items. So using LLD won't work > unless that is fully supported as LLDB won't be able to like the debug info > on the fly, nor will dsymutil have a debug map to use in order to make a > dSYM file. > > > > > PS: I am not proposing to do anything to the existing dotest tests (they > > need access to a running process anyway, so playing with the linker will > > not help). I am just trying to make new debug-info specific tests > > universally available. > > What I was trying to say is this is fine if we always compile/link as an > ELF file with some triple that uses ELF for these tests. They just probably > shouldn't be mach-o. > > Greg > > > > > > > On Thu, 19 Apr 2018 at 17:24, Greg Clayton wrote: > > > >> The last I knew LLD doesn't work on mach-o very well, so be sure to not > > require LLD for linking any Darwin executables. > > > >>> On Apr 19, 2018, at 6:42 AM, Pavel Labath via lldb-dev < > > lldb-dev@lists.llvm.org> wrote: > >>> > >>> Hello all, > >>> > >>> currently we have a couple of tests, in-tree or under review, which are > >>> very close to being host-independent. The only part they are missing is > > the > >>> ability to link a intermediate object file: > >>> - the ppc64 test in https://reviews.llvm.org/D44437 needs a linker to > >>> resolve relocations in the debug info (*) > >>> - the PDB tests under lit/SymbolFile/PDB need a linker to produce the > >>> program database. > >>> > >>> I think it would be great if everyone were able to run these tests and > >>> verify they don't regress them before they actually push a patch. > >>> > >>> Apart from that, I have started looking at writing some non-execution > > debug > >>> info (**) tests as a part of adding DWARF v5 accelerator table support > > to > >>> lldb (both to test the new implementation, and to make sure I don't > > regress > >>> existing ones). Ideally I'd like to make sure that everyone is able to > > run > >>> them, regardless of their primary (or only) development platform. For > > this, > >>> I also need a linker capable of running everywhere (*) > >>> > >>> To achieve these goals, I'd like to propose that we add LLD as a > > (optional, > >>> but strongly recommended) dependency for running tests and start using > > it > >>> in the tests I mention. Doing this would optional in the sense that the > >>> tests would be marked "REQUIRED: lld", and simply skipped if lld is not > >>> available (so the tests would still be green). I say "strongly > > recommended" > >>> because not having lld checked out should not be an excuse for breaking > > the > >>> test, and the patch author should pro-actively revert a patch which > > breaks > >>> such tests and investigate. > >>> > >>> I hope this proposal is not too controversial. LLD is already required > > on > >>> windows to run dotest tests. Also, all monorepo users likely have it > >>> already, or it is very easy for them to enable it. For non-monorepo > > users > >>> it should be a matter of checking out one extra repository. Please Let > > me > >>> know what you think. > >>> > >>> pavel > >>> > >>> (*) our ELF parser has very limited support for applying debug info > >>> relocations -- it only works for x86, and only a couple of relocations > > are > >>> currently implemented. It would be possible to remove the linker > > dependency > >>> by implementing these (essentially, doing the link ourselves -- this is > >>> what llvm does), but given the large number of architectures and > > relocation > >>> types, combined with the long term goal of reusing the llvm's ELF > > parser, > >>> this does not seem like a worthwhile goal. Also, it does not help the > >>> windows situation, as in the PDB model it's the linker who produces the > >>> pdb's. > >>> > >>> (**) I'll write a separate email about this, but what I'm essentially > >>> thinking of is producing a stand-alone module (either from .yaml, .s,
Re: [lldb-dev] Advice on architectures with multiple address spaces
> On Apr 19, 2018, at 9:44 AM, Greg Clayton via lldb-dev
> wrote:
>
>
>
>> On Apr 19, 2018, at 6:51 AM, Zdenek Prikryl via lldb-dev
>> wrote:
>>
>> Hi lldb developers,
>>
>> I've been researching using lldb + gdbserver stub that is based on Harvard
>> architecture with multiple address spaces (one program, multiple data). The
>> commonly adopted approach is that everything is mapped to a single "virtual"
>> address space. The stub reads/writes from/to the right memory based on the
>> "virtual" addresses. But I'd like to use real addresses with address space
>> id instead. So, I've started looking at what has to be changed.
>>
>> I've enhanced read/write commands (e.g. memory read --as ...) and RSP
>> protocol (new packet) so that the stub can read/write properly. That wasn't
>> that complicated.
>
> It might be nice to add a new RSP protocol packet that asks for the address
> space names/values:
>
> qGetAddressSpaces
>
> which would return something like:
>
> 1:text;2:data1,3:data2
>
> or it would return not supported. If we get a valid return value from
> qGetAddressSpaces, then it enables the use of the new packet you added above.
> Else it defaults to using the old memory read functions.
>
>
>>
>> Now I've hit an issue with expressions (LLVMUserExpression.cpp) and local
>> variables (DWARFExpressions.cpp). There is a lot of memory read/write
>> functions that take just an address argument. Is the only way to go to patch
>> all these calls? Has anybody solved it differently?
>
> My quick take is that any APIs that take just a lldb::addr_t would need to
> take something like:
>
> struct SpaceAddress {
> static constexpr uint32_t kNoSpace = 0;
> lldb::addr_t addr;
> uint32_t space;
> };
>
I'm curious why you are suggesting another kind of address, rather than adding
this functionality to Address? When you actually go to resolve an Address in a
target with a process you should have everything you need to know to give it
the proper space. Then fixing the expression evaluator (and anything else that
needs fixing) would be a matter of consistently using Address rather than
lldb::addr_t. That seems general goodness, since converting to an lldb::addr_t
loses information.
Jim
> We would need a default value for "space" (feel free to rename) that
> indicates the default address space as most of our architectures would not
> need this support. If we added a constructor like:
>
> SpaceAddress(lldb::addr_t a) : addr(a), space(kNoSpace) {}
>
> Then all usages of the APIs that used to take just a "lldb::addr_t" would
> implicitly call this constructor and continue to act as needed. Then we would
> need to allow lldb_private::Address objects to resolve to a SpaceAddress:
>
> SpaceAddress lldb_private::Address::GetSpaceAddress(Target *target) const;
>
> Since each lldb_private::Address has a section and each section knows its
> address space. Then the tricky part is finding all locations in the
> expression parser and converting those to track and use SpaceAddress. We
> would probably need to modify the allocate memory packets in the RSP protocol
> to be able to allocate memory in any address space as well.
>
> I didn't spend much time think about correct names above, so feel free to
> suggest alternate naming.
>
> Best advice:
> - make things "just work" to keep changes to a minimum and allowing
> lldb::addr_t to implicitly convert to a SpaceAddress easily
> - when modifying RSP, make sure to check for existence of new feature before
> enabling it
> - query for address space names so when we dump SpaceAddress we can show
> something that means something to the user. This means we would need to query
> the address space names from the current lldb_private::Process for display.
>
> Submitting might go easier if we break it down into chunks:
> 1 - add SpaceAddress and modify all needed APIs to use it
> 2 - add ProcessGDBRemote changes that enable this support
>
> It will be great to support this as a first class citizen within LLDB. You
> might ask the Hexagon folks if they have done anything in case they already
> support this is some shape or form.
>
> Greg Clayton
>
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Proposal: Using LLD in tests
We already don't really have any. All of the makefiles that build the inferiors use a gcc style command line syntax, and the way we made that work on Windows is to use clang for the compiler. So MSVC is already out of the equation. There may be one or two instances where we check in a PDB file, but that doesn't really scale. I've been working a lot on the PDB support in LLVM and we have some good coverage there that our PDBs are "as good as" Microsoft PDBs, and in the future we have plans to have a debug info test suite that tests LLD-generated PDBs with Microsoft debuggers. If we have coverage of that, I think that should be sufficient and maybe not worth complicating the LLDB test suite matrix over trying to test debugging with MSVC generated PDBs. On Thu, Apr 19, 2018 at 10:19 AM Leonard Mosescu via lldb-dev < lldb-dev@lists.llvm.org> wrote: > the PDB tests under lit/SymbolFile/PDB need a linker to produce the program >> database > > > With this proposal, would we preserve any coverage for MSVC produced debug > information? > > On Thu, Apr 19, 2018 at 9:47 AM, Greg Clayton via lldb-dev < > lldb-dev@lists.llvm.org> wrote: > >> >> >> > On Apr 19, 2018, at 9:39 AM, Pavel Labath wrote: >> > >> > Yes, I considered the lld+Mach-O situation. I understand that it does >> not >> > work very well, but I don't know what exactly that means. >> > >> > However, I am not sure that we even need a linker for Mach-O. As I >> > understand it, in a darwin world, the linker does not even touch the >> debug >> > info, and lldb is already capable of reading the debug info from .o >> files. >> > So at least for non dsym-case it seems to me we should be good. It may >> be >> > possible we need a linker for the dSYM (i don't know exactly how that >> > works), in which case we may just not be able to test dsym this way, but >> > even that will be better than what we have now. >> >> The problem is the linker must produce a debug map in the symbol table >> that contains the addresses of all linked items. So using LLD won't work >> unless that is fully supported as LLDB won't be able to like the debug info >> on the fly, nor will dsymutil have a debug map to use in order to make a >> dSYM file. >> >> > >> > PS: I am not proposing to do anything to the existing dotest tests (they >> > need access to a running process anyway, so playing with the linker will >> > not help). I am just trying to make new debug-info specific tests >> > universally available. >> >> What I was trying to say is this is fine if we always compile/link as an >> ELF file with some triple that uses ELF for these tests. They just probably >> shouldn't be mach-o. >> >> Greg >> >> > >> > >> > On Thu, 19 Apr 2018 at 17:24, Greg Clayton wrote: >> > >> >> The last I knew LLD doesn't work on mach-o very well, so be sure to not >> > require LLD for linking any Darwin executables. >> > >> >>> On Apr 19, 2018, at 6:42 AM, Pavel Labath via lldb-dev < >> > lldb-dev@lists.llvm.org> wrote: >> >>> >> >>> Hello all, >> >>> >> >>> currently we have a couple of tests, in-tree or under review, which >> are >> >>> very close to being host-independent. The only part they are missing >> is >> > the >> >>> ability to link a intermediate object file: >> >>> - the ppc64 test in https://reviews.llvm.org/D44437 needs a linker to >> >>> resolve relocations in the debug info (*) >> >>> - the PDB tests under lit/SymbolFile/PDB need a linker to produce the >> >>> program database. >> >>> >> >>> I think it would be great if everyone were able to run these tests and >> >>> verify they don't regress them before they actually push a patch. >> >>> >> >>> Apart from that, I have started looking at writing some non-execution >> > debug >> >>> info (**) tests as a part of adding DWARF v5 accelerator table support >> > to >> >>> lldb (both to test the new implementation, and to make sure I don't >> > regress >> >>> existing ones). Ideally I'd like to make sure that everyone is able to >> > run >> >>> them, regardless of their primary (or only) development platform. For >> > this, >> >>> I also need a linker capable of running everywhere (*) >> >>> >> >>> To achieve these goals, I'd like to propose that we add LLD as a >> > (optional, >> >>> but strongly recommended) dependency for running tests and start using >> > it >> >>> in the tests I mention. Doing this would optional in the sense that >> the >> >>> tests would be marked "REQUIRED: lld", and simply skipped if lld is >> not >> >>> available (so the tests would still be green). I say "strongly >> > recommended" >> >>> because not having lld checked out should not be an excuse for >> breaking >> > the >> >>> test, and the patch author should pro-actively revert a patch which >> > breaks >> >>> such tests and investigate. >> >>> >> >>> I hope this proposal is not too controversial. LLD is already required >> > on >> >>> windows to run dotest tests. Also, all monorepo users likely have it >> >>> already, or it is very easy for them to enable
Re: [lldb-dev] Proposal: Using LLD in tests
On Thu, 19 Apr 2018 at 18:19, Leonard Mosescu wrote: >>the PDB tests under lit/SymbolFile/PDB need a linker to produce the program database > With this proposal, would we preserve any coverage for MSVC produced debug information? Well.. the question there is what are you trying to test? Is it the fact your debugger works with a particular compiler+linker combination (note that those tests already compile with clang-cl), or that your pdb-parsing code is sane. (integration vs. regression test). Historically we've only had the former kind of tests (dotest), and we've had the ability (and used it) to run those tests against different kinds of compilers. This is all nice, but it means that a specific test will be testing a different thing for every person who runs it. That's why I would like to build up a suite of more regression-like tests (*). I would say that the tests under lit/*** should be regression tests and our goal should be to remove as many system dependencies as possible, and leave the job of testing integration with a specific toolchain to "dotest" tests (**). Technically, the answer to your question is "no", because currently dotest tests don't know how to work with cl+link. Making that work would be an interesting project (although a bit annoying as the Makefiles are full of gcc-isms). However, I don't think that should stop us here. (*) Ideally I would like to leave even the compiler out of the equation for these tests, and make it so that the tests always run on the exact same set of bytes. I am hoping I will be able to write at least some tests using .s files. However, I don't think I will do that for all of them, because these files can be long/verbose/tedious to write. (**) However, even "dotest" tests should have a "default" mode which is as hermetic as possible. ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Advice on architectures with multiple address spaces
> On Apr 19, 2018, at 10:35 AM, Jim Ingham wrote:
>
>
>
>> On Apr 19, 2018, at 9:44 AM, Greg Clayton via lldb-dev
>> wrote:
>>
>>
>>
>>> On Apr 19, 2018, at 6:51 AM, Zdenek Prikryl via lldb-dev
>>> wrote:
>>>
>>> Hi lldb developers,
>>>
>>> I've been researching using lldb + gdbserver stub that is based on Harvard
>>> architecture with multiple address spaces (one program, multiple data). The
>>> commonly adopted approach is that everything is mapped to a single
>>> "virtual" address space. The stub reads/writes from/to the right memory
>>> based on the "virtual" addresses. But I'd like to use real addresses with
>>> address space id instead. So, I've started looking at what has to be
>>> changed.
>>>
>>> I've enhanced read/write commands (e.g. memory read --as ...) and RSP
>>> protocol (new packet) so that the stub can read/write properly. That wasn't
>>> that complicated.
>>
>> It might be nice to add a new RSP protocol packet that asks for the address
>> space names/values:
>>
>> qGetAddressSpaces
>>
>> which would return something like:
>>
>> 1:text;2:data1,3:data2
>>
>> or it would return not supported. If we get a valid return value from
>> qGetAddressSpaces, then it enables the use of the new packet you added
>> above. Else it defaults to using the old memory read functions.
>>
>>
>>>
>>> Now I've hit an issue with expressions (LLVMUserExpression.cpp) and local
>>> variables (DWARFExpressions.cpp). There is a lot of memory read/write
>>> functions that take just an address argument. Is the only way to go to
>>> patch all these calls? Has anybody solved it differently?
>>
>> My quick take is that any APIs that take just a lldb::addr_t would need to
>> take something like:
>>
>> struct SpaceAddress {
>> static constexpr uint32_t kNoSpace = 0;
>> lldb::addr_t addr;
>> uint32_t space;
>> };
>>
>
> I'm curious why you are suggesting another kind of address, rather than
> adding this functionality to Address? When you actually go to resolve an
> Address in a target with a process you should have everything you need to
> know to give it the proper space. Then fixing the expression evaluator (and
> anything else that needs fixing) would be a matter of consistently using
> Address rather than lldb::addr_t. That seems general goodness, since
> converting to an lldb::addr_t loses information.
If we accept lldb_private::Address in all APIs that take a lldb::addr_t
currently, then we need to always be able to get to the target in case we need
to add code to resolve the address everywhere. I am thinking of SpaceAddress as
an augmented lldb::addr_t instead of a section + offset style address. Also,
there will be addresses in the code and data that do not exist in actual
sections. Not saying that you couldn't use lldb_private::Address. I am open to
suggestions though. So your though it remove all API that take lldb::addr_t and
use lldb_private::Address everywhere all the time?
>
> Jim
>
>
>> We would need a default value for "space" (feel free to rename) that
>> indicates the default address space as most of our architectures would not
>> need this support. If we added a constructor like:
>>
>> SpaceAddress(lldb::addr_t a) : addr(a), space(kNoSpace) {}
>>
>> Then all usages of the APIs that used to take just a "lldb::addr_t" would
>> implicitly call this constructor and continue to act as needed. Then we
>> would need to allow lldb_private::Address objects to resolve to a
>> SpaceAddress:
>>
>> SpaceAddress lldb_private::Address::GetSpaceAddress(Target *target) const;
>>
>> Since each lldb_private::Address has a section and each section knows its
>> address space. Then the tricky part is finding all locations in the
>> expression parser and converting those to track and use SpaceAddress. We
>> would probably need to modify the allocate memory packets in the RSP
>> protocol to be able to allocate memory in any address space as well.
>>
>> I didn't spend much time think about correct names above, so feel free to
>> suggest alternate naming.
>>
>> Best advice:
>> - make things "just work" to keep changes to a minimum and allowing
>> lldb::addr_t to implicitly convert to a SpaceAddress easily
>> - when modifying RSP, make sure to check for existence of new feature before
>> enabling it
>> - query for address space names so when we dump SpaceAddress we can show
>> something that means something to the user. This means we would need to
>> query the address space names from the current lldb_private::Process for
>> display.
>>
>> Submitting might go easier if we break it down into chunks:
>> 1 - add SpaceAddress and modify all needed APIs to use it
>> 2 - add ProcessGDBRemote changes that enable this support
>>
>> It will be great to support this as a first class citizen within LLDB. You
>> might ask the Hexagon folks if they have done anything in case they already
>> support this is some shape or form.
>>
>> Greg Clayton
>>
>>
Re: [lldb-dev] Proposal: Using LLD in tests
> > we have some good coverage there that our PDBs are "as good as" Microsoft > PDBs, and in the future we have plans to have a debug info test suite that > tests LLD-generated PDBs with Microsoft debuggers. > Thanks Zach. What I was asking is exactly the other half of this equation. Testing LLDB with MSVC produced PDBs should be complementary and I doubt we can get full coverage by just testing from the other direction (it's plausible that even with equivalent semantic information, certain patterns may only occur in MSVC produced debug information) IMO reliable support for MSVC generated binaries and debug information is critical since even with a LLVM/LLD toolchain you'll still have system/3rd party modules, right? Anyway, I was curious if any of this is in scope for Pavel's proposal and the answer seems to be no, thanks everyone. On Thu, Apr 19, 2018 at 10:44 AM, Pavel Labath wrote: > On Thu, 19 Apr 2018 at 18:19, Leonard Mosescu wrote: > > >>the PDB tests under lit/SymbolFile/PDB need a linker to produce the > program database > > > > With this proposal, would we preserve any coverage for MSVC produced > debug information? > > > Well.. the question there is what are you trying to test? Is it the fact > your debugger works with a particular compiler+linker > combination (note that those tests already compile with clang-cl), or that > your pdb-parsing code is sane. (integration vs. regression test). > > Historically we've only had the former kind of tests (dotest), and we've > had the ability (and used it) to run those tests against different kinds of > compilers. This is all nice, but it means that a specific test will be > testing a different thing for every person who runs it. That's why I would > like to build up a suite of more regression-like tests (*). I would say > that the tests under lit/*** should be regression tests and our goal should > be to remove as many system dependencies as possible, and leave the job of > testing integration with a specific toolchain to "dotest" tests (**). > > Technically, the answer to your question is "no", because currently dotest > tests don't know how to work with cl+link. Making that work would be an > interesting project (although a bit annoying as the Makefiles are full of > gcc-isms). However, I don't think that should stop us here. > > (*) Ideally I would like to leave even the compiler out of the equation for > these tests, and make it so that the tests always run on the exact same set > of bytes. I am hoping I will be able to write at least some tests using .s > files. However, I don't think I will do that for all of them, because these > files can be long/verbose/tedious to write. > > (**) However, even "dotest" tests should have a "default" mode which is as > hermetic as possible. > ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
[lldb-dev] FileSpec and normalization questions
We currently have DWARF that has a DW_AT_comp_dir that is set to "./" followed
by any number of extra '/' characters. I would like to have this path
normalized as we parse the DWARF so we don't end up with line tables with a
variety of ".//+" prefixes on each source file.
While looking to solve this issue, I took a look at the functionality that is
in FileSpec right now. In:
void FileSpec::SetFile(llvm::StringRef pathname, bool resolve, PathSyntax
syntax);
This function always calls a cheaper normalize function:
namespace {
void Normalize(llvm::SmallVectorImpl &path, FileSpec::PathSyntax
syntax);
}
This function does nothing for posix paths, but switches backward slashes to
forward slashes.
We have a FileSpec function named FileSpec::GetNormalizedPath() which will do
much more path normalization on a path by removing redundant "." and ".." and
"//".
I can fix my DWARF issue in a few ways:
1 - fix FileSpec::SetFile() to still call ::Normalize() but have it do more
work and have it normalize out the and redundant or relative path info
2 - call FileSpec::GetNormalizedPath() on each comp dir before using it to
actually normalize it
The main question is: do we want paths floating around LLDB that aren't
normalized? It seems like having a mixture of theses path will lead to issues
in LLDB so I would vote for solution #1.
Also, looking at the tests for normalizing paths I found the following pairs of
pre-normalized and post-normalization paths for posix:
{"//", "//"},
{"//net", "//net"},
Why wouldn't we reduce "//" to just "/" for posix? And why wouldn't we reduce
"//net" to "/net"?
Also I found:
{"./foo", "foo"},
Do we prefer to not have "./foo" to stay as "./foo"? Seems like if users know
that their debug info has line table entries that are "./foo/foo.c" that they
might type this in:
(lldb) b ./foo/foo.c:12
But this will fail since it might not match the "foo/foo.c:12" that might
result from path normalization. We don't normalize right now so it doesn't
matter and things would match, but part of my fix is normalizing a path in the
DWARF that is currently ".foo/foo.c" down to either "./foo/foo.c" or
"foo/foo.c" so it will matter depending on what we decide here.
Any input would be appreciated.
Greg Clayton
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] FileSpec and normalization questions
IIRC We have path normalization functions in llvm, have you looked at them?
Thanks,
--
Davide
On Thu, Apr 19, 2018 at 11:14 AM, Greg Clayton via lldb-dev
wrote:
> We currently have DWARF that has a DW_AT_comp_dir that is set to "./"
> followed by any number of extra '/' characters. I would like to have this
> path normalized as we parse the DWARF so we don't end up with line tables
> with a variety of ".//+" prefixes on each source file.
>
> While looking to solve this issue, I took a look at the functionality that
> is in FileSpec right now. In:
>
> void FileSpec::SetFile(llvm::StringRef pathname, bool resolve, PathSyntax
> syntax);
>
>
> This function always calls a cheaper normalize function:
>
> namespace {
> void Normalize(llvm::SmallVectorImpl &path, FileSpec::PathSyntax
> syntax);
> }
>
> This function does nothing for posix paths, but switches backward slashes to
> forward slashes.
>
> We have a FileSpec function named FileSpec::GetNormalizedPath() which will
> do much more path normalization on a path by removing redundant "." and ".."
> and "//".
>
> I can fix my DWARF issue in a few ways:
> 1 - fix FileSpec::SetFile() to still call ::Normalize() but have it do more
> work and have it normalize out the and redundant or relative path info
> 2 - call FileSpec::GetNormalizedPath() on each comp dir before using it to
> actually normalize it
>
> The main question is: do we want paths floating around LLDB that aren't
> normalized? It seems like having a mixture of theses path will lead to
> issues in LLDB so I would vote for solution #1.
>
> Also, looking at the tests for normalizing paths I found the following pairs
> of pre-normalized and post-normalization paths for posix:
>
> {"//", "//"},
> {"//net", "//net"},
>
> Why wouldn't we reduce "//" to just "/" for posix? And why wouldn't we
> reduce "//net" to "/net"?
>
> Also I found:
>
> {"./foo", "foo"},
>
> Do we prefer to not have "./foo" to stay as "./foo"? Seems like if users
> know that their debug info has line table entries that are "./foo/foo.c"
> that they might type this in:
>
> (lldb) b ./foo/foo.c:12
>
> But this will fail since it might not match the "foo/foo.c:12" that might
> result from path normalization. We don't normalize right now so it doesn't
> matter and things would match, but part of my fix is normalizing a path in
> the DWARF that is currently ".foo/foo.c" down to either
> "./foo/foo.c" or "foo/foo.c" so it will matter depending on what we decide
> here.
>
> Any input would be appreciated.
>
> Greg Clayton
>
>
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] FileSpec and normalization questions
The last time I looked at the llvm functions they only support the path syntax
of the llvm host, which won't do for lldb. But maybe they have gotten more
general recently?
Jim
> On Apr 19, 2018, at 11:16 AM, Davide Italiano via lldb-dev
> wrote:
>
> IIRC We have path normalization functions in llvm, have you looked at them?
>
> Thanks,
>
> --
> Davide
>
> On Thu, Apr 19, 2018 at 11:14 AM, Greg Clayton via lldb-dev
> wrote:
>> We currently have DWARF that has a DW_AT_comp_dir that is set to "./"
>> followed by any number of extra '/' characters. I would like to have this
>> path normalized as we parse the DWARF so we don't end up with line tables
>> with a variety of ".//+" prefixes on each source file.
>>
>> While looking to solve this issue, I took a look at the functionality that
>> is in FileSpec right now. In:
>>
>> void FileSpec::SetFile(llvm::StringRef pathname, bool resolve, PathSyntax
>> syntax);
>>
>>
>> This function always calls a cheaper normalize function:
>>
>> namespace {
>> void Normalize(llvm::SmallVectorImpl &path, FileSpec::PathSyntax
>> syntax);
>> }
>>
>> This function does nothing for posix paths, but switches backward slashes to
>> forward slashes.
>>
>> We have a FileSpec function named FileSpec::GetNormalizedPath() which will
>> do much more path normalization on a path by removing redundant "." and ".."
>> and "//".
>>
>> I can fix my DWARF issue in a few ways:
>> 1 - fix FileSpec::SetFile() to still call ::Normalize() but have it do more
>> work and have it normalize out the and redundant or relative path info
>> 2 - call FileSpec::GetNormalizedPath() on each comp dir before using it to
>> actually normalize it
>>
>> The main question is: do we want paths floating around LLDB that aren't
>> normalized? It seems like having a mixture of theses path will lead to
>> issues in LLDB so I would vote for solution #1.
>>
>> Also, looking at the tests for normalizing paths I found the following pairs
>> of pre-normalized and post-normalization paths for posix:
>>
>> {"//", "//"},
>> {"//net", "//net"},
>>
>> Why wouldn't we reduce "//" to just "/" for posix? And why wouldn't we
>> reduce "//net" to "/net"?
>>
>> Also I found:
>>
>> {"./foo", "foo"},
>>
>> Do we prefer to not have "./foo" to stay as "./foo"? Seems like if users
>> know that their debug info has line table entries that are "./foo/foo.c"
>> that they might type this in:
>>
>> (lldb) b ./foo/foo.c:12
>>
>> But this will fail since it might not match the "foo/foo.c:12" that might
>> result from path normalization. We don't normalize right now so it doesn't
>> matter and things would match, but part of my fix is normalizing a path in
>> the DWARF that is currently ".foo/foo.c" down to either
>> "./foo/foo.c" or "foo/foo.c" so it will matter depending on what we decide
>> here.
>>
>> Any input would be appreciated.
>>
>> Greg Clayton
>>
>>
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] FileSpec and normalization questions
On Thu, Apr 19, 2018 at 11:14 AM Greg Clayton via lldb-dev <
lldb-dev@lists.llvm.org> wrote:
>
> Also, looking at the tests for normalizing paths I found the following
> pairs of pre-normalized and post-normalization paths for posix:
>
> {"//", "//"},
> {"//net", "//net"},
>
> Why wouldn't we reduce "//" to just "/" for posix? And why wouldn't we
> reduce "//net" to "/net"?
>
>
I don't know what the author of this test had in mind, but from the POSIX
spec:
http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap04.html#tag_04_11
> A pathname that begins with two successive slashes may be interpreted in
an implementation-defined manner, although more than two leading slashes
shall be treated as a single slash.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] FileSpec and normalization questions
On Thu, Apr 19, 2018 at 11:19 AM Jim Ingham via lldb-dev < lldb-dev@lists.llvm.org> wrote: > The last time I looked at the llvm functions they only support the path > syntax of the llvm host, which won't do for lldb. But maybe they have > gotten more general recently? > > Yes in fact I was the one who updated them to make them more general. You can now specify an enumeration parameter which will cause the algorithm to treat paths as either windows, posix, or whatever the host is. ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Advice on architectures with multiple address spaces
> On Apr 19, 2018, at 10:54 AM, Greg Clayton wrote:
>
>
>
>> On Apr 19, 2018, at 10:35 AM, Jim Ingham wrote:
>>
>>
>>
>>> On Apr 19, 2018, at 9:44 AM, Greg Clayton via lldb-dev
>>> wrote:
>>>
>>>
>>>
On Apr 19, 2018, at 6:51 AM, Zdenek Prikryl via lldb-dev
wrote:
Hi lldb developers,
I've been researching using lldb + gdbserver stub that is based on Harvard
architecture with multiple address spaces (one program, multiple data).
The commonly adopted approach is that everything is mapped to a single
"virtual" address space. The stub reads/writes from/to the right memory
based on the "virtual" addresses. But I'd like to use real addresses with
address space id instead. So, I've started looking at what has to be
changed.
I've enhanced read/write commands (e.g. memory read --as ...) and RSP
protocol (new packet) so that the stub can read/write properly. That
wasn't that complicated.
>>>
>>> It might be nice to add a new RSP protocol packet that asks for the address
>>> space names/values:
>>>
>>> qGetAddressSpaces
>>>
>>> which would return something like:
>>>
>>> 1:text;2:data1,3:data2
>>>
>>> or it would return not supported. If we get a valid return value from
>>> qGetAddressSpaces, then it enables the use of the new packet you added
>>> above. Else it defaults to using the old memory read functions.
>>>
>>>
Now I've hit an issue with expressions (LLVMUserExpression.cpp) and local
variables (DWARFExpressions.cpp). There is a lot of memory read/write
functions that take just an address argument. Is the only way to go to
patch all these calls? Has anybody solved it differently?
>>>
>>> My quick take is that any APIs that take just a lldb::addr_t would need to
>>> take something like:
>>>
>>> struct SpaceAddress {
>>> static constexpr uint32_t kNoSpace = 0;
>>> lldb::addr_t addr;
>>> uint32_t space;
>>> };
>>>
>>
>> I'm curious why you are suggesting another kind of address, rather than
>> adding this functionality to Address? When you actually go to resolve an
>> Address in a target with a process you should have everything you need to
>> know to give it the proper space. Then fixing the expression evaluator (and
>> anything else that needs fixing) would be a matter of consistently using
>> Address rather than lldb::addr_t. That seems general goodness, since
>> converting to an lldb::addr_t loses information.
>
> If we accept lldb_private::Address in all APIs that take a lldb::addr_t
> currently, then we need to always be able to get to the target in case we
> need to add code to resolve the address everywhere. I am thinking of
> SpaceAddress as an augmented lldb::addr_t instead of a section + offset style
> address. Also, there will be addresses in the code and data that do not exist
> in actual sections. Not saying that you couldn't use lldb_private::Address. I
> am open to suggestions though. So your though it remove all API that take
> lldb::addr_t and use lldb_private::Address everywhere all the time?
It has always bugged me that we have these two ways of specifying addresses.
Are there many/any places that have to resolve an Address to a real address in
a process that don't have a Target readily available? That would surprise me.
I would much rather centralize on one way than adding a third.
Jim
>>
>> Jim
>>
>>
>>> We would need a default value for "space" (feel free to rename) that
>>> indicates the default address space as most of our architectures would not
>>> need this support. If we added a constructor like:
>>>
>>> SpaceAddress(lldb::addr_t a) : addr(a), space(kNoSpace) {}
>>>
>>> Then all usages of the APIs that used to take just a "lldb::addr_t" would
>>> implicitly call this constructor and continue to act as needed. Then we
>>> would need to allow lldb_private::Address objects to resolve to a
>>> SpaceAddress:
>>>
>>> SpaceAddress lldb_private::Address::GetSpaceAddress(Target *target) const;
>>>
>>> Since each lldb_private::Address has a section and each section knows its
>>> address space. Then the tricky part is finding all locations in the
>>> expression parser and converting those to track and use SpaceAddress. We
>>> would probably need to modify the allocate memory packets in the RSP
>>> protocol to be able to allocate memory in any address space as well.
>>>
>>> I didn't spend much time think about correct names above, so feel free to
>>> suggest alternate naming.
>>>
>>> Best advice:
>>> - make things "just work" to keep changes to a minimum and allowing
>>> lldb::addr_t to implicitly convert to a SpaceAddress easily
>>> - when modifying RSP, make sure to check for existence of new feature
>>> before enabling it
>>> - query for address space names so when we dump SpaceAddress we can show
>>> something that means something to the user. This means we would need to
>>> query the address sp
Re: [lldb-dev] FileSpec and normalization questions
> On Apr 19, 2018, at 11:21 AM, Zachary Turner wrote: > > > > On Thu, Apr 19, 2018 at 11:19 AM Jim Ingham via lldb-dev > wrote: > The last time I looked at the llvm functions they only support the path > syntax of the llvm host, which won't do for lldb. But maybe they have gotten > more general recently? > > > Yes in fact I was the one who updated them to make them more general. You > can now specify an enumeration parameter which will cause the algorithm to > treat paths as either windows, posix, or whatever the host is. If I were guessing, I would have guessed that! Since you probably know how the llvm functions work better than most, were there technical reasons why, having done the work on the llvm side, you didn't adopt them in lldb? Or was it just a matter of time? Jim ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Advice on architectures with multiple address spaces
Hexagon has a single address space, so we don't need to do anything like this. When I worked on Motorola 56xxx DSPs we had memory spaces, but we didn't use RSP. We had our own protocol that used a struct for addreses, with the space (an enum, defined per supported core) and a uint32_t (later 2 of them) for the address. Ted -- Qualcomm Innovation Center, Inc. The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project > -Original Message- > From: lldb-dev [mailto:lldb-dev-boun...@lists.llvm.org] On Behalf Of Greg > Clayton via lldb-dev > Sent: Thursday, April 19, 2018 11:45 AM > To: Zdenek Prikryl > Cc: lldb-dev@lists.llvm.org > Subject: Re: [lldb-dev] Advice on architectures with multiple address spaces You might ask > the Hexagon folks if they have done anything in case they already support this > is some shape or form. > > Greg Clayton > ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] FileSpec and normalization questions
I can't see any reason why we benefit from having these differently spelled
equivalent paths floating around. You want to be careful to preserve the path
syntax, since it would be weird to be cross debugging to a Windows machine and
have to type Posix paths. But other than that, removing all the extraneous
cruft seems goodness to me.
Were you proposing just that you get the DWARF parser to do this, or were you
proposing that that become a requirement for SymbolFile parsers, so that we can
drop all the code that normalizes paths from debug information before
comparisons? We still have to normalize paths from users, but it would be nice
not to have to do it if we know the source is debug info.
Jim
> On Apr 19, 2018, at 11:14 AM, Greg Clayton via lldb-dev
> wrote:
>
> We currently have DWARF that has a DW_AT_comp_dir that is set to "./"
> followed by any number of extra '/' characters. I would like to have this
> path normalized as we parse the DWARF so we don't end up with line tables
> with a variety of ".//+" prefixes on each source file.
>
> While looking to solve this issue, I took a look at the functionality that is
> in FileSpec right now. In:
>
> void FileSpec::SetFile(llvm::StringRef pathname, bool resolve, PathSyntax
> syntax);
>
>
> This function always calls a cheaper normalize function:
>
> namespace {
> void Normalize(llvm::SmallVectorImpl &path, FileSpec::PathSyntax
> syntax);
> }
>
> This function does nothing for posix paths, but switches backward slashes to
> forward slashes.
>
> We have a FileSpec function named FileSpec::GetNormalizedPath() which will do
> much more path normalization on a path by removing redundant "." and ".." and
> "//".
>
> I can fix my DWARF issue in a few ways:
> 1 - fix FileSpec::SetFile() to still call ::Normalize() but have it do more
> work and have it normalize out the and redundant or relative path info
> 2 - call FileSpec::GetNormalizedPath() on each comp dir before using it to
> actually normalize it
>
> The main question is: do we want paths floating around LLDB that aren't
> normalized? It seems like having a mixture of theses path will lead to issues
> in LLDB so I would vote for solution #1.
>
> Also, looking at the tests for normalizing paths I found the following pairs
> of pre-normalized and post-normalization paths for posix:
>
> {"//", "//"},
> {"//net", "//net"},
>
> Why wouldn't we reduce "//" to just "/" for posix? And why wouldn't we reduce
> "//net" to "/net"?
>
> Also I found:
>
> {"./foo", "foo"},
>
> Do we prefer to not have "./foo" to stay as "./foo"? Seems like if users know
> that their debug info has line table entries that are "./foo/foo.c" that they
> might type this in:
>
> (lldb) b ./foo/foo.c:12
>
> But this will fail since it might not match the "foo/foo.c:12" that might
> result from path normalization. We don't normalize right now so it doesn't
> matter and things would match, but part of my fix is normalizing a path in
> the DWARF that is currently ".foo/foo.c" down to either "./foo/foo.c"
> or "foo/foo.c" so it will matter depending on what we decide here.
>
> Any input would be appreciated.
>
> Greg Clayton
>
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] FileSpec and normalization questions
We actually use it in some places, but it's limited. Before I did that was when I added the PathSyntax to FileSpec which essentially servers the same purpose. We could in theory drop PathSyntax now that LLVM supports all of the same functionality. It's a pretty invasive refactor though which I never had time to do. I think I might have tried to replace some of the low level functions in FileSpec with the LLVM equivalents and gotten a few test failures, but I didn't have time to investigate. It would be a worthwhile experiment for someone to try again if they have some cycles. On Thu, Apr 19, 2018 at 11:28 AM Jim Ingham wrote: > > > On Apr 19, 2018, at 11:21 AM, Zachary Turner wrote: > > > > > > > > On Thu, Apr 19, 2018 at 11:19 AM Jim Ingham via lldb-dev < > lldb-dev@lists.llvm.org> wrote: > > The last time I looked at the llvm functions they only support the path > syntax of the llvm host, which won't do for lldb. But maybe they have > gotten more general recently? > > > > > > Yes in fact I was the one who updated them to make them more general. > You can now specify an enumeration parameter which will cause the algorithm > to treat paths as either windows, posix, or whatever the host is. > > If I were guessing, I would have guessed that! > > Since you probably know how the llvm functions work better than most, were > there technical reasons why, having done the work on the llvm side, you > didn't adopt them in lldb? Or was it just a matter of time? > > Jim > > > ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Proposal: Using LLD in tests
Our Windows buildbots use msys for gnuisms. The makefiles in the test suite run fine with minimal modifications (just the object delete hack Zach put in to use del instead of rm; msys make doesn't accept cmd syntax while Cygwin make does). Now, that's using clang to build Hexagon binaries, but teaching the makefile to use cl syntax shouldn't be too hard. I've seen it done before; same makefile for windows and various unix derivatives, detect what OS you were running on and set CFLAGS/CXXFLAGS/LDFLAGS accordingly. Ted -- Qualcomm Innovation Center, Inc. The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project > -Original Message- > From: lldb-dev [mailto:lldb-dev-boun...@lists.llvm.org] On Behalf Of Pavel > Labath via lldb-dev > Sent: Thursday, April 19, 2018 12:45 PM > To: Leonard Mosescu > Cc: aaron.lee.sm...@gmail.com; LLDB > Subject: Re: [lldb-dev] Proposal: Using LLD in tests > > On Thu, 19 Apr 2018 at 18:19, Leonard Mosescu > wrote: > > >>the PDB tests under lit/SymbolFile/PDB need a linker to produce > >> the > program database > > > > With this proposal, would we preserve any coverage for MSVC produced > debug information? > > > Well.. the question there is what are you trying to test? Is it the fact your > debugger works with a particular compiler+linker combination (note that those > tests already compile with clang-cl), or that your pdb-parsing code is sane. > (integration vs. regression test). > > Historically we've only had the former kind of tests (dotest), and we've had > the > ability (and used it) to run those tests against different kinds of > compilers. This > is all nice, but it means that a specific test will be testing a different > thing for > every person who runs it. That's why I would like to build up a suite of more > regression-like tests (*). I would say that the tests under lit/*** should be > regression tests and our goal should be to remove as many system > dependencies as possible, and leave the job of testing integration with a > specific toolchain to "dotest" tests (**). > > Technically, the answer to your question is "no", because currently dotest > tests > don't know how to work with cl+link. Making that work would be an interesting > project (although a bit annoying as the Makefiles are full of gcc-isms). > However, I don't think that should stop us here. > > (*) Ideally I would like to leave even the compiler out of the equation for > these > tests, and make it so that the tests always run on the exact same set of > bytes. I > am hoping I will be able to write at least some tests using .s files. > However, I > don't think I will do that for all of them, because these files can be > long/verbose/tedious to write. > > (**) However, even "dotest" tests should have a "default" mode which is as > hermetic as possible. > ___ > lldb-dev mailing list > lldb-dev@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] FileSpec and normalization questions
> On Apr 19, 2018, at 11:33 AM, Jim Ingham wrote:
>
> I can't see any reason why we benefit from having these differently spelled
> equivalent paths floating around. You want to be careful to preserve the
> path syntax, since it would be weird to be cross debugging to a Windows
> machine and have to type Posix paths. But other than that, removing all the
> extraneous cruft seems goodness to me.
We translate all windows slashes all the time in any FileSpec, so it doesn't
really matter what the user types.
> Were you proposing just that you get the DWARF parser to do this, or were you
> proposing that that become a requirement for SymbolFile parsers, so that we
> can drop all the code that normalizes paths from debug information before
> comparisons? We still have to normalize paths from users, but it would be
> nice not to have to do it if we know the source is debug info.
>
I was proposing to fix FileSpec::SetFile(), when false is passed for the
"resolve" parameter, to always normalize all paths. Or I could just fix this
inside of SymbolFileDWARF. I would rather do this at the "FileSpec" level since
it will normalize our comparisons and just remove a ton of issue we can run
into. So I vote for fixing FileSpec to "do the right thing".
Greg
> Jim
>
>
>> On Apr 19, 2018, at 11:14 AM, Greg Clayton via lldb-dev
>> wrote:
>>
>> We currently have DWARF that has a DW_AT_comp_dir that is set to "./"
>> followed by any number of extra '/' characters. I would like to have this
>> path normalized as we parse the DWARF so we don't end up with line tables
>> with a variety of ".//+" prefixes on each source file.
>>
>> While looking to solve this issue, I took a look at the functionality that
>> is in FileSpec right now. In:
>>
>> void FileSpec::SetFile(llvm::StringRef pathname, bool resolve, PathSyntax
>> syntax);
>>
>>
>> This function always calls a cheaper normalize function:
>>
>> namespace {
>> void Normalize(llvm::SmallVectorImpl &path, FileSpec::PathSyntax
>> syntax);
>> }
>>
>> This function does nothing for posix paths, but switches backward slashes to
>> forward slashes.
>>
>> We have a FileSpec function named FileSpec::GetNormalizedPath() which will
>> do much more path normalization on a path by removing redundant "." and ".."
>> and "//".
>>
>> I can fix my DWARF issue in a few ways:
>> 1 - fix FileSpec::SetFile() to still call ::Normalize() but have it do more
>> work and have it normalize out the and redundant or relative path info
>> 2 - call FileSpec::GetNormalizedPath() on each comp dir before using it to
>> actually normalize it
>>
>> The main question is: do we want paths floating around LLDB that aren't
>> normalized? It seems like having a mixture of theses path will lead to
>> issues in LLDB so I would vote for solution #1.
>>
>> Also, looking at the tests for normalizing paths I found the following pairs
>> of pre-normalized and post-normalization paths for posix:
>>
>> {"//", "//"},
>> {"//net", "//net"},
>>
>> Why wouldn't we reduce "//" to just "/" for posix? And why wouldn't we
>> reduce "//net" to "/net"?
>>
>> Also I found:
>>
>> {"./foo", "foo"},
>>
>> Do we prefer to not have "./foo" to stay as "./foo"? Seems like if users
>> know that their debug info has line table entries that are "./foo/foo.c"
>> that they might type this in:
>>
>> (lldb) b ./foo/foo.c:12
>>
>> But this will fail since it might not match the "foo/foo.c:12" that might
>> result from path normalization. We don't normalize right now so it doesn't
>> matter and things would match, but part of my fix is normalizing a path in
>> the DWARF that is currently ".foo/foo.c" down to either
>> "./foo/foo.c" or "foo/foo.c" so it will matter depending on what we decide
>> here.
>>
>> Any input would be appreciated.
>>
>> Greg Clayton
>>
>> ___
>> lldb-dev mailing list
>> lldb-dev@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
