Re: [lldb-dev] [llvm-dev] LLVM 9.0.1-rc2 has been tagged

2019-12-11 Thread Diana Picus via lldb-dev
Hi,

Uploaded ARM & AArch64:
b0e4160da05a2a734cc2e73b925003a7595798f9544c522fa3e3e5815451b861
clang+llvm-9.0.1-rc2-aarch64-linux-gnu.tar.xz
2c5c08fd4a5fb290d3f148b806bb708832239030fe39529746de3e746bbf5cdd
clang+llvm-9.0.1-rc2-armv7a-linux-gnueabihf.tar.xz

Same as rc1.

Cheers,
Diana

On Sat, 7 Dec 2019 at 04:03, Tom Stellard via llvm-dev
 wrote:
>
> Hi,
>
> I've tagged LLVM 9.0.1-rc2.  Testers can begin testing and uploading binaries.
> If all goes well, this will be the last -rc.
>
> -Tom
>
> ___
> LLVM Developers mailing list
> llvm-...@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


[lldb-dev] The future of modern-type-lookup in LLDB

2019-12-11 Thread Raphael “Teemperor” Isemann via lldb-dev
Hi,

some of you may have seen that I’ve been working on the ‘modern-type-lookup’ 
mode in LLDB over the last months. I thought it’s a good time now to start a 
discussion about what’s the current state, goals and the future of this feature.

Some background about the topic to get everyone on the same page:
* LLDB uses Clang’s AST to model the types/functions/variables/etc. in the 
target program.
* Clang’s AST is usually represented as a ASTContext (= AST + associated data).
* LLDB has several ASTContexts for different purposes:
  - Usually we have several ASTContexts that just contain the AST nodes we 
create from debug information on the disk.
  - There are some minor ones floating around like the ASTContext we use to 
store AST nodes that we load from Clang modules (*.pcm files).
  - There are temporary ASTContexts we create to run expressions (every 
expression is its own temporary ASTContext that gets deleted after we ran the 
expression).
  - There is one ASTContext that is the one we put in all the persistent 
$-variables and their associated types.
* The last two ASTContexts don’t have any associated data source that they 
represent but are supposed to be ‘filled' by the other two kinds of ASTContexts.
* To fill an ASTContext we move AST nodes from one ASTContext to another.
* Moving ASTNodes is done with clang’s ASTImporter implementation which 
‘imports’ AST nodes.
* Nearly all importing in LLDB is done lazily which the ASTImporter calls 
‘MinimalImport’. This means the ASTImporter only imports what it needs for 
certain declarations and then imports the rest as it is needed.
* The lazy importing is a big source of problems in LLDB. If we don’t correctly 
import enough information to make Clang happy then we usually end up hitting an 
assert. However it is also avoiding the loading unnecessary debug information 
from disk which makes LLDB faster. There are no accurate numbers on how much 
faster as we don’t have an easy way to run LLDB without MinimalImport.

Now let’s move on to the modern-type-lookup part.

What is modern-type-lookup?
* modern-type-lookup is a flag that makes LLDB use `clang::ExternalASTMerger` 
instead of directly using the `clang::ASTImporter` via our ClangASTImporter 
wrapper.
* `clang::ExternalASTMerger` is some kind of manager for clang’s ASTImporter. 
It keeps track of several ASTImporter instances (that each have an associate 
ASTContext source) and one target ASTContext that all ASTImporters import nodes 
into. When the ASTImporters import certain nodes into the single target 
ASTContext it does the bookkeeping to associate the imported information with 
the ASTImporter/source ASTContext.
* The ExternalASTMerger also does some other smaller book keeping such as 
having a ‘reverse ASTImporter’ for every ‘ASTImporter’.
* The ExternalASTMerger is only used by LLDB (and clang-import-test which is 
its testing binary).

What is good about modern-type-lookup:
* The ExternalASTMerger is better code than our current system of directly 
(ab-)using the ASTImporter. Most notably it doesn’t use globals like our 
current system (yay).
* The ExternalASTMerger is easer to test because it comes with a testing binary 
(clang-import-test) and it doesn’t depend on anything beside the ASTImporter. 
In comparison our current system depends on literally all of LLDB.
* It brings better organisation to our ASTImporter network which allows us to 
implement some tricky features (e.g. https://reviews.llvm.org/D67803).
* It actually is a quite small (but sadly very invasive) change into LLDB and 
Clang.

What is bad and ugly about modern-type-lookup:
* modern-type-lookup in LLDB was completely untested until recently. The idea 
was the testing was done by running the whole test suite with the setting 
enabled by default [1] and then fix the found issues [2], but none of this 
happened for various reasons. The only dedicated tests for it are the ones I 
added while I was trying to see what it even does (see the 
functionalities/modern-type-lookup folder).
* It doesn’t work as of now. I fixed most of the failing tests but the 
remaining ones are not trivial to fix (and fixing some of them seems to break 
others again). Here is a full test run of the test suit with modern-type-lookup 
enabled by default [3]. Note that is was run on Linux and we have more tests 
and failures that are only running on macOS (Mostly Objective-C tests).
* We need to maintain both systems in theory. In practice we just ignore 
modern-type-lookup as it had no tests. Some of the test failures are due to 
features we only added to the old system (e.g. the import-std-module failures 
are relying on features of the old system. Non-trivial -gmodules importing is 
also broken).
* It wasn’t developed in a way that allows an incremental migration. If we ever 
turn it on by default we can only do it by doing one big switch over. We also 
can’t just turn it on for certain parts of LLDB or anything like that.
* It’s not clear if the