[lldb-dev] [Bug 24679] New: Remove pexpect dependency from tests unless it's absolutely necessary
https://llvm.org/bugs/show_bug.cgi?id=24679 Bug ID: 24679 Summary: Remove pexpect dependency from tests unless it's absolutely necessary Product: lldb Version: unspecified Hardware: All OS: All Status: NEW Severity: normal Priority: P Component: All Bugs Assignee: lldb-dev@lists.llvm.org Reporter: ztur...@google.com CC: llvm-b...@lists.llvm.org Classification: Unclassified pexpect should only be used in tests that require by definition the ability to launch an lldb.exe from the command line in order to test argument parsing, etc. There are tests that use pexpect anyway even though this is not required. Those tests should be re-written using the SB API, or if that is not possible, unit tests. For starters, this includes the tests in TestCompletion.py, but new instances of this occuring should be added to this bug for tracking. -- You are receiving this mail because: You are the assignee for the bug. ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
[lldb-dev] Portable tests that create threads
Historically the pattern for tests that need to test situations involving threads has been to write the tests using pthreads, and then a common pattern on the python side of the test is to assert that the actual number of threads equals the expected number of threads. To deal with the pthread portability issue, I ported most of these tests over to std::threads, but there is still another issue with them. On Windows we can't assume anything about the number of threads in a program. For example, a single-threaded hello world program actually has 3 threads, 2 of which are reserved for use by the operating system. And the fact that there's 2 is just arbitrary, that might not remain the same between compiler versions, operating system versions, etc. And to make matters worse it might not even remain constant through the life of the program (the assumption just being that we cannot make any assumptions, since we don't control the threads). So I need to find a way to write these tests portably. Do all platforms have a way to assign a thread a name? Looking in Host/*/ThisThread.cpp, it seems: 1) Linux and MacOSX inferiors can use pthread_setname_np 2) FreeBSD inferiors can use pthread_set_name_np 3) Windows inferiors can raise a magic exception which the debugger is expected to recognize Based on this, it seems that one possible way to write these tests portably is to have the test inferiors assign names to the threads, and have the tests index the threads by name instead of by ordinal. Does anyone have any thoughts on this or other possible approaches? I really would like to avoid having different inferiors or different tests for all of this functionality, as it will be very easy to get out of sync. ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Portable tests that create threads
One idea is add the ability to discover which of any of the current threads are or are not user created. This might help us. On MacOSX, the main thread would always be considered user created, and any other thread, it is quite easy to tell. User created threads could be ones whose bottom stack frame is "thread_start" frame #8: 0x7fff9b47c41d libsystem_pthread.dylib thread_start + 13 Where OS created threads have their last frames in "start_wqthread" or "_dispatch_mgr_thread": ... frame #2: 0x7fff9b47c40d libsystem_pthread.dylib start_wqthread + 13 or: ... frame #2: 0x7fff9c298a6a libdispatch.dylib _dispatch_mgr_thread + 52 So it might be easy to add API to SBThread like: bool SBThread::IsUserCreated(); Then we could teach the tests to filter out any threads that are not user created? Greg > On Sep 2, 2015, at 12:31 PM, Zachary Turner via lldb-dev > wrote: > > Historically the pattern for tests that need to test situations involving > threads has been to write the tests using pthreads, and then a common pattern > on the python side of the test is to assert that the actual number of threads > equals the expected number of threads. > > To deal with the pthread portability issue, I ported most of these tests over > to std::threads, but there is still another issue with them. > > On Windows we can't assume anything about the number of threads in a program. > For example, a single-threaded hello world program actually has 3 threads, 2 > of which are reserved for use by the operating system. And the fact that > there's 2 is just arbitrary, that might not remain the same between compiler > versions, operating system versions, etc. And to make matters worse it might > not even remain constant through the life of the program (the assumption just > being that we cannot make any assumptions, since we don't control the > threads). > > So I need to find a way to write these tests portably. > > Do all platforms have a way to assign a thread a name? Looking in > Host/*/ThisThread.cpp, it seems: > > 1) Linux and MacOSX inferiors can use pthread_setname_np > 2) FreeBSD inferiors can use pthread_set_name_np > 3) Windows inferiors can raise a magic exception which the debugger is > expected to recognize > > Based on this, it seems that one possible way to write these tests portably > is to have the test inferiors assign names to the threads, and have the tests > index the threads by name instead of by ordinal. > > Does anyone have any thoughts on this or other possible approaches? > > I really would like to avoid having different inferiors or different tests > for all of this functionality, as it will be very easy to get out of sync. > ___ > lldb-dev mailing list > lldb-dev@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Portable tests that create threads
That might work, on Windows I think we could identify this by looking for ntdll.dll!TppWorkerThread() in the stack frame. This might not be exactly the same across OS versions, but we could handle all cases as new ones are discovered. We would have to make sure that all developers are educated on the distinction, so that only user threads are queried in future tests, but it has the advantage of not needing any modifications to the inferior On Wed, Sep 2, 2015 at 1:06 PM Greg Clayton wrote: > One idea is add the ability to discover which of any of the current > threads are or are not user created. This might help us. > > On MacOSX, the main thread would always be considered user created, and > any other thread, it is quite easy to tell. User created threads could be > ones whose bottom stack frame is "thread_start" > > > frame #8: 0x7fff9b47c41d libsystem_pthread.dylib thread_start + 13 > > Where OS created threads have their last frames in "start_wqthread" or > "_dispatch_mgr_thread": > > ... > frame #2: 0x7fff9b47c40d libsystem_pthread.dylib start_wqthread + > 13 > > or: > > ... > frame #2: 0x7fff9c298a6a libdispatch.dylib _dispatch_mgr_thread + > 52 > > > So it might be easy to add API to SBThread like: > > bool > SBThread::IsUserCreated(); > > Then we could teach the tests to filter out any threads that are not user > created? > > Greg > > > > On Sep 2, 2015, at 12:31 PM, Zachary Turner via lldb-dev < > lldb-dev@lists.llvm.org> wrote: > > > > Historically the pattern for tests that need to test situations > involving threads has been to write the tests using pthreads, and then a > common pattern on the python side of the test is to assert that the actual > number of threads equals the expected number of threads. > > > > To deal with the pthread portability issue, I ported most of these tests > over to std::threads, but there is still another issue with them. > > > > On Windows we can't assume anything about the number of threads in a > program. For example, a single-threaded hello world program actually has 3 > threads, 2 of which are reserved for use by the operating system. And the > fact that there's 2 is just arbitrary, that might not remain the same > between compiler versions, operating system versions, etc. And to make > matters worse it might not even remain constant through the life of the > program (the assumption just being that we cannot make any assumptions, > since we don't control the threads). > > > > So I need to find a way to write these tests portably. > > > > Do all platforms have a way to assign a thread a name? Looking in > Host/*/ThisThread.cpp, it seems: > > > > 1) Linux and MacOSX inferiors can use pthread_setname_np > > 2) FreeBSD inferiors can use pthread_set_name_np > > 3) Windows inferiors can raise a magic exception which the debugger is > expected to recognize > > > > Based on this, it seems that one possible way to write these tests > portably is to have the test inferiors assign names to the threads, and > have the tests index the threads by name instead of by ordinal. > > > > Does anyone have any thoughts on this or other possible approaches? > > > > I really would like to avoid having different inferiors or different > tests for all of this functionality, as it will be very easy to get out of > sync. > > ___ > > lldb-dev mailing list > > lldb-dev@lists.llvm.org > > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev > > ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
[lldb-dev] [Bug 24681] New: Allow LLDB to distinguish between user threads and system threads
https://llvm.org/bugs/show_bug.cgi?id=24681 Bug ID: 24681 Summary: Allow LLDB to distinguish between user threads and system threads Product: lldb Version: unspecified Hardware: PC OS: Windows NT Status: NEW Severity: normal Priority: P Component: All Bugs Assignee: lldb-dev@lists.llvm.org Reporter: ztur...@google.com CC: llvm-b...@lists.llvm.org Classification: Unclassified Various tests are written to test multithreaded functionality of LLDB by creating a number of threads, and then verifying that the expected and actual numbers of threads match. This doesn't work in the presence of threads which are created by the operating system, and on Windows this can happen even in a trivial single-threaded hello world application. To fix this, we should add to LLDB the notion of a user-created thread versus a system thread, and expose this through the SB API in a way that the tests can query only the user threads. On Windows this can be done by checking for the presence of ntdll!TppWorkerThread in the call stack, and on Mac this can be done by checking for start_wqthread or _dispatch_mgr_thread as the last frame of the callstack. Similar methods probably can be found for other platforms. For now, the following tests are XFAIL'ed on Windows until this is resolved: TestExitDuringStep.ExitDuringStepTestCase.test_step_in_with_dwarf TestExitDuringStep.ExitDuringStepTestCase.test_step_over_with_dwarf TestExitDuringStep.ExitDuringStepTestCase.test_thread_state_is_stopped_with_dwarf TestExitDuringStep.ExitDuringStepTestCase.test_with_dwarf TestThreadExit.ThreadExitTestCase.test_with_dwarf TestThreadStepOut.ThreadStepOutTestCase.test_python_with_dwarf -- You are receiving this mail because: You are the assignee for the bug. ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Portable tests that create threads
I don't think that's a good plan. It's the implementation of std::thread that created the thread pool. It's quite possible that it then re-purposing thread pool threads as user threads. This gets into implementation details that we'd have to figure out and which could change. Even with the equivalent of a single-threaded Hello World program, Zach and I are seeing a different number of threads on different versions of Windows. I'm in favor of having the inferior somehow designate the threads it's intentionally creating (or causing to be created) and having the tests look for those designations. I like the naming idea. On Wed, Sep 2, 2015 at 1:17 PM, Zachary Turner via lldb-dev < lldb-dev@lists.llvm.org> wrote: > That might work, on Windows I think we could identify this by looking > for ntdll.dll!TppWorkerThread() in the stack frame. This might not be > exactly the same across OS versions, but we could handle all cases as new > ones are discovered. > > We would have to make sure that all developers are educated on the > distinction, so that only user threads are queried in future tests, but it > has the advantage of not needing any modifications to the inferior > > On Wed, Sep 2, 2015 at 1:06 PM Greg Clayton wrote: > >> One idea is add the ability to discover which of any of the current >> threads are or are not user created. This might help us. >> >> On MacOSX, the main thread would always be considered user created, and >> any other thread, it is quite easy to tell. User created threads could be >> ones whose bottom stack frame is "thread_start" >> >> >> frame #8: 0x7fff9b47c41d libsystem_pthread.dylib thread_start + 13 >> >> Where OS created threads have their last frames in "start_wqthread" or >> "_dispatch_mgr_thread": >> >> ... >> frame #2: 0x7fff9b47c40d libsystem_pthread.dylib start_wqthread + >> 13 >> >> or: >> >> ... >> frame #2: 0x7fff9c298a6a libdispatch.dylib _dispatch_mgr_thread + >> 52 >> >> >> So it might be easy to add API to SBThread like: >> >> bool >> SBThread::IsUserCreated(); >> >> Then we could teach the tests to filter out any threads that are not user >> created? >> >> Greg >> >> >> > On Sep 2, 2015, at 12:31 PM, Zachary Turner via lldb-dev < >> lldb-dev@lists.llvm.org> wrote: >> > >> > Historically the pattern for tests that need to test situations >> involving threads has been to write the tests using pthreads, and then a >> common pattern on the python side of the test is to assert that the actual >> number of threads equals the expected number of threads. >> > >> > To deal with the pthread portability issue, I ported most of these >> tests over to std::threads, but there is still another issue with them. >> > >> > On Windows we can't assume anything about the number of threads in a >> program. For example, a single-threaded hello world program actually has 3 >> threads, 2 of which are reserved for use by the operating system. And the >> fact that there's 2 is just arbitrary, that might not remain the same >> between compiler versions, operating system versions, etc. And to make >> matters worse it might not even remain constant through the life of the >> program (the assumption just being that we cannot make any assumptions, >> since we don't control the threads). >> > >> > So I need to find a way to write these tests portably. >> > >> > Do all platforms have a way to assign a thread a name? Looking in >> Host/*/ThisThread.cpp, it seems: >> > >> > 1) Linux and MacOSX inferiors can use pthread_setname_np >> > 2) FreeBSD inferiors can use pthread_set_name_np >> > 3) Windows inferiors can raise a magic exception which the debugger is >> expected to recognize >> > >> > Based on this, it seems that one possible way to write these tests >> portably is to have the test inferiors assign names to the threads, and >> have the tests index the threads by name instead of by ordinal. >> > >> > Does anyone have any thoughts on this or other possible approaches? >> > >> > I really would like to avoid having different inferiors or different >> tests for all of this functionality, as it will be very easy to get out of >> sync. >> > ___ >> > lldb-dev mailing list >> > lldb-dev@lists.llvm.org >> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev >> >> > ___ > lldb-dev mailing list > lldb-dev@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev > > ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Portable tests that create threads
> On Sep 2, 2015, at 1:06 PM, Greg Clayton via lldb-dev > wrote: > > One idea is add the ability to discover which of any of the current threads > are or are not user created. This might help us. > > On MacOSX, the main thread would always be considered user created, and any > other thread, it is quite easy to tell. User created threads could be ones > whose bottom stack frame is "thread_start" > > >frame #8: 0x7fff9b47c41d libsystem_pthread.dylib thread_start + 13 > > Where OS created threads have their last frames in "start_wqthread" or > "_dispatch_mgr_thread": > >... >frame #2: 0x7fff9b47c40d libsystem_pthread.dylib start_wqthread + 13 > > or: > >... >frame #2: 0x7fff9c298a6a libdispatch.dylib _dispatch_mgr_thread + 52 > > > So it might be easy to add API to SBThread like: > > bool > SBThread::IsUserCreated(); > > Then we could teach the tests to filter out any threads that are not user > created? I'm not sure on OS X this is a terribly useful distinction. There are lots of threads that are created to run user code through the dispatch library. They won't start with "thread_start" but I don't think most users would think of them as "not user threads..." Jim > > Greg > > >> On Sep 2, 2015, at 12:31 PM, Zachary Turner via lldb-dev >> wrote: >> >> Historically the pattern for tests that need to test situations involving >> threads has been to write the tests using pthreads, and then a common >> pattern on the python side of the test is to assert that the actual number >> of threads equals the expected number of threads. >> >> To deal with the pthread portability issue, I ported most of these tests >> over to std::threads, but there is still another issue with them. >> >> On Windows we can't assume anything about the number of threads in a >> program. For example, a single-threaded hello world program actually has 3 >> threads, 2 of which are reserved for use by the operating system. And the >> fact that there's 2 is just arbitrary, that might not remain the same >> between compiler versions, operating system versions, etc. And to make >> matters worse it might not even remain constant through the life of the >> program (the assumption just being that we cannot make any assumptions, >> since we don't control the threads). >> >> So I need to find a way to write these tests portably. >> >> Do all platforms have a way to assign a thread a name? Looking in >> Host/*/ThisThread.cpp, it seems: >> >> 1) Linux and MacOSX inferiors can use pthread_setname_np >> 2) FreeBSD inferiors can use pthread_set_name_np >> 3) Windows inferiors can raise a magic exception which the debugger is >> expected to recognize >> >> Based on this, it seems that one possible way to write these tests portably >> is to have the test inferiors assign names to the threads, and have the >> tests index the threads by name instead of by ordinal. >> >> Does anyone have any thoughts on this or other possible approaches? >> >> I really would like to avoid having different inferiors or different tests >> for all of this functionality, as it will be very easy to get out of sync. >> ___ >> lldb-dev mailing list >> lldb-dev@lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev > > ___ > lldb-dev mailing list > lldb-dev@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
[lldb-dev] type completion
I'm trying to implement a DWARFASTParser for go, and have hit an issue with
fields for structs.
As I understand it, DWARFASTParserClang loads minimal type info for structs
at first, and registers the type in SymbolFileDWARF's
m_forward_decl_clang_type_to_die.
Then later when you call type.GetFullCompilerType() it calls back into the
DWARFASTParser to complete the type.
But it seems like the SBValue's in the python API only get Forward types,
and never complete them. For example, I do:
var = frame.FindVariable('theStruct')
self.assertEqual(2, var.GetNumChildren())
But I get 0 children, because the type is incomplete and there doesn't seem
to be any way to complete it.
ValueObject::GetNumChildren goes through
ValueObject::MaybeCalculateCompleteType, which seems promising. But it only
completes the type for objc.
Would a clang variable already have a full type at this point for some
reason? If so, how do I arrange that for go?
Also, TypeSystem has a GetCompleteType method. I don't understand when this
would be called or how I can implement it without a reference to
SymbolFileDWARF.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] type completion
> On Sep 2, 2015, at 3:15 PM, Ryan Brown via lldb-dev
> wrote:
>
> I'm trying to implement a DWARFASTParser for go, and have hit an issue with
> fields for structs.
> As I understand it, DWARFASTParserClang loads minimal type info for structs
> at first, and registers the type in SymbolFileDWARF's
> m_forward_decl_clang_type_to_die.
> Then later when you call type.GetFullCompilerType() it calls back into the
> DWARFASTParser to complete the type.
>
> But it seems like the SBValue's in the python API only get Forward types, and
> never complete them. For example, I do:
> var = frame.FindVariable('theStruct')
> self.assertEqual(2, var.GetNumChildren())
>
> But I get 0 children, because the type is incomplete and there doesn't seem
> to be any way to complete it.
> ValueObject::GetNumChildren goes through
> ValueObject::MaybeCalculateCompleteType, which seems promising. But it only
> completes the type for objc.
> Would a clang variable already have a full type at this point for some
> reason? If so, how do I arrange that for go?
>
> Also, TypeSystem has a GetCompleteType method. I don't understand when this
> would be called or how I can implement it without a reference to
> SymbolFileDWARF.
The story goes:
lldb_private::ClangASTContext inherits from lldb_private::TypeSystem so it is
the type system. The ClangASTContext also signs up to be a
clang::ExternalASTSource so that it can complete types via:
ClangASTContext::CompleteTagDecl()
ClangASTContext::CompleteObjCInterfaceDecl()
ClangASTContext::CompleteTagDecl() completes C and C++ structs, unions and
classes.
ClangASTContext::CompleteObjCInterfaceDecl() completes Objective C stuff.
The function that actually figures out how many children will be routed through
"CompilerType::GetNumChildren(bool)" with code like this:
size_t
ValueObjectVariable::CalculateNumChildren()
{
CompilerType type(GetCompilerType());
if (!type.IsValid())
return 0;
const bool omit_empty_base_classes = true;
return type.GetNumChildren(omit_empty_base_classes);
}
CompilerType passes the ball back to the TypeSystem class (ClangASTContext in
this case):
uint32_t
CompilerType::GetNumChildren (bool omit_empty_base_classes) const
{
if (!IsValid())
return 0;
return m_type_system->GetNumChildren(m_type, omit_empty_base_classes);
}
Down in ClangASTContext it does:
uint32_t
ClangASTContext::GetNumChildren (void* type, bool omit_empty_base_classes)
{
if (!type)
return 0;
uint32_t num_children = 0;
clang::QualType qual_type(GetQualType(type));
const clang::Type::TypeClass type_class = qual_type->getTypeClass();
switch (type_class)
{
...
case clang::Type::Record:
if (GetCompleteQualType (getASTContext(), qual_type))
{
So it is a static function named GetCompleteQualType() in ClangASTContext.cpp
that completes the type if it needs to:
static bool
GetCompleteQualType (clang::ASTContext *ast, clang::QualType qual_type, bool
allow_completion = true)
{
const clang::Type::TypeClass type_class = qual_type->getTypeClass();
switch (type_class)
{
...
case clang::Type::Record:
case clang::Type::Enum:
{
const clang::TagType *tag_type =
llvm::dyn_cast(qual_type.getTypePtr());
if (tag_type)
{
clang::TagDecl *tag_decl = tag_type->getDecl();
if (tag_decl)
{
if (tag_decl->isCompleteDefinition())
return true;
if (!allow_completion)
return false;
if (tag_decl->hasExternalLexicalStorage())
{
if (ast)
{
clang::ExternalASTSource *external_ast_source =
ast->getExternalSource();
if (external_ast_source)
{
external_ast_source->CompleteType(tag_decl);
So it will ask the clang::ExternalASTSource to complete the type which should
call ClangASTContext::CompleteTagDecl() to complete your type.
So this is how it would work if you used DWARFASTParserClang. Are you using
that? Or are you using your own DWARFASTParserGo? If so, then you need to
implement this type completion in your own TypeSystem. Your type system
subclass is required to implement many different function for type
introspection, one of which is:
class TypeSystem {
virtual uint32_t
GetNumChildren (void *type, bool omit_empty_base_classes) = 0;
};
So the clang version in ClangASTContext::GetNumChildren() knows to complete the
type if it a forward declaration. If you have a GoASTContext class that
inherits from TypeSystem, then all function that might need to know about the
contents of a class or struct, will need
Re: [lldb-dev] type completion
Also: lldb_private::TypeSystem has a "SymbolFile *" registered with it:
virtual SymbolFile *
GetSymbolFile () const
{
return m_sym_file;
}
// Returns true if the symbol file changed during the set accessor.
virtual void
SetSymbolFile (SymbolFile *sym_file)
{
m_sym_file = sym_file;
}
So it can use the:
bool
SymbolFile::CompleteType (CompilerType &clang_type);
This is what ClangASTContext::CompleteTagDecl() uses:
void
ClangASTContext::CompleteTagDecl (void *baton, clang::TagDecl *decl)
{
ClangASTContext *ast = (ClangASTContext *)baton;
SymbolFile *sym_file = ast->GetSymbolFile();
if (sym_file)
{
CompilerType clang_type = GetTypeForDecl (decl);
if (clang_type)
sym_file->CompleteType (clang_type);
}
}
> On Sep 2, 2015, at 3:48 PM, Greg Clayton via lldb-dev
> wrote:
>
>>
>> On Sep 2, 2015, at 3:15 PM, Ryan Brown via lldb-dev
>> wrote:
>>
>> I'm trying to implement a DWARFASTParser for go, and have hit an issue with
>> fields for structs.
>> As I understand it, DWARFASTParserClang loads minimal type info for structs
>> at first, and registers the type in SymbolFileDWARF's
>> m_forward_decl_clang_type_to_die.
>> Then later when you call type.GetFullCompilerType() it calls back into the
>> DWARFASTParser to complete the type.
>>
>> But it seems like the SBValue's in the python API only get Forward types,
>> and never complete them. For example, I do:
>> var = frame.FindVariable('theStruct')
>> self.assertEqual(2, var.GetNumChildren())
>>
>> But I get 0 children, because the type is incomplete and there doesn't seem
>> to be any way to complete it.
>> ValueObject::GetNumChildren goes through
>> ValueObject::MaybeCalculateCompleteType, which seems promising. But it only
>> completes the type for objc.
>> Would a clang variable already have a full type at this point for some
>> reason? If so, how do I arrange that for go?
>>
>> Also, TypeSystem has a GetCompleteType method. I don't understand when this
>> would be called or how I can implement it without a reference to
>> SymbolFileDWARF.
>
> The story goes:
>
> lldb_private::ClangASTContext inherits from lldb_private::TypeSystem so it is
> the type system. The ClangASTContext also signs up to be a
> clang::ExternalASTSource so that it can complete types via:
>
> ClangASTContext::CompleteTagDecl()
> ClangASTContext::CompleteObjCInterfaceDecl()
>
> ClangASTContext::CompleteTagDecl() completes C and C++ structs, unions and
> classes.
>
> ClangASTContext::CompleteObjCInterfaceDecl() completes Objective C stuff.
>
> The function that actually figures out how many children will be routed
> through "CompilerType::GetNumChildren(bool)" with code like this:
>
> size_t
> ValueObjectVariable::CalculateNumChildren()
> {
>CompilerType type(GetCompilerType());
>
>if (!type.IsValid())
>return 0;
>
>const bool omit_empty_base_classes = true;
>return type.GetNumChildren(omit_empty_base_classes);
> }
>
>
> CompilerType passes the ball back to the TypeSystem class (ClangASTContext
> in this case):
>
> uint32_t
> CompilerType::GetNumChildren (bool omit_empty_base_classes) const
> {
>if (!IsValid())
>return 0;
>return m_type_system->GetNumChildren(m_type, omit_empty_base_classes);
> }
>
>
> Down in ClangASTContext it does:
>
>
> uint32_t
> ClangASTContext::GetNumChildren (void* type, bool omit_empty_base_classes)
> {
>if (!type)
>return 0;
>
>uint32_t num_children = 0;
>clang::QualType qual_type(GetQualType(type));
>const clang::Type::TypeClass type_class = qual_type->getTypeClass();
>switch (type_class)
>{
>...
>case clang::Type::Record:
>if (GetCompleteQualType (getASTContext(), qual_type))
>{
>
>
>
>
> So it is a static function named GetCompleteQualType() in ClangASTContext.cpp
> that completes the type if it needs to:
>
>
> static bool
> GetCompleteQualType (clang::ASTContext *ast, clang::QualType qual_type, bool
> allow_completion = true)
> {
>const clang::Type::TypeClass type_class = qual_type->getTypeClass();
>switch (type_class)
>{
>...
>case clang::Type::Record:
>case clang::Type::Enum:
>{
>const clang::TagType *tag_type =
> llvm::dyn_cast(qual_type.getTypePtr());
>if (tag_type)
>{
>clang::TagDecl *tag_decl = tag_type->getDecl();
>if (tag_decl)
>{
>if (tag_decl->isCompleteDefinition())
>return true;
>
>if (!allow_completion)
>return false;
>
>if (tag_decl->hasExternalLexicalStorage())
>{
>if (ast)
>{
>clang::ExternalASTSource *external_ast_source =
> ast->getExternalSource()
Re: [lldb-dev] type completion
Note that completing types is a bit more complex with clang::ASTContext based
types because of it often uses the clang::Decl to do the type completion so we
need to maintain that mapping. It might not be that hard in Go as you could
just do:
CompilerType my_type = ...;
if (my_type.GetCompleteType())
{
...
}
And this could just call down into your SymbolFile. But in clang we often get
to the point where we are playing around with types from a pure
clang::ASTContext stand point (like in the expression parser) and the clang
expression parser knows how to complete types via clang::ExternalASTSource, so
the expression parser when playing around with clang::Decl objects can cause
them to complete themselves. This allows us to hand the clang expression parser
a forward decl to a "Foo" type and let the expression parser complete the type
only if it needs to. If the expression parser just has a "Foo *" that it uses
and it just passed that to a function, it never needs to complete "Foo", but if
you do "Foo *foo = ...; foo->m_int" then you would need to complete it. So we
make all struct, unions, and classes able to lazily complete themselves for
performance reasons.
Greg
> On Sep 2, 2015, at 4:07 PM, Greg Clayton wrote:
>
> Also: lldb_private::TypeSystem has a "SymbolFile *" registered with it:
>
>virtual SymbolFile *
>GetSymbolFile () const
>{
>return m_sym_file;
>}
>
>// Returns true if the symbol file changed during the set accessor.
>virtual void
>SetSymbolFile (SymbolFile *sym_file)
>{
>m_sym_file = sym_file;
>}
>
> So it can use the:
>
> bool
> SymbolFile::CompleteType (CompilerType &clang_type);
>
> This is what ClangASTContext::CompleteTagDecl() uses:
>
> void
> ClangASTContext::CompleteTagDecl (void *baton, clang::TagDecl *decl)
> {
>ClangASTContext *ast = (ClangASTContext *)baton;
>SymbolFile *sym_file = ast->GetSymbolFile();
>if (sym_file)
>{
>CompilerType clang_type = GetTypeForDecl (decl);
>if (clang_type)
>sym_file->CompleteType (clang_type);
>}
> }
>
>
>> On Sep 2, 2015, at 3:48 PM, Greg Clayton via lldb-dev
>> wrote:
>>
>>>
>>> On Sep 2, 2015, at 3:15 PM, Ryan Brown via lldb-dev
>>> wrote:
>>>
>>> I'm trying to implement a DWARFASTParser for go, and have hit an issue with
>>> fields for structs.
>>> As I understand it, DWARFASTParserClang loads minimal type info for structs
>>> at first, and registers the type in SymbolFileDWARF's
>>> m_forward_decl_clang_type_to_die.
>>> Then later when you call type.GetFullCompilerType() it calls back into the
>>> DWARFASTParser to complete the type.
>>>
>>> But it seems like the SBValue's in the python API only get Forward types,
>>> and never complete them. For example, I do:
>>> var = frame.FindVariable('theStruct')
>>> self.assertEqual(2, var.GetNumChildren())
>>>
>>> But I get 0 children, because the type is incomplete and there doesn't seem
>>> to be any way to complete it.
>>> ValueObject::GetNumChildren goes through
>>> ValueObject::MaybeCalculateCompleteType, which seems promising. But it only
>>> completes the type for objc.
>>> Would a clang variable already have a full type at this point for some
>>> reason? If so, how do I arrange that for go?
>>>
>>> Also, TypeSystem has a GetCompleteType method. I don't understand when this
>>> would be called or how I can implement it without a reference to
>>> SymbolFileDWARF.
>>
>> The story goes:
>>
>> lldb_private::ClangASTContext inherits from lldb_private::TypeSystem so it
>> is the type system. The ClangASTContext also signs up to be a
>> clang::ExternalASTSource so that it can complete types via:
>>
>> ClangASTContext::CompleteTagDecl()
>> ClangASTContext::CompleteObjCInterfaceDecl()
>>
>> ClangASTContext::CompleteTagDecl() completes C and C++ structs, unions and
>> classes.
>>
>> ClangASTContext::CompleteObjCInterfaceDecl() completes Objective C stuff.
>>
>> The function that actually figures out how many children will be routed
>> through "CompilerType::GetNumChildren(bool)" with code like this:
>>
>> size_t
>> ValueObjectVariable::CalculateNumChildren()
>> {
>> CompilerType type(GetCompilerType());
>>
>> if (!type.IsValid())
>> return 0;
>>
>> const bool omit_empty_base_classes = true;
>> return type.GetNumChildren(omit_empty_base_classes);
>> }
>>
>>
>> CompilerType passes the ball back to the TypeSystem class (ClangASTContext
>> in this case):
>>
>> uint32_t
>> CompilerType::GetNumChildren (bool omit_empty_base_classes) const
>> {
>> if (!IsValid())
>> return 0;
>> return m_type_system->GetNumChildren(m_type, omit_empty_base_classes);
>> }
>>
>>
>> Down in ClangASTContext it does:
>>
>>
>> uint32_t
>> ClangASTContext::GetNumChildren (void* type, bool omit_empty_base_classes)
>> {
>> if (!type)
>> return 0;
>>
>> uint32_t num_children = 0;
>> clang::QualType qual_type(GetQualType(type));
>
[lldb-dev] cmake build broken from ExpressionParser changes
The change for the ExpressionParser plugin didn't have changes in it for the cmake or Makefile-based build systems. I'll check in the cmake changes once my test build completes successfully. - Bruce ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
[lldb-dev] September LLVM bay-area social!!!
It's that time again! Sorry for the late reminder, but tomorrow is the first Thursday of the month and we will once again be gathering at the Tied House at 7pm! Come, Joon us to debate new IR constructs, crazy C++ features and any and everything else LLVM! The meetup link is: http://www.meetup.com/LLVM-Bay-Area-Social/events/224477737/ See you all there! ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Re: [lldb-dev] Portable tests that create threads
I think the thread naming is a reasonable way to go. > 1) Linux and MacOSX inferiors can use pthread_setname_np > 2) FreeBSD inferiors can use pthread_set_name_np IIRC Linux and FreeBSD both are limited to a very short thread name, much shorter than OS X and, if I'm not mistaken, Windows as well. So we need to not go crazy on thread naming conventions. The code has it, but I think the limit on Linux and FreeBSD was something like 16 characters. On Wed, Sep 2, 2015 at 2:48 PM, Jim Ingham via lldb-dev < lldb-dev@lists.llvm.org> wrote: > > > On Sep 2, 2015, at 1:06 PM, Greg Clayton via lldb-dev < > lldb-dev@lists.llvm.org> wrote: > > > > One idea is add the ability to discover which of any of the current > threads are or are not user created. This might help us. > > > > On MacOSX, the main thread would always be considered user created, and > any other thread, it is quite easy to tell. User created threads could be > ones whose bottom stack frame is "thread_start" > > > > > >frame #8: 0x7fff9b47c41d libsystem_pthread.dylib thread_start + 13 > > > > Where OS created threads have their last frames in "start_wqthread" or > "_dispatch_mgr_thread": > > > >... > >frame #2: 0x7fff9b47c40d libsystem_pthread.dylib start_wqthread + > 13 > > > > or: > > > >... > >frame #2: 0x7fff9c298a6a libdispatch.dylib _dispatch_mgr_thread + > 52 > > > > > > So it might be easy to add API to SBThread like: > > > > bool > > SBThread::IsUserCreated(); > > > > Then we could teach the tests to filter out any threads that are not > user created? > > I'm not sure on OS X this is a terribly useful distinction. There are > lots of threads that are created to run user code through the dispatch > library. They won't start with "thread_start" but I don't think most users > would think of them as "not user threads..." > > Jim > > > > > > Greg > > > > > >> On Sep 2, 2015, at 12:31 PM, Zachary Turner via lldb-dev < > lldb-dev@lists.llvm.org> wrote: > >> > >> Historically the pattern for tests that need to test situations > involving threads has been to write the tests using pthreads, and then a > common pattern on the python side of the test is to assert that the actual > number of threads equals the expected number of threads. > >> > >> To deal with the pthread portability issue, I ported most of these > tests over to std::threads, but there is still another issue with them. > >> > >> On Windows we can't assume anything about the number of threads in a > program. For example, a single-threaded hello world program actually has 3 > threads, 2 of which are reserved for use by the operating system. And the > fact that there's 2 is just arbitrary, that might not remain the same > between compiler versions, operating system versions, etc. And to make > matters worse it might not even remain constant through the life of the > program (the assumption just being that we cannot make any assumptions, > since we don't control the threads). > >> > >> So I need to find a way to write these tests portably. > >> > >> Do all platforms have a way to assign a thread a name? Looking in > Host/*/ThisThread.cpp, it seems: > >> > >> 1) Linux and MacOSX inferiors can use pthread_setname_np > >> 2) FreeBSD inferiors can use pthread_set_name_np > >> 3) Windows inferiors can raise a magic exception which the debugger is > expected to recognize > >> > >> Based on this, it seems that one possible way to write these tests > portably is to have the test inferiors assign names to the threads, and > have the tests index the threads by name instead of by ordinal. > >> > >> Does anyone have any thoughts on this or other possible approaches? > >> > >> I really would like to avoid having different inferiors or different > tests for all of this functionality, as it will be very easy to get out of > sync. > >> ___ > >> lldb-dev mailing list > >> lldb-dev@lists.llvm.org > >> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev > > > > ___ > > lldb-dev mailing list > > lldb-dev@lists.llvm.org > > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev > > ___ > lldb-dev mailing list > lldb-dev@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev > -- -Todd ___ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
