llunak created this revision.
llunak added reviewers: clayborg, labath.
llunak added a project: LLDB.
Herald added subscribers: usaxena95, JDevlieghere, kadircet.
Herald added a project: All.
llunak requested review of this revision.
Herald added subscribers: lldb-commits, ilya-biryukov.
If LLDB index cache is enabled and everything is cached, then loading of debug
info is essentially single-threaded. This patch parallelizes module loading in
DynamicLoaderPOSIXDYLD::LoadAllCurrentModules(), which may greatly reduce the
load time if the debugged program uses a large number of binaries (as opposed
to monolithic programs where this presumably doesn't make a difference). In my
specific case of LibreOffice Calc this reduces startup time from 6s to 2s.
One problem with this change is that it creates a threadpool-within-threadpool
situation in ManualDWARFIndex::Index(), and so the number of threads running at
the same time may not be properly limited. I think the threadpool in
ManualDWARFIndex::Index() is still useful even with this change because of the
case of a monolithic program that's not cached.
If not limiting thread count properly is considered a problem, I think a
solution to that is using a global semaphore to limit tasks from both places.
There's a simple Semaphore class in clangd (with clangd-specific trace code, so
I guess it'd need to be copy&pasted into lldb) and the semaphore object would
need to be stored somewhere (and I don't know where).
I have not checked if all the relevant code is thread-safe, but since all of it
is already running in another thread I assume that to be the case.
Presumably a similar change could be done for other platforms (I have no plans
to do so).
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D122975
Files:
lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp
Index: lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp
===================================================================
--- lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp
+++ lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp
@@ -24,6 +24,7 @@
#include "lldb/Utility/LLDBLog.h"
#include "lldb/Utility/Log.h"
#include "lldb/Utility/ProcessInfo.h"
+#include "llvm/Support/ThreadPool.h"
#include <memory>
@@ -602,9 +603,24 @@
m_process->PrefetchModuleSpecs(
module_names, m_process->GetTarget().GetArchitecture().GetTriple());
- for (I = m_rendezvous.begin(), E = m_rendezvous.end(); I != E; ++I) {
- ModuleSP module_sp =
+ std::vector<DYLDRendezvous::iterator> infos;
+ for (I = m_rendezvous.begin(), E = m_rendezvous.end(); I != E; ++I)
+ infos.push_back(I);
+ std::vector<ModuleSP> loaded_modules;
+ loaded_modules.resize(infos.size());
+ llvm::ThreadPool pool(llvm::optimal_concurrency(infos.size()));
+ auto load_module_fn = [&](size_t idx) {
+ DYLDRendezvous::iterator I = infos[idx];
+ loaded_modules[idx] =
LoadModuleAtAddress(I->file_spec, I->link_addr, I->base_addr, true);
+ };
+ for (size_t i = 0; i < infos.size(); ++i)
+ pool.async(load_module_fn, i);
+ pool.wait();
+
+ for (size_t i = 0; i < infos.size(); ++i) {
+ DYLDRendezvous::iterator I = infos[i];
+ ModuleSP module_sp = loaded_modules[i];
if (module_sp.get()) {
LLDB_LOG(log, "LoadAllCurrentModules loading module: {0}",
I->file_spec.GetFilename());
Index: lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp
===================================================================
--- lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp
+++ lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp
@@ -24,6 +24,7 @@
#include "lldb/Utility/LLDBLog.h"
#include "lldb/Utility/Log.h"
#include "lldb/Utility/ProcessInfo.h"
+#include "llvm/Support/ThreadPool.h"
#include <memory>
@@ -602,9 +603,24 @@
m_process->PrefetchModuleSpecs(
module_names, m_process->GetTarget().GetArchitecture().GetTriple());
- for (I = m_rendezvous.begin(), E = m_rendezvous.end(); I != E; ++I) {
- ModuleSP module_sp =
+ std::vector<DYLDRendezvous::iterator> infos;
+ for (I = m_rendezvous.begin(), E = m_rendezvous.end(); I != E; ++I)
+ infos.push_back(I);
+ std::vector<ModuleSP> loaded_modules;
+ loaded_modules.resize(infos.size());
+ llvm::ThreadPool pool(llvm::optimal_concurrency(infos.size()));
+ auto load_module_fn = [&](size_t idx) {
+ DYLDRendezvous::iterator I = infos[idx];
+ loaded_modules[idx] =
LoadModuleAtAddress(I->file_spec, I->link_addr, I->base_addr, true);
+ };
+ for (size_t i = 0; i < infos.size(); ++i)
+ pool.async(load_module_fn, i);
+ pool.wait();
+
+ for (size_t i = 0; i < infos.size(); ++i) {
+ DYLDRendezvous::iterator I = infos[i];
+ ModuleSP module_sp = loaded_modules[i];
if (module_sp.get()) {
LLDB_LOG(log, "LoadAllCurrentModules loading module: {0}",
I->file_spec.GetFilename());
_______________________________________________
lldb-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
