[PATCH] D93452: [clangd] Trim memory after buildINdex

2020-12-18 Thread Quentin Chateau via Phabricator via cfe-commits
qchateau updated this revision to Diff 312781. qchateau added a comment. Herald added subscribers: llvm-commits, dexonsmith, hiraditya. Herald added a project: LLVM. - Add MallocTrim to llvm::sys::Process - Implement malloc trimming in ClangdLSPServer Repository: rG LLVM Github Monorepo CHANG

[PATCH] D93452: [clangd] Trim memory after buildINdex

2020-12-18 Thread Nathan James via Phabricator via cfe-commits
njames93 added a comment. In D93452#2460563 , @sammccall wrote: > I think we should pass a nonzero `pad` to malloc_trim of several MB. It only > affects the main thread, but the main thread is constantly allocating small > short-lived objects (e.g. JSON

[PATCH] D93452: [clangd] Trim memory after buildINdex

2020-12-17 Thread Quentin Chateau via Phabricator via cfe-commits
qchateau planned changes to this revision. qchateau added a comment. > Periodic malloc_trim seems like it solves a real problem, one I don't > personally fully understand but we may never do - I feel convinced we should > use it anyway. It probably has some overhead, who knows how much. It does

[PATCH] D93452: [clangd] Trim memory after buildINdex

2020-12-17 Thread Sam McCall via Phabricator via cfe-commits
sammccall added a comment. Wow, all abstractions truly are lies. Thanks for the investigation! After doing some reading[1], my understanding is roughly: - our pattern of allocations causes glibc malloc to retain a *much* larger heap than our outstanding allocations, growing without (or with lar

[PATCH] D93452: [clangd] Trim memory after buildINdex

2020-12-17 Thread Quentin Chateau via Phabricator via cfe-commits
qchateau updated this revision to Diff 312500. qchateau added a comment. Herald added a subscriber: mgorny. Add macro to use malloc_trim only if available Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D93452/new/ https://reviews.llvm.org/D93452 Fil

[PATCH] D93452: [clangd] Trim memory after buildINdex

2020-12-17 Thread Quentin Chateau via Phabricator via cfe-commits
qchateau added a comment. > Side note is this memory failing to free behaviour observed on windows? No idea, I only develop on linux. Looking at the the github issues, it seems people that had the issue were also using the linux version...so we can't conclude anything about windows =

[PATCH] D93452: [clangd] Trim memory after buildINdex

2020-12-17 Thread Nathan James via Phabricator via cfe-commits
njames93 added a comment. `malloc_trim` isn't available on windows so this wont work, so this should likely be ifdef'd out on windows builds. Side note is this memory failing to free behaviour observed on windows? Comment at: clang-tools-extra/clangd/index/FileIndex.cpp:51 +v

[PATCH] D93452: [clangd] Trim memory after buildINdex

2020-12-17 Thread Quentin Chateau via Phabricator via cfe-commits
qchateau created this revision. qchateau added a reviewer: sammccall. Herald added subscribers: usaxena95, kadircet, arphaman. qchateau requested review of this revision. Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov. Herald added a project: clang. This diff addresses the issue of