[
https://issues.apache.org/jira/browse/IMPALA-14527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-14527 started by Michael Smith.
----------------------------------------------
> Switch to mold as the default linker
> ------------------------------------
>
> Key: IMPALA-14527
> URL: https://issues.apache.org/jira/browse/IMPALA-14527
> Project: IMPALA
> Issue Type: Task
> Components: Infrastructure
> Reporter: Michael Smith
> Assignee: Michael Smith
> Priority: Major
>
> The mold linker better utilizes concurrency to speed up linking a single
> binary. This has two benefits
> 1. Development when relinking e.g. impalad will be significantly faster (
> 2. During a full build that links many test binaries, we can use
> MOLD_LINKER=1 to link a single binary at a time - using significantly less
> memory - while still fully utilizing all CPU cores.
> Full builds are also a bit faster. After priming ccache with a full build;
> with mold rebuilds take ~2m33s and memory use remains under 10GiB (24 cores)
> {code}
> $ env MOLD_JOBS=1 IMPALA_LINKER=mold time ./buildall.sh -skiptests
> ...
> 313.23user 67.68system 2:33.65elapsed 247%CPU (0avgtext+0avgdata
> 5664100maxresident)k
> 16inputs+34793584outputs (216major+12078088minor)pagefaults 0swaps
> {code}
> With gold, it takes ~3m15s and maxes out my memory (64GiB) requiring
> additional swap
> {code}
> $ env IMPALA_LINKER=gold time ./buildall.sh -skiptests
> ...
> 261.36user 71.43system 3:15.41elapsed 170%CPU (0avgtext+0avgdata
> 3655132maxresident)k
> 7485904inputs+34794136outputs (88605major+11836509minor)pagefaults 0swaps
> {code}
> Performance of the resulting binaries looks essentially the same:
> https://jenkins.impala.io/job/perf-AB-test-ub2004/336/artifact/Impala/perf_results/latest/performance_result.txt.
> Let's switch to using IMPALA_LINKER=mold by default.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
