Hi, On Wed, Jan 15, 2020 at 02:38:26PM +0000, David Aldrich wrote:
> We build large C++ projects using Jenkins for Linux platforms. These build > jobs currently run on a single executor. I want 'make -j' or ninja to > compile concurrently on several cores to speed up the build time. How can > I cause a job to utilise multiple executors on an agent in this way? If you want to go for a 1:1 executor to CPU mapping, then using -j inside the build and combining it with the Heavy Job plugin to allocate more CPUs is indeed the way to go. The downside is that it will wait until that many executors are available, and IIRC there is an anti-starvation mechanism that also blocks other jobs from executing on the chosen node. Depending on your setup, you might as well have the best result with a setup like "four executors, and all jobs use "-j$((($(nproc)+3)/4)" or something similar. For the most flexible result, I suggest defining an environment variable in the agent config that defines the -j option to be used on that node, then you have both the number of executors and the number of threads per executor in the agent config, and if you see that it doesn't work optimally, you can tune there. Two or four CPUs per executor are good if you have many builds, while two or four executors with half/quarter of the CPUs each are good if you build a massive project seldom. The Heavy Jobs plugin is for mixing strategies -- e.g. you'd define a -j4 job for incremental builds that run quickly either way except if there is a big change, and a heavyweight job that takes over half the node for the daily full rebuild/warning generation/... task. Simon -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/20200115145555.GA23937%40psi5.com.
