On 9/18/23 08:00, Peter Humphrey wrote:
Hello list,
We've had a few discussions here on how to balance the parameters to emerge
to make the most of the resources available. Here's another idea:
One the one hand, big jobs should be able to use the maximum CPU
performance and RAM capacity, but on the other we don't want to flood the
system.
Therefore, I think it would be useful to be able to specify in env and
package.env that a job should be run on its own - if any other emerge jobs are
scheduled, wait until they're finished. Combine that with a specific MAKEOPTS,
and we'd have a more flexible deployment of resouces.
Is this feasible? What have I not thought of?
I've had exactly the same thought for some time now. My guess is that
it is theoretically possible to add some USE flag or ENV var for portage
to recognize, but I don't know the portage internals well enough to
guess how much effort it would be. Given that portage orders ebuilds in
a single emerge session based on some dependency graph, that seems like
a good place to put the necessary hooks.
As a starting point, one option might be to create a special/magic
ebuild and make it a dependency of those jobs that need to be run alone,
and have something about it that won't run if anything else is still
running. But, I don't know if those pre-checks (such as checking for
enough RAM and/or disk space) can be run at build time and not just at
portage startup time. The other possible problem with that approach
would be to be sure that ebuild gets run separately for each other
ebuild that depends on it - not all of them depending on it being run
once.Also, those blocking ebuilds have work so that if several of them
are queued (and running their "wait for everything else to finish"
scripts - exactly one of them needs to start. I don't know if those
pre-check scripts count as running before or within the ebuild itself.
