On Monday, 18 September 2023 13:59:03 BST Jack wrote: > On 9/18/23 08:00, Peter Humphrey wrote: > > Hello list, > > > > We've had a few discussions here on how to balance the parameters to > > emerge > > to make the most of the resources available. Here's another idea: > > > > One the one hand, big jobs should be able to use the maximum CPU > > performance and RAM capacity, but on the other we don't want to flood the > > system. > > > > Therefore, I think it would be useful to be able to specify in env and > > package.env that a job should be run on its own - if any other emerge jobs > > are scheduled, wait until they're finished. Combine that with a specific > > MAKEOPTS, and we'd have a more flexible deployment of resouces. > > > > Is this feasible? What have I not thought of? > > I've had exactly the same thought for some time now. My guess is that > it is theoretically possible to add some USE flag or ENV var for portage > to recognize, but I don't know the portage internals well enough to > guess how much effort it would be. Given that portage orders ebuilds in > a single emerge session based on some dependency graph, that seems like > a good place to put the necessary hooks. > > As a starting point, one option might be to create a special/magic > ebuild and make it a dependency of those jobs that need to be run alone, > and have something about it that won't run if anything else is still > running. But, I don't know if those pre-checks (such as checking for > enough RAM and/or disk space) can be run at build time and not just at > portage startup time. The other possible problem with that approach > would be to be sure that ebuild gets run separately for each other > ebuild that depends on it - not all of them depending on it being run > once.Also, those blocking ebuilds have work so that if several of them > are queued (and running their "wait for everything else to finish" > scripts - exactly one of them needs to start. I don't know if those > pre-check scripts count as running before or within the ebuild itself.
It may be less complex than you think, Jack. I envisage a package being marked as solitary, and when portage reaches that package, it waits until all current jobs have finished, then it starts the solitary package with the environment specified for it, and it doesn't start the next one until that one has finished. The dependency calculation shouldn't need to be changed. It seems simple the way I see it. -- Regards, Peter.
