Re: Building with many cores without OOM

Hi, Let me plop in right into this discussion with no general solution and more things to think about. For the context I'm packaging Java things, and Java has historically been notoriously bad at guessing how much memory it could actually use on a given system. I'm not sure things are much be

Re: Building with many cores without OOM

Hi! On Thu, 2024-12-05 at 09:23:24 +0100, Helmut Grohne wrote: > On Wed, Dec 04, 2024 at 02:03:29PM +0100, Guillem Jover wrote: > > On Thu, 2024-11-28 at 10:54:37 +0100, Helmut Grohne wrote: > > > For one thing, I propose extending debhelper to provide > > > --min-ram-per-parallel-core as that see

Re: Building with many cores without OOM

Hi Guillem and others, Thanks for your extensive reply and the followup clarifying the inside-out and outside-in distinction. On Wed, Dec 04, 2024 at 02:03:29PM +0100, Guillem Jover wrote: > On Thu, 2024-11-28 at 10:54:37 +0100, Helmut Grohne wrote: > > I think this demonstrates that we probably

Re: Building with many cores without OOM

Hi! On Wed, 2024-12-04 at 14:37:45 +, Stefano Rivera wrote: > Hi Guillem (2024.12.04_13:03:29_+) > > > Are there other layers that could reasonably be used to implement a more > > > general form of parallelism limiting based on system RAM? Ideally, we'd > > > consolidate these implementati

Re: Building with many cores without OOM

Hi Guillem (2024.12.04_13:03:29_+) > > Are there other layers that could reasonably be used to implement a more > > general form of parallelism limiting based on system RAM? Ideally, we'd > > consolidate these implementations into fewer places. > > I think adding this in dpkg-buildpackage itse

Re: Building with many cores without OOM

Hi! On Wed, 2024-12-04 at 14:03:30 +0100, Guillem Jover wrote: > On Thu, 2024-11-28 at 10:54:37 +0100, Helmut Grohne wrote: > > Are there other layers that could reasonably be used to implement a more > > general form of parallelism limiting based on system RAM? Ideally, we'd > > consolidate these

Re: Building with many cores without OOM

Hi! On Thu, 2024-11-28 at 10:54:37 +0100, Helmut Grohne wrote: > I am one of those who builds a lot of different packages with different > requirements and found that picking a good parallel=... value in > DEB_BUILD_OPTIONS is hard. Go too low and your build takes very long. Go > too high and you

Re: Building with many cores without OOM

Helmut Grohne: Hi Guillem and other developers, I am one of those who builds a lot of different packages with different requirements and found that picking a good parallel=... value in DEB_BUILD_OPTIONS is hard. Go too low and your build takes very long. Go too high and you swap until the OOM ki