Pierre Neidhardt writes:
> Update: I've been using --max-jobs=2 by default for about 2 weeks now,
> and it feels like a much smoother experience overall: faster downloads, faster
> builds.
>
> This is obviously a very dumb "optimization" but at least it serves to
> underline that Guix could still
Hi Mark,
Mark H Weaver skribis:
> For these reasons, I'm inclined to think that parallel downloads is the
> wrong approach. If a single download process is not making efficient
> use of the available bandwidth, I'd be more inclined to look carefully
> at why it's fai
Hi,
Leo Famulari skribis:
> On Sat, Nov 09, 2019 at 06:40:56PM +0100, Ludovic Courtès wrote:
>> Like I wrote, it’s not that simple (we’d first need the daemon to
>> distinguish substitution jobs from other jobs, but note that there are
>> also “downloads” that are actually derivation builds), an
On Wed, Nov 13, 2019 at 11:16:53AM -0500, Mark H Weaver wrote:
> For these reasons, I'm inclined to think that parallel downloads is the
> wrong approach. If a single download process is not making efficient
> use of the available bandwidth, I'd be more inclined to look care
load them sequentially.
I'll also note that parallel downloads would increase the memory usage
on the server. If users, on average, configured 4 parallel downloads, I
guess that would have the effect of multiplying the server memory usage
by about 4. It might also create an incentive for u
Hi Efraim,
On Wed, 13 Nov 2019 at 08:44, Efraim Flashner wrote:
> > guix build `guix show PKG | recsel -R dependencies`
> > guix build PKG --no-substitutes
> 'guix environment PKG -- guix build PKG --no-subsitutes'
Thank you.
It is a better solution, indeed! :-)
All the best,
simon
On Tue, Nov 12, 2019 at 05:48:16PM +0100, zimoun wrote:
> Hi,
> It is not related with parallel download but on old machines "guix
> build --no-substitutes" can eat a lot of resources; for example if the
> package has a lot of dependencies. I would like to be able to list
> which dependencies I wan
On Sat, Nov 09, 2019 at 06:40:56PM +0100, Ludovic Courtès wrote:
> Like I wrote, it’s not that simple (we’d first need the daemon to
> distinguish substitution jobs from other jobs, but note that there are
> also “downloads” that are actually derivation builds), and it’s not
> clear to me that it’s
Hi,
It is not related with parallel download but on old machines "guix
build --no-substitutes" can eat a lot of resources; for example if the
package has a lot of dependencies. I would like to be able to list
which dependencies I want to build and which I want to substitute.
Maybe it is already pos
Hi everyone,
I’ve been watching this from afar and one thing and while I have to agree with
this:
> .. I suspect there’s little to
> be gained by having several connections in parallel.
I do have to say that more fine grained concurrency would really help speed up
builds without substitutes.
Hello,
Pierre Neidhardt skribis:
> Ludovic Courtès writes:
>
>> it’s not supposed to be
>> faster to download 10 things in parallel from ci.guix.gnu.org, than to
>> download them sequentially.
>
> I was thinking a little bit ahead in the future, e.g. when we have more
> build farms (if it ever
Hi Pierre!
Pierre Neidhardt skribis:
> Ludovic Courtès writes:
>
>>> - Can we configure the default value?
>>
>> Yup, just pass ‘--max-jobs=N’ to the daemon.
>
> So I suppose you mean to use the `extra-options` field, e.g.
>
> (guix-service-type config =>
>(guix-configuratio
On +2019-11-06 16:34:17 +0100, Ludovic Courtès wrote:
> Hi,
>
> Pierre Neidhardt skribis:
>
> > A few questions:
> >
> > - How does --mac-jobs work with regard to progress bars?
>
> When there are several jobs running at the same time, it doesn’t display
> any progress bar. (In theory, it coul
Hi,
Pierre Neidhardt skribis:
> A few questions:
>
> - How does --mac-jobs work with regard to progress bars?
When there are several jobs running at the same time, it doesn’t display
any progress bar. (In theory, it could do a multi-line display but that
wouldn’t work with all terminals, it’s
Hi,
Joshua Branson skribis:
> Pierre Neidhardt writes:
>
>> Hi,
>>
>> Is there any plan to support parallel downloads? Currently downloads are a
>> bottleneck for `guix install / upgrade`, parallel downloads could reduce
>> the operation duration by an
Hi Pierre,
Pierre Neidhardt skribis:
> I'm not sure I understand: if I run
>
> guix build -M 4 vlc
>
> download progress bars are updated one at a time. Or am I mistaken?
With -M4 you could potentially have 4 substitutions (downloads) going on
in parallel.
Tobias is right: the daemon works in
Pierre Neidhardt writes:
> Hi,
>
> Is there any plan to support parallel downloads? Currently downloads are a
> bottleneck for `guix install / upgrade`, parallel downloads could reduce
> the operation duration by an order of magnitude.
On a related note, would getting g
Hi Pierre,
Maybe you would like an concurrent output in this spirit:
https://joeyh.name/code/concurrent-output/
Right?
However, I am not sure that parallel downloads drastically change the
performance (bottleneck) because first often it is not linear and
second it strongly depends on the
Hi Tobias,
I'm not sure I understand: if I run
--8<---cut here---start->8---
guix build -M 4 vlc
--8<---cut here---end--->8---
download progress bars are updated one at a time. Or am I mistaken?
Also is there a way to chan
Hullo Pierre!
Pierre Neidhardt 写道：
Is there any plan to support parallel downloads?
Guix already downloads sources and substitutes in parallel with
other builds/downloads through the --max-jobs option.
You could add a separate knob for downloads that defaults to
--max-jobs. Or even
Hi,
Is there any plan to support parallel downloads? Currently downloads are a
bottleneck for `guix install / upgrade`, parallel downloads could reduce
the operation duration by an order of magnitude.
Thoughts?
--
Pierre Neidhardt
https://ambrevar.xyz/
signature.asc
Description: PGP
21 matches
Mail list logo
