Hello! Ricardo Wurmus <rek...@elephly.net> skribis:
> Ludovic Courtès <ludovic.cour...@inria.fr> writes: > >> Ricardo Wurmus <rek...@elephly.net> skribis: [...] >>> Currently, there’s no way to tell if the derivations listed under “These >>> derivations will be built” are expensive package builds or just simple >>> graft derivations. >> >> Indeed. A simple trick would be to (ab)use the environment variable >> part of derivations as a property list, the way Nix has traditionally >> done it (see ‘user+system-env-vars’ in (guix derivations)). >> >> So we could have, say, a ‘hint’ environment variable, and the UI would >> use that to determine if it’s a graft. > > This sounds like a good trick to me. I think it would be great to give > more hints to the UI and make it clearer to users what work they can > expect Guix to perform. In a similar vein, the attached module provides code to estimate package build time based on locally-available build logs. It can be used to show hints like this: --8<---------------cut here---------------start------------->8--- The following derivations would be built (estimated time: 54 mn): /gnu/store/3627svyhih9cfss8gnxllp9nmxqp23cq-gcc-4.9.4.drv /gnu/store/3didvp9c3sfqwmb9kkdr211vg5myygsf-gcc-4.9.4.tar.xz.drv --8<---------------cut here---------------end--------------->8--- or: --8<---------------cut here---------------start------------->8--- The following derivations would be built (estimated time: 22 hr): /gnu/store/safjgjqhxlf59rknygqdfq175cl5wvks-rust-1.27.2.drv /gnu/store/v154ah7f8wqcga104df9ldb25bjk2pm8-rustc-1.27.2-src.tar.gz.drv /gnu/store/nz2xzl1yizcwcxhnw2w5mdqnp3q1gggx-rustc-1.25.0-src.tar.gz.drv /gnu/store/wn422brymn6ysxq07090ijb4wx78dc1l-rustc-1.24.1-src.tar.gz.drv /gnu/store/3cj5083j5aq7az7n5dmkds77g84xqc33-rust-bootstrap-1.22.1.drv /gnu/store/rm1nghqrvw43wlbwws49rw921qh58m35-rustc-1.23.0-src.tar.xz.drv /gnu/store/a9gzrq0bsc3ynfj4cnrsxxd2jwjgf4zj-rust-1.23.0.drv /gnu/store/bmbpbhgby84yrj0mvkv279cy2ira4xqf-rustc-1.24.1-src.tar.xz.drv /gnu/store/bxxyzp6kzq88ii4aindgnggbb2m193rk-rust-1.24.1.drv /gnu/store/r26s0y5fi055pclhnivvf63llbrj54yw-rustc-1.25.0-src.tar.xz.drv /gnu/store/x4l0rsqni9lglfzazkjyxqjp432yr33s-rustc-1.26.2-src.tar.gz.drv /gnu/store/03y9zf5imbm0ni1llcmxixb8c78nmxdd-rustc-1.26.2-src.tar.xz.drv /gnu/store/ici0m0bia0f6f4wh0ykn12x6wg1ckck0-rust-1.25.0.drv /gnu/store/2l6fn1nxs2sfl93khki5jzz6dh7gfqpr-rust-1.26.2.drv /gnu/store/9bn1gxnsc59zi8bdpvfgqcjpczmk3ch0-rustc-1.27.2-src.tar.xz.drv --8<---------------cut here---------------end--------------->8--- (That’s from my x86_64 laptop.) The obvious downside is that it works by first retrieving the names of the files under /var/log/guix/drvs, and then opening, decompressing, and parsing the candidate log files. That typically takes a few seconds on a recent SSD laptop, but clearly we don’t want to do that every time. We could maintain a cache, but even then, it might still be too expensive. Perhaps we should keep build times in the database somehow; the daemon can keep it up-to-date. Thoughts? There’s the obvious downside that both approaches rely on having previously built the package, but I think that’s a necessary limitation, unless we are to resort to external services (which could hardly provide estimates that make sense for the user’s machine anyway.) Ludo’.
;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2018 Ludovic Courtès <l...@gnu.org> ;;; ;;; This file is part of GNU Guix. ;;; ;;; GNU Guix is free software; you can redistribute it and/or modify it ;;; under the terms of the GNU General Public License as published by ;;; the Free Software Foundation; either version 3 of the License, or (at ;;; your option) any later version. ;;; ;;; GNU Guix is distributed in the hope that it will be useful, but ;;; WITHOUT ANY WARRANTY; without even the implied warranty of ;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;;; GNU General Public License for more details. ;;; ;;; You should have received a copy of the GNU General Public License ;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. (define-module (guix build-logs) #:use-module (guix config) #:use-module (guix store) #:use-module (srfi srfi-1) #:use-module (guix utils) #:use-module (ice-9 match) #:use-module (ice-9 ftw) #:use-module (ice-9 regex) #:use-module (ice-9 rdelim) #:export (%log-directory log-file-build-phases log-file-build-time estimated-build-time)) (define %log-directory (string-append (dirname %state-directory) ; XXX "/log/guix/drvs")) (define %end-of-phase-rx (make-regexp "^phase [`']([[:graph:]]+)' succeeded after ([0-9.]+) seconds$")) (define (log-file-build-phases file) "Interpret the build log in FILE and return an alist of name/duration pairs for each build phase, such as: ((unpack . 1.3) (configure . 4.2) (build . 383.8) …) Duration is expressed in seconds. Return the empty list if no build phase information could be extracted from FILE." (define compression (cond ((string-suffix? ".gz" file) 'gzip) ((string-suffix? ".bz2" file) 'bzip2) ((string-suffix? ".xz" file) 'xz) (else 'none))) (call-with-input-file file (lambda (input) (call-with-decompressed-port compression input (lambda (port) (set-port-conversion-strategy! port 'substitute) (let loop ((result '())) (match (read-line port) ((? eof-object?) (reverse result)) (line (match (regexp-exec %end-of-phase-rx line) (#f (loop result)) (hit (loop (alist-cons (string->symbol (match:substring hit 1)) (string->number (match:substring hit 2)) result)))))))))))) (define (log-file-build-time file) "Return the total build time described by FILE, a build log, or zero if build phase information was not found." (match (log-file-build-phases file) (((names . durations) ...) (if (memq 'install names) (reduce + 0 durations) 0)))) (define (matching-log-files package) (define noop (lambda (file stat result) result)) (file-system-fold (const #t) (lambda (file stat result) ;leaf (let* ((base (basename (file-sans-extension (file-sans-extension file)))) (dash (string-index base #\-)) (full (string-drop base (+ dash 1)))) (call-with-values (lambda () (package-name->name+version full #\-)) (lambda (p v) (if (and (string=? p package) (not (string-suffix? ".gz" v)) (not (string-suffix? ".bz2" v)) (not (string-suffix? ".xz" v)) (not (string-suffix? ".lz" v)) (not (string-suffix? ".zip" v))) (cons (list file p v) result) result))))) noop ;down noop ;up noop ;skip (lambda (file stat error result) ;error result) '() %log-directory)) (define %not-dot (char-set-complement (char-set #\.))) (define (version-distance version reference) "Compute a super rough estimate of the distance of VERSION to REFERENCE, both of which being version strings." (let* ((reference (string-tokenize reference %not-dot)) (version (string-tokenize version %not-dot)) (len (length reference))) (let loop ((i len) (reference reference) (version version) (distance 0)) (match version (() distance) ((head . tail) (match reference (() distance) ((ref-head . ref-tail) (loop (- i 1) ref-tail tail (if (string=? ref-head head) distance (+ distance i)))))))))) (define (estimated-build-time package version) "Return the estimate time it takes to build PACKAGE at VERSION, or #f if no such estimate is available." (let ((logs (sort (matching-log-files package) (match-lambda* (((file1 _ version1) (file2 _ version2)) (< (version-distance version1 version) (version-distance version2 version))))))) (any (match-lambda ((log package version) (let ((duration (log-file-build-time log))) (and (not (zero? duration)) duration)))) logs)))
