[PATCH 1/1] import: Add gopkg importer.

[Rouby Pierre-Antoine]

* guix/import/gopkg.scm: New file.
* guix/scripts/import/gopkg.scm: New file.
* guix/scripts/import.scm: Add 'gopkg'.
* Makefile.am: Add 'gopkg' importer in modules list.
---
 Makefile.am   |   1 +
 guix/import/gopkg.scm | 294 ++
 guix/scripts/import.scm   |   2 +-
 guix/scripts/import/gopkg.scm |  99 
 4 files changed, 395 insertions(+), 1 deletion(-)
 create mode 100644 guix/import/gopkg.scm
 create mode 100644 guix/scripts/import/gopkg.scm

diff --git a/Makefile.am b/Makefile.am
index 9f134c970..e103517fc 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -183,6 +183,7 @@ MODULES =   \
   guix/import/hackage.scm  \
   guix/import/elpa.scm \
   guix/import/texlive.scm  \
+  guix/import/gopkg.scm\
   guix/scripts.scm \
   guix/scripts/download.scm\
   guix/scripts/perform-download.scm\
diff --git a/guix/import/gopkg.scm b/guix/import/gopkg.scm
new file mode 100644
index 0..451e94a8e
--- /dev/null
+++ b/guix/import/gopkg.scm
@@ -0,0 +1,294 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Pierre-Antoine Rouby 
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see .
+
+(define-module (guix import gopkg)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 regex)
+  #:use-module ((ice-9 rdelim) #:select (read-line))
+  #:use-module (git)
+  #:use-module (guix hash)
+  #:use-module (guix base32)
+  #:use-module (guix serialization)
+  #:use-module (guix utils)
+  #:use-module (guix build utils)
+  #:use-module (srfi srfi-11)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:export (gopkg->guix-package))
+
+(define (vcs-file? file stat)
+  (case (stat:type stat)
+((directory)
+ (member (basename file) '(".bzr" ".git" ".hg" ".svn" "CVS")))
+((regular)
+ ;; Git sub-modules have a '.git' file that is a regular text file.
+ (string=? (basename file) ".git"))
+(else
+ #f)))
+
+(define (file-hash->base32 file)
+  (let-values (((port get-hash) (open-sha256-port)))
+(write-file file port #:select? (negate vcs-file?))
+(force-output port)
+(bytevector->nix-base32-string (get-hash
+
+(define (append-inputs inputs name)
+  (append inputs
+  (list
+   (list name
+ (list 'unquote
+   (string->symbol name))
+
+(define (package-name url)
+  (string-downcase
+   (string-append "go-"
+  (string-replace-substring
+   (string-replace-substring url
+ "/" "-")
+   "." "-"
+
+(define (cut-url url)
+  (string-replace-substring
+   (cond
+((string-match "http://";  url)
+ (string-replace-substring url "http://"; ""))
+((string-match "https://"; url)
+ (string-replace-substring url "https://"; ""))
+((string-match "git://"   url)
+ (string-replace-substring url "git://" ""))
+(else
+ (values url)))
+   ".git" ""))
+
+(define (url-to-path url)
+  (string-replace-substring
+   (string-append "/tmp/"
+  (cut-url url))
+   "." "-"))
+
+;; HACK system exec
+(define (git-checkout directory commit)
+  (let ((command (string-append "cd " directory " &&"
+"git checkout " commit
+" > /dev/null 2> /dev/null"))) ; HACK no 
command output
+(if (not (or (equal? commit "0")
+ (equal? commit "XXX")
+ (equal? commit "master")))
+(system command
+
+(define (git-clone url commit)
+  (define (clone-in-dir url directory)
+(mkdir-p directory)
+(clone url directory (clone-init-options))
+(git-checkout directory commit)
+(values directory))
+
+  (let ((directory (url-to-path url)))
+(if (not (file-exists? (string-append directory)))
+(clone-in-dir url directory)
+(values directory
+
+(define (comment? line)
+  (eq? (string-ref (string-trim line) 0) #\#))
+
+(define (attribute? line str)
+  (equal? (string-trim-right
+   (string-trim
+(car (string-split line #\= str))
+
+(define (attribute-by-name line name)
+  (string-trim

a guile based build tool at the base of a guile-build-system

[Catonano]

2018-04-23 10:20 GMT+02:00 Ludovic Courtès :

> Hello Ricardo & all!
>
> Ricardo Wurmus  skribis:
>
> > I’m happy to announce that the group I’m working with has released a
> > preprint of a paper on reproducibility with the title:
> >
> > Reproducible genomics analysis pipelines with GNU Guix
> > https://www.biorxiv.org/content/early/2018/04/11/298653
> >
> > We built a collection of bioinformatics pipelines and packaged them with
> > GNU Guix, and then looked at the degree to which the software achieves
> > bit-reproducibility (spoiler: ~98%), analysed sources of non-determinism
> > (e.g. time stamps), discussed experimental reproducibility at runtime
> > (e.g. random number generators, kernel+glibc interface, etc) and
> > commented on the idea of using “containers” (or application bundles)
> > instead.
>
> Very impressive piece of work!  I think it’s important to stress that
> reproducible builds is a crucial foundation for reproducible
> computational experiments, and this paper does a great job at this.
>
> Also nice that you show you can have these bit-reproducible pipelines
> formalized in Guix *and* produce a ready-to-use “container image.”
>
> Hopefully we can soon address the remaining sources of non-determinism
> shown in Table 3 (I think you already addressed some of them in the
> meantime, didn’t you?).
>
> The bit I’m less comfortable with is Autotools.  I do understand how it
> helps capture configure-time dependencies, and how it generally helps
> people package and use the software; I think it’s one of the best tools
> for the job.  However it’s also hard to learn and, whether it’s
> justified or not, it’s considered “scary.”
>
> Given the intended audience, I wonder how we could provide a simpler
> path to achieve the same goal.  It could be a set of Autoconf macros
> leading to high-level ‘configure.ac’ files without any line of shell
> code, or it could be Guix interpreting a top-level .scm or JSON file,
> both of which would ideally be easier to write for bioinformaticians.
>
> What are your thoughts on this?
>

I have explored the possibility to create a guile based tool for building
guile packages

That could be easily expanded to support non guile based projects

It could reproduce all thhe tests that Autoconf generated scripts perform

I thought it would be nice to reproduce the record format used in Guildhall

It seems to me that the pacages defined in pkg-list.scm files are not using
the standard srfi-9 syntax

I also noticed that the guix/records.scm file

So I embarked in the process of learning how the guile macros are used there

It was a tumble, substantially

I couldn't read the code, too much and too articulated

The guile manual was not very helpfful

So I took this article
https://www.cs.indiana.edu/~dyb/pubs/tr356.pdf

to a print shop to have it printed and be able to read it peacefully

I read it. so now I know at least what the whole thing is about

I made this little (< 7 Mb) footage of me expanding a guix defined macro
for creating a record
http://catonano.v22018025836661967.nicesrv.de/resources/videos/expanding.flv

Too bad, it expands the thing completely, so I end up with some code
creating and manipulating structures, rather than srffi-9 records

I remebered aout a macroexpand-1 macro doing only one pass of expansion but
that's in Clojure !
I remembered incorrectly !

So this is where I'm at and where I've been

A stumble in yakk shaving, substantially.

I could give up on macros and use srfi-9 records for now.

The next step would be to test if some guile deps are available (calling
the line I already have briefly discussed here)

should anyone be working on a guile based buld tool, I'd love them to make
their efforts known to the community, maybe publish a repo somewhere


>
> Anyway, kudos on this, thank you!
>
> Ludo’.
>
>

Re: Org-mode for Haunt

[Catonano]

2018-04-26 19:59 GMT+02:00 swedebugia :

> Hi
> The source code is not available via the git repository on that page and
> neither anywhere else according to a quick search.
> Is 0.3 the latest release?
>

You could try to contact Wingo. The last line of the page about
guile-present invites to contact him directly in order to send feedback

Should the code be lost, Guile offers support for creating parsers. There
are a couple of chapters in the manual about that

I suppose that the Markdown parser is built along those lines (I just
suppose: I haven't taken a look at the code)

So you could duplicate that and then start tweaking it in order to make it
gradually become a org-mode parser

Re: Use .authinfo for git credentials

[Pierre Neidhardt]

Yes, that sounds like the right approach.  I'll look into packaging it then!

-- 
Pierre Neidhardt

The other line moves faster.


signature.asc
Description: PGP signature

[no subject]

[Catonano]

it's 17:05 of april 27th

I just pulled guix and tried to build it

with this result

impossibile aprire ./doc/guix.fr.texi: No such file or directory

ERROR: In procedure lstat:
In procedure lstat: File o directory non esistente:
"/home/catonano/GNU/guix/doc/contributing.fr.texi"

Re: create a symlink

[Danny Milosavljevic]

Hi Rene,

are you sure you put the call into guix/scripts/system.scm ?

There are a lot of #$ things there already - should work.

In gnu/build/install.scm it's not going to work.

The Guix high-level packaging stuff is not available build-side, only host-side.

Can you provide a patch of what you did?



pgpbAp91LbsXN.pgp
Description: OpenPGP digital signature

Re:

[Pierre Neidhardt]

I've run into the same issue.  After a `make clean` it did build properly 
though.

-- 
Pierre Neidhardt

Art is anything you can get away with.
-- Marshall McLuhan.


signature.asc
Description: PGP signature

Re: Generating wrappers for execution in non-root non-Guix contexts

[Ludovic Courtès]

Hello!

Chris Marusich  skribis:

> ludovic.cour...@inria.fr (Ludovic Courtès) writes:
>
>> Hello Guix!
>>
>> The hack below allows ‘guix pack’ to produce wrappers that allow,
>> through user namespaces, programs to automatically relocate themselves
>> when you run them unprivileged on a machine that lacks Guix.
>
> That's really cool!
>
> I've noticed that when running in a chroot-like environment, sometimes
> programs expect certain files to exist that don't - for example, device
> files in /dev, procfs files in /proc, or even things like
> /etc/resolv.conf.  Does this wrapper automatically create those kinds of
> files, or would programs that want to access those kinds of files still
> need some special love on an case-by-case basis?

The wrapper automatically bind-mounts every entry in /, such that the
only difference compared to the “real” system is the extra /gnu/store.


Note: we had this discussion about ‘guix run’ with Mike Gerwitz and
Rutger not long ago (to run applications in isolated environments).  In
a pretty similar way, we could generate least-authority wrappers for
what you install with ‘guix package’.  Like, one could write:

  guix package -i icecat --least-authority

or something like that.  Food for thought…

Ludo’.

GSoC-2018

[Sandeep Subramanian]

Hi all,

I am Sandeep Subramanian (uniq10) and I have been selected for the
GSoC-2018 project "Continue rewrite build daemon in Guile Scheme".
(Project description:
https://libreplanet.org/wiki/Group:Guix/GSoC-2018#Continue_rewrite_build_daemon_in_Guile_Scheme
)

I am very excited to work on this project and am looking forward to
interacting
with the community here and on the #guix IRC channel (Nick: uniq10).
I will mostly be active from UTC 08:00 to UTC 20:00.

I hope to, with your help, successfully complete this project and make a
meaningful contribution to the community.

Re:

[Catonano]

2018-04-27 18:30 GMT+02:00 Pierre Neidhardt :

>
> I've run into the same issue.  After a `make clean` it did build properly
> though.
>

yes, thanks

Re: GSoC-2018

[Catonano]

2018-04-27 19:43 GMT+02:00 Sandeep Subramanian <
sandeepsubramania...@gmail.com>:

> Hi all,
>
> I am Sandeep Subramanian (uniq10) and I have been selected for the
> GSoC-2018 project "Continue rewrite build daemon in Guile Scheme".
> (Project description:
> https://libreplanet.org/wiki/Group:Guix/GSoC-2018#Continue_
> rewrite_build_daemon_in_Guile_Scheme
> )
>
> I am very excited to work on this project and am looking forward to
> interacting
> with the community here and on the #guix IRC channel (Nick: uniq10).
> I will mostly be active from UTC 08:00 to UTC 20:00.
>
> I hope to, with your help, successfully complete this project and make a
> meaningful contribution to the community.
>
>
welcome and good luck !!

Should python-build-system packages have native-inputs?

[Chris Marusich]

Hi Guix,

I've noticed that a fair number of packages in gnu/packages/python.scm
using the python-build-system declare native-inputs.  I suspect that in
every case, these should actually just be inputs.  I also suspect that
this is benign, except perhaps for the fact that it may confuse
Pythonistas who (like myself) initially started out by looking at these
packages as examples of how to get started defining packages in Guix.

The python-build-system's "lower" procedure (in
guix/build-system/python.scm) explicitly forbids cross-compilation:

--8<---cut here---start->8---
(define* (lower name
#:key source inputs native-inputs outputs system target
(python (default-python))
#:allow-other-keys
#:rest arguments)
  "Return a bag for NAME."
  (define private-keywords
'(#:source #:target #:python #:inputs #:native-inputs))

  (and (not target)   ;XXX: no cross-compilation
   (bag
 (name name)
 (system system)
 (host-inputs `(,@(if source
  `(("source" ,source))
  '())
,@inputs

;; Keep the standard inputs of 'gnu-build-system'.
,@(standard-packages)))
 (build-inputs `(("python" ,python)
 ,@native-inputs))
 (outputs outputs)
 (build python-build)
 (arguments (strip-keyword-arguments private-keywords arguments)
--8<---cut here---end--->8---

As for the native-inputs, they get stored in the bag's build-inputs,
which eventually find their way to the "inputs" keyword argument used on
the build side by the various build phases.  In fact, the inputs,
propagated-inputs, and native-inputs of any package that uses the
python-build-system are all put into this "inputs" keyword argument.

With this in mind, I have two questions:

* Should we change these native-inputs to inputs to prevent confusion?
  I can personally vouch for the fact that the presence of native-inputs
  in python-build-system packages confused the heck out of me at first!

* Are there any circumstances under which it actually WOULD make sense
  to cross-compile a Python package?

For now, I think the answers to these questions are "sure" and "probably
not", respectively.  I'm very curious to hear your thoughts about the
second question, in particular!

-- 
Chris


signature.asc
Description: PGP signature