Ben Woodcroft <b.woodcr...@uq.edu.au> writes:
>> Subject: [PATCH 1/3] gnu: python-pysam: Update to 0.9.1.4.
> I'm not sure whether this is a product of the upgrade or not, but I
> notice this in the build log. I think it is harmless though, WDYT?
>
> starting phase `validate-runpath'
> validating RUNPATH of 10 binaries in
> "/gnu/store/bpiq3lm6b1kpf54i1vj2dl09ff293wic-python-pysam-0.9.1.4/lib"...
> /gnu/store/bpiq3lm6b1kpf54i1vj2dl09ff293wic-python-pysam-0.9.1.4/lib/python3.4/site-packages/pysam-0.9.1.4-py3.4-linux-x86_64.egg/pysam/libchtslib.cpython-34m.so:
>
> warning: RUNPATH contains bogus entries: ("pysam" "."
> "build/lib.linux-x86_64-3.4/pysam")
I don't see this in the previous version, so it is a regression.
However, it should be mostly harmless. Readelf reports (when compiled
with external htslib, see below):
0x000000000000001d (RUNPATH) Library runpath:
[/gnu/store/m4gc2wx4q9if1vrhgclpspdil7rqsn21-python-3.4.3/lib:/gnu/store/ba22myqvxccwmmjwwq665rc43hanycxy-htslib-1.3.1/lib:build/lib.linux-x86_64-3.4/pysam:$ORIGIN:/gnu/store/m9vxvhdj691bq1f85lpflvnhcvrdilih-glibc-2.23/lib:/gnu/store/9nifwk709wajpyfwa0jzaa3p6mf10vxs-gcc-4.9.3-lib/lib:/gnu/store/xl19qrfzga52vrvp4ncccwjlnrjqwj95-ncurses-6.0/lib:/gnu/store/5992iq1v7arqa14ym3di58n4la0893nv-zlib-1.2.8/lib:/gnu/store/9nifwk709wajpyfwa0jzaa3p6mf10vxs-gcc-4.9.3-lib/lib/gcc/x86_64-unknown-linux-gnu/4.9.3/../../..]
Compared to the runpath of the same file currently in Guix:
0x000000000000001d (RUNPATH) Library runpath:
[/gnu/store/m4gc2wx4q9if1vrhgclpspdil7rqsn21-python-3.4.3/lib:/gnu/store/m9vxvhdj691bq1f85lpflvnhcvrdilih-glibc-2.23/lib:/gnu/store/9nifwk709wajpyfwa0jzaa3p6mf10vxs-gcc-4.9.3-lib/lib:/gnu/store/xl19qrfzga52vrvp4ncccwjlnrjqwj95-ncurses-6.0/lib:/gnu/store/5992iq1v7arqa14ym3di58n4la0893nv-zlib-1.2.8/lib:/gnu/store/9nifwk709wajpyfwa0jzaa3p6mf10vxs-gcc-4.9.3-lib/lib/gcc/x86_64-unknown-linux-gnu/4.9.3/../../..]
If a folder named "$CWD/build/lib.linux-x86_64-3.4/pysam exists, it
could potentially allow for code injection, which is troubling.
I opened an issue on their tracker, but don't think it's worth holding
the patch: https://github.com/pysam-developers/pysam/issues/347
>
> Also, I notice that pysam bundles htslib, bcftools and samtools C code.
> Hopefully it should be straightforward enough to remove htslib as there
> are install instructions, I'm not sure about the other two. This
> shouldn't block the patch here, but would you mind taking a look?
> http://pysam.readthedocs.io/en/latest/installation.html#installation
I had a go at this, and also enabled tests since I was reading the build
system anyway. Samtools and bcftools does not seem possible to un-bundle
at this time, but htslib was straightforward.
Enabling tests required packaging bcftools as well. I've attached the
full patch series below. The first three commits are unchanged.
Thanks!
>From 0a6220a664ecf3aa967211db5bf9e9b9e539220d Mon Sep 17 00:00:00 2001
From: Marius Bakke <mba...@fastmail.com>
Date: Fri, 16 Sep 2016 18:05:15 +0100
Subject: [PATCH 1/6] gnu: python-pysam: Update to 0.9.1.4.
* gnu/packages/bioinformatics.scm (python-pysam, python2-pysam): Update
to 0.9.1.4.
---
gnu/packages/bioinformatics.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 7a27e0b..47b884e 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -1290,13 +1290,13 @@ multiple sequence alignments.")
(define-public python-pysam
(package
(name "python-pysam")
- (version "0.8.4")
+ (version "0.9.1.4")
(source (origin
(method url-fetch)
(uri (pypi-uri "pysam" version))
(sha256
(base32
- "1slx5mb94mzm5qzk52q270sab0sar95j67w1g1k452nz3s9j7krh"))))
+ "1i1djacqbr88y7w18b4aa78zxnsyr4sz7yqdq2spi7gs0y6pzvjn"))))
(build-system python-build-system)
(arguments
`(#:tests? #f ; tests are excluded in the manifest
--
2.10.0
>From 6a8ddd17d7f11e0687217c8649a452a1a56c596c Mon Sep 17 00:00:00 2001
From: Marius Bakke <mba...@fastmail.com>
Date: Sat, 17 Sep 2016 10:33:28 +0100
Subject: [PATCH 2/6] gnu: Add python-pyxb.
* gnu/packages/xml.scm (python-pyxb, python2-pyxb): New variables.
---
gnu/packages/xml.scm | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/gnu/packages/xml.scm b/gnu/packages/xml.scm
index 7befad5..cd3426b 100644
--- a/gnu/packages/xml.scm
+++ b/gnu/packages/xml.scm
@@ -13,6 +13,7 @@
;;; Copyright © 2016 Jan Nieuwenhuizen <jann...@gnu.org>
;;; Copyright © 2016 ng0 <n...@we.make.ritual.n0.is>
;;; Copyright © 2016 Tobias Geerinckx-Rice <m...@tobias.gr>
+;;; Copyright © 2016 Marius Bakke <mba...@fastmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -729,6 +730,30 @@ different Unicode encodings which happen automatically during
parsing/saving.")
(license license:expat)))
+(define-public python-pyxb
+ (package
+ (name "python-pyxb")
+ (version "1.2.5")
+ (source (origin
+ (method url-fetch)
+ (uri (pypi-uri "PyXB" version))
+ (sha256
+ (base32
+ "0rzzwibfqa28gxgcxx4cybx1qcg0g6fand06ykj3gz7z5kp653sf"))))
+ (build-system python-build-system)
+ (home-page "http://pyxb.sourceforge.net/";)
+ (synopsis "Python XML Schema Bindings")
+ (description
+ "PyXB (\"pixbee\") is a pure Python package that generates Python source
+code for classes that correspond to data structures defined by XMLSchema.")
+ (license (list license:asl2.0 ; Most files.
+ license:expat ; pyxb/utils/six.py
+ license:gpl2 ; bundled jquery in doc is dual MIT/GPL2
+ license:psfl)))) ; pyxb/utils/activestate.py
+
+(define-public python2-pyxb
+ (package-with-python2 python-pyxb))
+
(define-public xmlto
(package
(name "xmlto")
--
2.10.0
>From 396cbcbfda24ce5cd1caecedf01407bf6a1f70a1 Mon Sep 17 00:00:00 2001
From: Marius Bakke <mba...@fastmail.com>
Date: Sat, 17 Sep 2016 10:42:56 +0100
Subject: [PATCH 3/6] gnu: python2-pbcore: Update to 1.2.10.
* gnu/packages/bioinformatics.scm (python2-pbcore): Update to 1.2.10.
[propagated-inputs]: New field. Add python2-pyxb.
---
gnu/packages/bioinformatics.scm | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 47b884e..a598351 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -3376,13 +3376,13 @@ interrupted by stop codons. OrfM finds and prints these ORFs.")
(define-public python2-pbcore
(package
(name "python2-pbcore")
- (version "1.2.8")
+ (version "1.2.10")
(source (origin
(method url-fetch)
(uri (pypi-uri "pbcore" version))
(sha256
(base32
- "02pfn5raa3zf739672bg0dkx7z3j2c4nx7vmpfjqy5b12jrqpymk"))))
+ "1kjmv891d6qbpp4shhhvkl02ff4q5xlpnls2513sm2cjcrs52f1i"))))
(build-system python-build-system)
(arguments `(#:python ,python-2)) ; pbcore requires Python 2.7
(inputs
@@ -3395,6 +3395,8 @@ interrupted by stop codons. OrfM finds and prints these ORFs.")
("python-nose" ,python2-nose)
("python-setuptools" ,python2-setuptools)
("python-sphinx" ,python2-sphinx)))
+ (propagated-inputs
+ `(("python-pyxb" ,python2-pyxb)))
(home-page "http://pacificbiosciences.github.io/pbcore/";)
(synopsis "Library for reading and writing PacBio data files")
(description
--
2.10.0
>From 2f61ecb17e155c41624a81d4d95d2e80823da02c Mon Sep 17 00:00:00 2001
From: Marius Bakke <mba...@fastmail.com>
Date: Thu, 22 Sep 2016 13:47:53 +0100
Subject: [PATCH 4/6] gnu: Add bcftools.
* gnu/packages/bioinformatics.scm (bcftools): New variable.
---
gnu/packages/bioinformatics.scm | 60 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 60 insertions(+)
diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index a598351..e9ab05f 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -265,6 +265,66 @@ instance, it implements several methods to assess contig-wise read coverage.")
BAM files.")
(license license:expat)))
+(define-public bcftools
+ (package
+ (name "bcftools")
+ (version "1.3.1")
+ (source (origin
+ (method url-fetch)
+ (uri (string-append
+ "https://github.com/samtools/bcftools/releases/download/";
+ version "/bcftools-" version ".tar.bz2"))
+ (sha256
+ (base32
+ "095ry68vmz9q5s1scjsa698dhgyvgw5aicz24c19iwfbai07mhqj"))
+ (modules '((guix build utils)))
+ (snippet
+ ;; Delete bundled htslib.
+ '(delete-file-recursively "htslib-1.3.1"))))
+ (build-system gnu-build-system)
+ (arguments
+ `(#:test-target "test"
+ #:make-flags (list
+ "USE_GPL=1"
+ (string-append "prefix=" (assoc-ref %outputs "out"))
+ (string-append
+ "HTSDIR=" (assoc-ref %build-inputs "htslib") "/include")
+ (string-append
+ "HTSLIB=" (assoc-ref %build-inputs "htslib") "/lib/libhts.a")
+ (string-append
+ "BGZIP=" (assoc-ref %build-inputs "htslib") "/bin/bgzip")
+ (string-append
+ "TABIX=" (assoc-ref %build-inputs "htslib") "/bin/tabix"))
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'patch-Makefile
+ (lambda _
+ (substitute* "Makefile"
+ ;; Do not attempt to build htslib.
+ (("^include \\$\\(HTSDIR\\)/htslib\\.mk") "")
+ ;; Link against GSL cblas.
+ (("-lcblas") "-lgslcblas"))
+ #t))
+ (delete 'configure)
+ (add-before 'check 'patch-/bin/bash
+ (lambda _
+ (substitute* "test/test.pl"
+ (("/bin/bash") (which "bash"))))))))
+ (native-inputs
+ `(("htslib" ,htslib)
+ ("perl" ,perl)))
+ (inputs
+ `(("gsl" ,gsl)
+ ("zlib" ,zlib)))
+ (home-page "https://samtools.github.io/bcftools/";)
+ (synopsis "Utilities for variant calling and manipulating VCFs and BCFs")
+ (description
+ "BCFtools is a set of utilities that manipulate variant calls in the
+Variant Call Format (VCF) and its binary counterpart BCF. All commands work
+transparently with both VCFs and BCFs, both uncompressed and BGZF-compressed.")
+ ;; The sources are dual MIT/GPL, but becomes GPL-only when USE_GPL=1.
+ (license (list license:gpl3+ license:expat))))
+
(define-public bedops
(package
(name "bedops")
--
2.10.0
>From 29d8aab712b611a6c786aa9b2c47f45e51203ca1 Mon Sep 17 00:00:00 2001
From: Marius Bakke <mba...@fastmail.com>
Date: Thu, 22 Sep 2016 14:08:05 +0100
Subject: [PATCH 5/6] gnu: python-pysam: Delete bundled htslib.
* gnu/packages/bioinformatics.scm (python-pysam)[source]: Add snippet
to delete htslib.
[arguments]: Adjust to modify-phases. Add htslib flags in 'set-flags.
[propagated-inputs]: New field. Add htslib.
---
gnu/packages/bioinformatics.scm | 24 +++++++++++++++++-------
1 file changed, 17 insertions(+), 7 deletions(-)
diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index e9ab05f..edfb943 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -1356,17 +1356,27 @@ multiple sequence alignments.")
(uri (pypi-uri "pysam" version))
(sha256
(base32
- "1i1djacqbr88y7w18b4aa78zxnsyr4sz7yqdq2spi7gs0y6pzvjn"))))
+ "1i1djacqbr88y7w18b4aa78zxnsyr4sz7yqdq2spi7gs0y6pzvjn"))
+ (modules '((guix build utils)))
+ (snippet
+ ;; Drop bundled htslib. TODO: Also remove samtools and bcftools.
+ '(delete-file-recursively "htslib"))))
(build-system python-build-system)
(arguments
`(#:tests? #f ; tests are excluded in the manifest
#:phases
- (alist-cons-before
- 'build 'set-flags
- (lambda _
- (setenv "LDFLAGS" "-lncurses")
- (setenv "CFLAGS" "-D_CURSES_LIB=1"))
- %standard-phases)))
+ (modify-phases %standard-phases
+ (add-before 'build 'set-flags
+ (lambda* (#:key inputs #:allow-other-keys)
+ (setenv "LDFLAGS" "-lncurses")
+ (setenv "HTSLIB_MODE" "external")
+ (setenv "HTSLIB_LIBRARY_DIR" (string-append
+ (assoc-ref inputs "htslib") "/lib"))
+ (setenv "HTSLIB_INCLUDE_DIR" (string-append
+ (assoc-ref inputs "htslib") "/include"))
+ (setenv "CFLAGS" "-D_CURSES_LIB=1"))))))
+ (propagated-inputs
+ `(("htslib" ,htslib)))
(inputs
`(("ncurses" ,ncurses)
("zlib" ,zlib)))
--
2.10.0
>From ab4f0a5174554a7f2b7b243b5488654de7d81ac1 Mon Sep 17 00:00:00 2001
From: Marius Bakke <mba...@fastmail.com>
Date: Thu, 22 Sep 2016 14:11:59 +0100
Subject: [PATCH 6/6] gnu: python-pysam: Enable tests.
* gnu/packages/bioinformatics.scm (python-pysam)[source]: Change from
PyPi to source archive due to missing test data.
[arguments]: Add check phase after install.
[native-inputs]: Add python-nose, samtools and bcftools.
---
gnu/packages/bioinformatics.scm | 33 ++++++++++++++++++++++++++++-----
1 file changed, 28 insertions(+), 5 deletions(-)
diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index edfb943..abbe601 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -1353,18 +1353,21 @@ multiple sequence alignments.")
(version "0.9.1.4")
(source (origin
(method url-fetch)
- (uri (pypi-uri "pysam" version))
+ ;; Test data is missing on PyPi.
+ (uri (string-append
+ "https://github.com/pysam-developers/pysam/archive/v";
+ version ".tar.gz"))
+ (file-name (string-append name "-" version ".tar.gz"))
(sha256
(base32
- "1i1djacqbr88y7w18b4aa78zxnsyr4sz7yqdq2spi7gs0y6pzvjn"))
+ "0y41ssbg6nvn2jgcbnrvkzblpjcwszaiv1rgyd8dwzjkrbfsgsmc"))
(modules '((guix build utils)))
(snippet
;; Drop bundled htslib. TODO: Also remove samtools and bcftools.
'(delete-file-recursively "htslib"))))
(build-system python-build-system)
(arguments
- `(#:tests? #f ; tests are excluded in the manifest
- #:phases
+ `(#:phases
(modify-phases %standard-phases
(add-before 'build 'set-flags
(lambda* (#:key inputs #:allow-other-keys)
@@ -1374,7 +1377,24 @@ multiple sequence alignments.")
(assoc-ref inputs "htslib") "/lib"))
(setenv "HTSLIB_INCLUDE_DIR" (string-append
(assoc-ref inputs "htslib") "/include"))
- (setenv "CFLAGS" "-D_CURSES_LIB=1"))))))
+ (setenv "CFLAGS" "-D_CURSES_LIB=1")))
+ (delete 'check)
+ (add-after 'install 'check
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (setenv "PYTHONPATH"
+ (string-append
+ (getenv "PYTHONPATH")
+ ":" (assoc-ref outputs "out")
+ "/lib/python"
+ (string-take (string-take-right
+ (assoc-ref inputs "python") 5) 3)
+ "/site-packages"))
+ ;; Step out of source dir so python does not import from CWD.
+ (chdir "tests")
+ (setenv "HOME" "/tmp")
+ (and (zero? (system* "make" "-C" "pysam_data"))
+ (zero? (system* "make" "-C" "cbcf_data"))
+ (zero? (system* "nosetests" "-v"))))))))
(propagated-inputs
`(("htslib" ,htslib)))
(inputs
@@ -1382,6 +1402,9 @@ multiple sequence alignments.")
("zlib" ,zlib)))
(native-inputs
`(("python-cython" ,python-cython)
+ ("python-nose" ,python-nose)
+ ("samtools" ,samtools)
+ ("bcftools" ,bcftools)
("python-setuptools" ,python-setuptools)))
(home-page "https://github.com/pysam-developers/pysam";)
(synopsis "Python bindings to the SAMtools C API")
--
2.10.0
