Re: [PATCH] WIP patches for the rust importer

[Efraim Flashner]

On Fri, Nov 29, 2019 at 04:59:31AM -0800, Martin Becze wrote:
> On 2019-11-28 12:22, Efraim Flashner wrote:
> > I'll take a look at it in a minute. I figured with the versioned
> > requirements we would always want to be specific in version numbers for
> > crate dependents so I figured it made sense. Also, if we did want to
> > provide an unversioned '-latest' version we could declare an extra
> > variable '(define-public rust-libc rust-libc-0.2)' and now rust-libc
> > points to rust-libc-0.2.
> 
> one thing to keep in mind is that (recursive-import-semver) is suppose
> to be generic so what ever naming logic we apply here for rust libs
> should be universal.

As far as other languages go, I'm pretty sure python only requests
major+minor version. Perl's minimum versions are normally a full version
string, so truncating that probably wouldn't be a good choice. Another
option would be to truncate to just major+minor (in the inputs) only
when it comes time to print out the package.

Also, we don't want Guix to think 1.2.3 can only be upgraded to 1.2.3.4
and not to 1.2.4.

-- 
Efraim Flashner  אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature

Re: [PATCH] WIP patches for the rust importer

[Efraim Flashner]

On Fri, Nov 29, 2019 at 07:59:35AM -0800, Martin Becze wrote:
> On 2019-11-28 12:22, Efraim Flashner wrote:
> > On Wed, Nov 27, 2019 at 04:36:20PM -0800, mjbe...@riseup.net wrote:
> >>
> >> > I'd love to see what you have so far if you want to share
> >>
> >> Okie Dokie, I posted it and cc'd ya.
> >>
> >> Also I took a look at your patches.
> >> 0001-import-crate-Don-t-include-optional-dependencies.patch should work
> >> just fine with my patch. But
> >> 0003-import-crate-Honor-versioned-dependencies-when-impor.patch will not
> >> work.
> >>
> >> I took a different route here with the naming. If you are interested take
> >> a look take a look at my second patch. (recusive-import-semver) only will
> >> add the version number to the name if the crate being imported is not the
> >> latest version. I thought this was more inline with the canonical names,
> >> but if always adding version number the export symbol is desirable it will
> >> simplify things.
> >>
> > 
> > I'll take a look at it in a minute. I figured with the versioned
> > requirements we would always want to be specific in version numbers for
> > crate dependents so I figured it made sense. Also, if we did want to
> > provide an unversioned '-latest' version we could declare an extra
> > variable '(define-public rust-libc rust-libc-0.2)' and now rust-libc
> > points to rust-libc-0.2.
> > 
> >> Also I added a function (find-packages-by-name*/direct) to packages.scm
> >> which will return the export symbol of a package that already exists. I
> >> use this in case there are some non-canocal export name already added.
> >>
> 
> I added the no-optional-dep logic to the recusive-semver patch
> (https://issues.guix.gnu.org/issue/38408), but it seems to break
> packages. I'm testing on the recursive importer on "hello-cli". Attach
> is the patch to add the logic and the  before and after output for "guix
> import crate -r hello-cli". Removing all the optional deps breaks clap
> here for some reason which I haven't figured out.

Looking at the two attached files I want to bring attention to the size
of them:
after.scm   [text/plain, base64, utf-8, 5.7K]
before.scm  [text/plain, base64, utf-8, 226K]

One way to fix this (in addition to your "default to the don't build"
argument) is to keep rust-clap in it's broken state and define a
different rust-clap for when we actually want it and to have that use
enough inputs to actually build. I do now actually realize that only
really works if we keep all the crates hidden, which I don't think we
want. So I guess that has us sending bug reports upstream that some of
the optional dependencies aren't actually optional.

In any case, I think it'd be better to skip the optional dependencies
and then add them back in as needed to actually build the crates we
want.


-- 
Efraim Flashner  אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature

Re: ARM VM with networking support?

[Mathieu Othacehe]

Hey Ludo,

> Here’s my tentative config file.

I just generated a disk-image with your config file! Just like you, I did
hit the "unsupported ioctl" issue on both master and core-updates
branch.

The reason is that the command run when producing the disk image is:

  qemu-arm qemu-system-arm disk-image-builder
^   ^
|   |-- Native qemu (so for armhf-linux architecture).
|
Run by binfmt because qemu-system-arm is a binary for armhf
architecture.

So a syscall issued somewhere in disk-image-builder goes through the
kernel emulated by qemu-system-arm and via qemu-arm to your host kernel.

The failing syscall is number 47601 (FS_IOC32_GETVERSION), I don't know
why it is not supported by our host kernel. However, using a
qemu-system-arm built for arm doesn't make much sense here, because we
add an unecessary (and failing) layer of emulation.

So what I would propose is to produce a disk-image using a qemu-system-*
built for the host architecture (and not the system specified by
--system argument). This remains true when cross-compiling a system.

The attached patch does thin in an ugly way but that solves the issue.

WDYT?

Mathieu
>From e107692f3a98c19d2457050635818226ee675c52 Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe 
Date: Sun, 1 Dec 2019 16:49:36 +0100
Subject: [PATCH] vm: Force native qemu usage.

---
 gnu/system/vm.scm | 16 +++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm
index 8609bd2ace..e760e7f42d 100644
--- a/gnu/system/vm.scm
+++ b/gnu/system/vm.scm
@@ -68,6 +68,7 @@
   #:use-module (gnu system uuid)
 
   #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-9)
   #:use-module (srfi srfi-26)
   #:use-module (rnrs bytevectors)
   #:use-module (ice-9 match)
@@ -141,6 +142,16 @@
  packages
   (list guile-gcrypt guile-sqlite3)))
 
+(define-record-type 
+  (%native-qemu qemu system)
+  native-qemu?
+  (qemu native-qemu-qemu)
+  (system native-qemu-system))
+
+(define-gexp-compiler (native-qemu-compiler (native-qemu ) system target)
+  (package->derivation (native-qemu-qemu native-qemu)
+   (native-qemu-system native-qemu)))
+
 (define* (expression->derivation-in-linux-vm name exp
  #:key
  (system (%current-system)) target
@@ -193,6 +204,9 @@ made available under the /xchg CIFS share."
 (reboot)
 (exit 1
 
+  (define qemu-native
+(%native-qemu qemu (@ (guix config) %system)))
+
   (let ((initrd (or initrd
 (base-initrd file-systems
  #:on-error 'backtrace
@@ -215,7 +229,7 @@ made available under the /xchg CIFS share."
(gnu build vm))
 
   (let* ((native-inputs
-  '#+(list qemu (canonical-package coreutils)))
+  '#+(list qemu-native (canonical-package coreutils)))
  (linux   (string-append #$linux "/"
  #$(system-linux-image-file-name)))
  (initrd  #$initrd)
-- 
2.24.0

Re: Packaging Jami progress

[Jan]

I see the changes got merged. Tested Jami and it works great :)
I guess the wrapper was the issue.
I am going to update it to the latest version, this time I'll try
sending the patches myself in order to learn something new.
I also told the devs about our success:
https://git.jami.net/savoirfairelinux/ring-project/issues/691
Should I ask them to add the option for Guix on this page?
https://jami.net/download-jami-linux/
I mean, do we consider the package to be more or less ready, so they
can show it?


Jan Wielkiewicz

Re: Packaging Jami progress

[Jan]

On Sun, 01 Dec 2019 18:32:36 +0100
Pierre Neidhardt  wrote:

> If it works for you, then I think it's ready!
I'll do some more tests - video/audio conference, but for now
everything works - chat, sending files.

As for updating, Sébastien Blin said that since then they had bumped
opendht and had patched ffmpeg:
https://git.jami.net/savoirfairelinux/ring-project/issues/691#note_16723

The first should be easy, but the second probably requires adding
ffmpeg-jami (if I'm correct).


Jan Wielkiewicz

Re: Updating Mono

[Gábor Boskovits]

Hello,

Brett Gilio  ezt írta (időpont: 2019. dec. 1., Vas 5:13):

>
> Hey all, I am currently working on updating the mono package that was
> first placed in our package definitions by Jan Nieuwenhuizen I believe.
>
> I am currently running into an issue where I am not able to understand
> which tests are failing, and thus how to disable them. By the looks of
> it, I can borrow the method of disabling tests used in the original
> recipe.
>
> Here is what I have for the updated package
>
> --8<---cut here---start->8---
> (define-public mono-6.4
>   (package
> (name "mono")
> (version "6.4.0.198")
> (source (origin
>   (method url-fetch)
>   (uri (string-append "
> http://download.mono-project.com/sources/mono/mono-";
>   version
>   ".tar.xz"))
>   (sha256
>(base32
> "1sn2fy49hr7acgy0cbf2r8cagis179ym63ryifgnzqr54n15426h"
> (build-system gnu-build-system)
> (arguments
>  `(#:parallel-tests? #f ; running tests in parallel fails
>#:phases
>(modify-phases %standard-phases
>  (replace 'bootstrap
>(lambda _
>  (copy-file "mono/mini/Makefile.am.in"
> "mono/mini/Makefile.in")
>  (invoke "autoreconf" "-vfi"))
> (native-inputs
>  `(("gettext" ,gettext-minimal)
>("glib" ,glib)
>("which" ,which)
>("cmake" ,cmake)
>("libtool" ,libtool)
>("autoconf" ,autoconf)
>("automake" ,automake)
>("libxslt" ,libxslt)
>("perl" ,perl)
>("python" ,python-2)))
> (synopsis "Compiler and libraries for the C# programming language")
> (description "Mono is a compiler, vm, debugger and set of libraries for
> C#, a C-style programming language from Microsoft that is very similar to
> Java.")
> (home-page "https://www.mono-project.com/";)
> (license license:x11)))
>
> (define-public mono mono-6.4)
> --8<---cut here---end--->8---
>
> Here is the point in the logs where things get a little screwy:
>
> --8<---cut here---start->8---
> 
> 4 test(s) passed, 0 test(s) did not pass.
> make[6]: Leaving directory
> '/tmp/guix-build-mono-6.4.0.198.drv-0/mono-6.4.0.198/mono/tests'
> make[6]: Entering directory
> '/tmp/guix-build-mono-6.4.0.198.drv-0/mono-6.4.0.198/mono/tests'
> Removing empty logs...
> find . '(' -name "*.stdout" -o -name "*.stderr" ')' -size 0 -exec rm {} \;
> make[6]: Leaving directory
> '/tmp/guix-build-mono-6.4.0.198.drv-0/mono-6.4.0.198/mono/tests'
> make[6]: Entering directory
> '/tmp/guix-build-mono-6.4.0.198.drv-0/mono-6.4.0.198/mono/tests'
> make -C testing_gac run-test
> make[7]: Entering directory
> '/tmp/guix-build-mono-6.4.0.198.drv-0/mono-6.4.0.198/mono/tests/testing_gac'
> true
> make test-app-both
> make[8]: Entering directory
> '/tmp/guix-build-mono-6.4.0.198.drv-0/mono-6.4.0.198/mono/tests/testing_gac'
> make[8]: *** No rule to make target 'app-both.cs', needed by
> 'app-both.exe'.  Stop.
> make[8]: Leaving directory
> '/tmp/guix-build-mono-6.4.0.198.drv-0/mono-6.4.0.198/mono/tests/testing_gac'
> make[7]: *** [Makefile:681: run-test] Error 2
> make[7]: Leaving directory
> '/tmp/guix-build-mono-6.4.0.198.drv-0/mono-6.4.0.198/mono/tests/testing_gac'
> make[6]: *** [Makefile:3494: runtest-gac-loading] Error 2
> make[6]: Leaving directory
> '/tmp/guix-build-mono-6.4.0.198.drv-0/mono-6.4.0.198/mono/tests'
> make[6]: Entering directory
> '/tmp/guix-build-mono-6.4.0.198.drv-0/mono-6.4.0.198/mono/tests'
> make -C assembly-load-reference run-test
> make[7]: Entering directory
> '/tmp/guix-build-mono-6.4.0.198.drv-0/mono-6.4.0.198/mono/tests/assembly-load-reference'
> true
> make run-assembly-load-reference-tests
> make[8]: Entering directory
> '/tmp/guix-build-mono-6.4.0.198.drv-0/mono-6.4.0.198/mono/tests/assembly-load-reference'
> make[8]: *** No rule to make target 'samedir/LoadFromMain.exe', needed by
> 'run-assembly-load-reference-tests'.  Stop.
> make[8]: Leaving directory
> '/tmp/guix-build-mono-6.4.0.198.drv-0/mono-6.4.0.198/mono/tests/assembly-load-reference'
> make[7]: *** [Makefile:681: run-test] Error 2
> make[7]: Leaving directory
> '/tmp/guix-build-mono-6.4.0.198.drv-0/mono-6.4.0.198/mono/tests/assembly-load-reference'
> make[6]: *** [Makefile:3500: runtest-assembly-load-reference] Error 2
> make[6]: Leaving directory
> '/tmp/guix-build-mono-6.4.0.198.drv-0/mono-6.4.0.198/mono/tests'
> make[5]: *** [Makefile:2663: check-local] Error 1
> make[5]: Leaving directory
> '/tmp/guix-build-mono-6.4.0.198.drv-0/mono-6.4.0.198/mono/tests'
> make[4]: *** [Makefile:2516: check-am] Error 2
> make[4]: Leaving directory
> '/tmp/guix-build-mono-6.4.0.198.drv-0/mono-6.4.0.198/mono/tests'
> make[3]: *** [Makefile:2363: check-recursive] Error 1
> make[3]: Leaving directory
> '/tmp/guix-build-mono-6.4.0.198.drv

Re: New POWER9 machines for the Guix build farm?

[dftxbs3e]

Hello,

There is libre firmware being worked on, latest news say that it does 
work but some features are not implemented. I have not tested that 
firmware however.


There is a contest organized by RaptorCS to liberate the firmware: 
https://www.raptorcs.com/TALOSII/nic_fw_contest.php


And here's the ongoing work:

https://github.com/meklort/bcm5719-fw

https://wiki.raptorcs.com/wiki/BCM5719

An interesting thing to note is that 10Gbit network cards from the same 
vendor (Broadcom) are implemented fully with hardware ASICs and do not 
require firmware at all. So that's a work around to the problem. source: 
developer of the libre firmware.


dftxbs3e


On 11/29/19 1:50 PM, Marco van Hulten wrote:

Hello—

On 29 Nov 09:25 dftxbs3e wrote:

I'm really happy to hear this! I also do own a POWER9 machine.

I have a Talos II with 2x 8 SMT4 cores POWER9 DD2.2 CPUs (64 threads),
64GB of RAM and an AMD WX7100 GPU (amdgpu libre driver).

I run Fedora on it since more than a year now, it's been my main
workstation without any major issue.

It's great to hear of such an experience!  I have an amd64 server at
home that makes quite a bit of noise and, standing next to the server,
an amd64 workstation that makes even more noise, both of which are over
10 years old.  I am considering replacing both systems for a single
Talos II.

According to the mainboard specifications [1], Ethernet is driven by a
proprietary chip (Broadcom BCM5719).  Is there libre firmware that is
shown to work reliably?

I guess it should be because it is FSF-certified to Respect Your
Freedom, but could someone confirm this specifically for the network
interface?

[1]: https://wiki.raptorcs.com/wiki/Talos_II


I'm happy to provide assistance, I encourage GNU Guix developers to join
#talos-workstation on Freenode's IRC, we have a friendly active
community that is always happy to help on POWER9 matters.

Fantastic to see this work out!


The other free modern architecture, RISC-V, is that like a sort of
competitor of POWER9?  Or is it more like Talos II being the obvious
choice for a server (and workstation) because it is available now/more
established?

—Marco

Re: [PATCH] WIP patches for the rust importer

[Martin Becze]

On 2019-12-01 08:54, Efraim Flashner wrote:

> Also, we don't want Guix to think 1.2.3 can only be upgraded to 1.2.3.4
> and not to 1.2.4.

I'm kinda lost on this one. Why would guix think 1.2.3 can only be
upgraded to 1.2.3.4?

Re: [PATCH] WIP patches for the rust importer

[Martin Becze]

On 2019-12-01 08:59, Efraim Flashner wrote:
> On Fri, Nov 29, 2019 at 07:59:35AM -0800, Martin Becze wrote:
>> On 2019-11-28 12:22, Efraim Flashner wrote:
>> > On Wed, Nov 27, 2019 at 04:36:20PM -0800, mjbe...@riseup.net wrote:
>> >>
>> >> > I'd love to see what you have so far if you want to share
>> >>
>> >> Okie Dokie, I posted it and cc'd ya.
>> >>
>> >> Also I took a look at your patches.
>> >> 0001-import-crate-Don-t-include-optional-dependencies.patch should work
>> >> just fine with my patch. But
>> >> 0003-import-crate-Honor-versioned-dependencies-when-impor.patch will not
>> >> work.
>> >>
>> >> I took a different route here with the naming. If you are interested take
>> >> a look take a look at my second patch. (recusive-import-semver) only will
>> >> add the version number to the name if the crate being imported is not the
>> >> latest version. I thought this was more inline with the canonical names,
>> >> but if always adding version number the export symbol is desirable it will
>> >> simplify things.
>> >>
>> >
>> > I'll take a look at it in a minute. I figured with the versioned
>> > requirements we would always want to be specific in version numbers for
>> > crate dependents so I figured it made sense. Also, if we did want to
>> > provide an unversioned '-latest' version we could declare an extra
>> > variable '(define-public rust-libc rust-libc-0.2)' and now rust-libc
>> > points to rust-libc-0.2.
>> >
>> >> Also I added a function (find-packages-by-name*/direct) to packages.scm
>> >> which will return the export symbol of a package that already exists. I
>> >> use this in case there are some non-canocal export name already added.
>> >>
>>
>> I added the no-optional-dep logic to the recusive-semver patch
>> (https://issues.guix.gnu.org/issue/38408), but it seems to break
>> packages. I'm testing on the recursive importer on "hello-cli". Attach
>> is the patch to add the logic and the  before and after output for "guix
>> import crate -r hello-cli". Removing all the optional deps breaks clap
>> here for some reason which I haven't figured out.
> 
> Looking at the two attached files I want to bring attention to the size
> of them:
> after.scm   [text/plain, base64, utf-8, 5.7K]
> before.scm  [text/plain, base64, utf-8, 226K]
> 
> One way to fix this (in addition to your "default to the don't build"
> argument) is to keep rust-clap in it's broken state and define a
> different rust-clap for when we actually want it and to have that use
> enough inputs to actually build. I do now actually realize that only
> really works if we keep all the crates hidden, which I don't think we
> want. So I guess that has us sending bug reports upstream that some of
> the optional dependencies aren't actually optional.
> 
> In any case, I think it'd be better to skip the optional dependencies
> and then add them back in as needed to actually build the crates we
> want.

oh to be more clear. Even with "#:skip-build? #t" on all the rust libs,
things fail if I omit one optional dependency from somewhere. I'm just
testing on hello-cli but hello-cli inculdes clap which has alot of
optional deps. In ./guix/build-systems/cargo.scm on line 186 it says 

"Cargo requires all transitive crate dependencies' sources to be
available
in its index, even if they are optional (this is so it can generate
deterministic Cargo.lock files regardless of the target platform or
enabled
features). Thus we need all transitive crate dependencies for any cargo
dev-dependencies, but this is only needed when building/testing a crate
directly
(i.e. we will never need transitive dev-dependencies for any dependency
crates)."

I haven't really dug into the cargo build-system yet but my guess is
that the problem lies there. Asooo i totally could have missed
something simple.. idk

Re: [PATCH] WIP patches for the rust importer

[Ivan Petkov]

Hi Martin!

> On Dec 1, 2019, at 7:17 PM, Martin Becze  wrote:
> 
> oh to be more clear. Even with "#:skip-build? #t" on all the rust libs,
> things fail if I omit one optional dependency from somewhere. I'm just
> testing on hello-cli but hello-cli inculdes clap which has alot of
> optional deps. In ./guix/build-systems/cargo.scm on line 186 it says 
> 
> "Cargo requires all transitive crate dependencies' sources to be
> available
> in its index, even if they are optional (this is so it can generate
> deterministic Cargo.lock files regardless of the target platform or
> enabled
> features). Thus we need all transitive crate dependencies for any cargo
> dev-dependencies, but this is only needed when building/testing a crate
> directly
> (i.e. we will never need transitive dev-dependencies for any dependency
> crates)."
> 
> I haven't really dug into the cargo build-system yet but my guess is
> that the problem lies there. Asooo i totally could have missed
> something simple.. idk

Yes, this is the primary limitation of packaging rust crates into guix
without manually needing to edit the Cargo.toml definition.

My opinion is that the easiest (to maintain) method for incorporating
rust packages into guix is to bite the bullet and perform a (source)
import of all transitive dependencies that a package might need.
Manually keeping up with tweaking the Cargo.toml file for every single
potential package, across every single published version will make
you go mad :)

Then the challenge would be how to automate the incremental importing
from crates.io  (for example, some crates require a specific 
x.y.z version of
a dependency, some are willing to work with x.y.*, some have ranges, etc.)
and making sure that all packaged crates point to the appropriate dependencies
as they get updated in guix.

—Ivan