Hi all!
TLDR if you just want to know what's the final patch that worked, grep for
"final patch which works is the following"
This is a long email, but it is only long because I want to be very clear and I
include a lot of snippets of code and logs.
I originally wrote my NSIS package (introduced in e214a22007) on top of
7150743522, which was before the GREAT CORE-UPDATES MERGE. However, after
rebasing it over the GREAT CORE-UPDATES MERGE, the package stopped working. My
investigations into this failure has revealed some insights which might be
useful for other cross-compiling packages and for resolving the long-standing
"important" issue #30756.
For some context, the NSIS package is somewhat special. It requires that both
native compilers and mingw-w64 cross-compilers be available, as NSIS is a
program used to create installers meant to be run on Windows.
My original method of achieving this in pre-GREAT CORE-UPDATES MERGE Guix can be
see in e214a22007, whereby I added a 'fix-env phase. In that 'fix-env phase, I
would remove mingw-w64 paths from CPLUS_INCLUDE_PATH, C_INCLUDE_PATH, and
LIBRARY_PATH, and put them in CROSS_CPLUS_INCLUDE_PATH, CROSS_C_INCLUDE_PATH,
and CROSS_LIBRARY_PATH respectively. This way, the native compilers won't think
that the mingw-w64 paths were something they were supposed to use and get
confused.
As part of the GREAT CORE-UPDATES MERGE, it would seem that CPLUS_INCLUDE_PATH
and C_INCLUDE_PATH are no longer set in the build environment. Rather, only
CPATH is set.
Now, you would think that the solution here is just to replace
CPLUS_INCLUDE_PATH and C_INCLUDE_PATH in my 'fix-env phase with CPATH like so:
diff --git a/gnu/packages/installers.scm b/gnu/packages/installers.scm
index c987254d61..9d29e32485 100644
--- a/gnu/packages/installers.scm
+++ b/gnu/packages/installers.scm
@@ -92,7 +92,7 @@
;; CROSS_-prefixed version of env vars
(setenv (string-append "CROSS_" env-name)
(filter-delimited-string env-val
mingw-path?))))
- '("CPLUS_INCLUDE_PATH" "LIBRARY_PATH"
"C_INCLUDE_PATH"))))
+ '("LIBRARY_PATH" "CPATH"))))
(add-before 'build 'fix-target-detection
(lambda _
;; NSIS target detection is screwed up, manually
However, that does not work. Trying to build this fix results in the curious
error that we encounter in the long-standing "important" issue #30756 (to be
clear, this is an error while in the `build' phase of the nsis-x86_64 package,
not mingw-w64):
In file included from
/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/include/c++/bits/stl_algo.h:59:0,
from
/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/include/c++/algorithm:62,
from Contrib/InstallOptions/InstallerOptions.cpp:23:
/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/include/c++/cstdlib:75:15:
fatal error: stdlib.h: No such file or directory
#include_next <stdlib.h>
^~~~~~~~~~
compilation terminated.
For some context, here is the full environment-variables file for that failed
build:
export
CPATH="/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/include:/gnu/store/qx7p7hiq90mi7r78hcr9cyskccy2j4bg-zlib-1.2.11/include:/gnu/store/l86azr7r3p5631wj3kk329jl1y1mpjgy-bzip2-1.0.6/include:/gnu/store/lbip9isk25isymvnb159l115xnacb5j8-xz-5.2.4/include:/gnu/store/6jdshxwdrad9mlhcqc9k0g24yw45rqf1-file-5.33/include:/gnu/store/2z9hsww76aag37p40671l9niq5pvvasx-gawk-5.0.1/include:/gnu/store/b5vpfzkr59bpgcsg1k9vvad9h5rwvpgk-make-4.2.1/include:/gnu/store/3hkdiscs4910r75njbrql10znxxn7chk-binutils-2.32/include:/gnu/store/x3jx25cd3q363mr7nbgzrhrv1vza6cf7-gcc-7.4.0/include:/gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/include:/gnu/store/7czrqpi0kwazras6pgyx0bhdn89pg0jl-linux-libre-headers-4.19.56/include"
export
CROSS_CPATH="/gnu/store/rabx878fijwgg0yl2bfsx0wvw7d7isnj-mingw-w64-x86_64-6.0.0/include"
export
CROSS_LIBRARY_PATH="/gnu/store/rabx878fijwgg0yl2bfsx0wvw7d7isnj-mingw-w64-x86_64-6.0.0/lib"
export GUIX_LD_WRAPPER_ALLOW_IMPURITIES="no"
export
GUIX_LOCPATH="/gnu/store/mmqp1xqffn6qw6v88i627c2bpbq36fcy-glibc-utf8-locales-2.29/lib/locale"
export HOME="/homeless-shelter"
export LC_ALL="en_US.utf8"
export
LIBRARY_PATH="/gnu/store/xiiafbq3c78kxkgv5zr54zpb8r4jz5ii-scons-python2-3.0.4/lib:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/lib:/gnu/store/qx7p7hiq90mi7r78hcr9cyskccy2j4bg-zlib-1.2.11/lib:/gnu/store/l86azr7r3p5631wj3kk329jl1y1mpjgy-bzip2-1.0.6/lib:/gnu/store/lbip9isk25isymvnb159l115xnacb5j8-xz-5.2.4/lib:/gnu/store/6jdshxwdrad9mlhcqc9k0g24yw45rqf1-file-5.33/lib:/gnu/store/2z9hsww76aag37p40671l9niq5pvvasx-gawk-5.0.1/lib:/gnu/store/3hkdiscs4910r75njbrql10znxxn7chk-binutils-2.32/lib:/gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib:/gnu/store/qky1x5bb2jygy58bn6y95ygfsmpakf52-glibc-2.29-static/lib:/gnu/store/mmqp1xqffn6qw6v88i627c2bpbq36fcy-glibc-utf8-locales-2.29/lib"
export NIX_BUILD_CORES="0"
export NIX_BUILD_TOP="/tmp/guix-build-nsis-x86_64-3.04.drv-0"
export NIX_STORE="/gnu/store"
export OLDPWD
export
PATH="/gnu/store/xiiafbq3c78kxkgv5zr54zpb8r4jz5ii-scons-python2-3.0.4/bin:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/bin:/gnu/store/9nspnmi8prly39p3xx4plvbk9dywhf4y-binutils-cross-x86_64-w64-mingw32-2.32/bin:/gnu/store/cnqpra8vr2l5fz00rr4yj4bp3hr00cfw-tar-1.32/bin:/gnu/store/py3k9zla9fj3z7430v4crqj5pyrsd3qj-gzip-1.10/bin:/gnu/store/l86azr7r3p5631wj3kk329jl1y1mpjgy-bzip2-1.0.6/bin:/gnu/store/lbip9isk25isymvnb159l115xnacb5j8-xz-5.2.4/bin:/gnu/store/6jdshxwdrad9mlhcqc9k0g24yw45rqf1-file-5.33/bin:/gnu/store/58sq8iabw3jkv0fvf95hd7sq2g4xcsnz-diffutils-3.7/bin:/gnu/store/v76scv4n63ip08g119rczh2mrw31zwpd-patch-2.7.6/bin:/gnu/store/g9d3wv1d68iflx57yp3mcp3k3sv8spsl-findutils-4.6.0/bin:/gnu/store/2z9hsww76aag37p40671l9niq5pvvasx-gawk-5.0.1/bin:/gnu/store/afmvfw1yhfal48n1kjq6bk6kcw8sc3db-sed-4.7/bin:/gnu/store/7iyvxhp2g3v3655zqwr6biz2h0lqv7pr-grep-3.3/bin:/gnu/store/9kzrrccpzl6i1sfwb0drb00gi2gwk0x0-coreutils-8.31/bin:/gnu/store/b5vpfzkr59bpgcsg1k9vvad9h5rwvpgk-make-4.2.1/bin:/gnu/store/29jhbbg1hf557x8j53f9sxd9imlmf02a-bash-minimal-5.0.7/bin:/gnu/store/nc5vlidpxbvngalng30nif8nb3j7gfy2-ld-wrapper-0/bin:/gnu/store/3hkdiscs4910r75njbrql10znxxn7chk-binutils-2.32/bin:/gnu/store/x3jx25cd3q363mr7nbgzrhrv1vza6cf7-gcc-7.4.0/bin:/gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/bin:/gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/sbin"
export PWD="/tmp/guix-build-nsis-x86_64-3.04.drv-0/nsis-3.04-src"
export SHLVL="1"
export SOURCE_DATE_EPOCH="1"
export TEMP="/tmp/guix-build-nsis-x86_64-3.04.drv-0"
export TEMPDIR="/tmp/guix-build-nsis-x86_64-3.04.drv-0"
export TMP="/tmp/guix-build-nsis-x86_64-3.04.drv-0"
export TMPDIR="/tmp/guix-build-nsis-x86_64-3.04.drv-0"
export out="/gnu/store/wnjkbggyv1jgc2ld9yrissdsjqjvnim8-nsis-x86_64-3.04"
So, to get a better understanding of what was going on, I went into the
environment, and ran:
$ x86_64-w64-mingw32-g++ -E -v -xc++ - < /dev/null > /dev/null
Which yielded:
Using built-in specs.
COLLECT_GCC=x86_64-w64-mingw32-g++
Target: x86_64-w64-mingw32
Configured with:
Thread model: win32
gcc version 7.4.0 (GCC)
COLLECT_GCC_OPTIONS='-E' '-v' '-shared-libgcc' '-mtune=generic' '-march=x86-64'
/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/libexec/gcc/x86_64-w64-mingw32/7.4.0/cc1plus
-E -quiet -v -U_REENTRANT - -mtune=generic -march=x86-64
ignoring nonexistent directory "/no-gcc-local-prefix/include"
ignoring nonexistent directory
"/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/lib/gcc/x86_64-w64-mingw32/7.4.0/../../../../x86_64-w64-mingw32/include"
ignoring nonexistent directory "/mingw/include"
#include "..." search starts here:
#include <...> search starts here:
/gnu/store/rabx878fijwgg0yl2bfsx0wvw7d7isnj-mingw-w64-x86_64-6.0.0/include
/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/include/c++
/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/include/c++/x86_64-w64-mingw32
/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/include/c++/backward
/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/lib/gcc/x86_64-w64-mingw32/7.4.0/include
/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/lib/gcc/x86_64-w64-mingw32/7.4.0/include-fixed
End of search list.
COMPILER_PATH=/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/libexec/gcc/x86_64-w64-mingw32/7.4.0/:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/libexec/gcc/x86_64-w64-mingw32/7.4.0/:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/libexec/gcc/x86_64-w64-mingw32/:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/lib/gcc/x86_64-w64-mingw32/7.4.0/:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/lib/gcc/x86_64-w64-mingw32/
CROSS_LIBRARY_PATH=/gnu/store/rabx878fijwgg0yl2bfsx0wvw7d7isnj-mingw-w64-x86_64-6.0.0/lib/:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/lib/gcc/x86_64-w64-mingw32/7.4.0/:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/lib/gcc/x86_64-w64-mingw32/7.4.0/../../../../x86_64-w64-mingw32/lib/
COLLECT_GCC_OPTIONS='-E' '-v' '-shared-libgcc' '-mtune=generic' '-march=x86-64'
So if we were to look back at the error from before, it seems that
gcc-cross-x86_64-w64-mingw32-7.4.0/include/c++/cstdlib.h was trying to include
the file stdlib.h in directories listed AFTER it in the list of search paths,
BUT stdlib.h actually resides in mingw-w64-x86_64-6.0.0/include, which is before
it in the effective list of search paths.
Note that the reason mingw-w64-x86_64-6.0.0/include is in this list at all is
because of the 'fix-env phase I added, which plucked it from CPATH and plopped
it into CROSS_CPATH.
I also took a look at Ubuntu's x86_64-w64-mingw32-g++ to see how they do it, and
it would seem that mingw-w64-x86_64-6.0.0/include should be the last thing on
that list:
ignoring nonexistent directory
"/usr/lib/gcc/x86_64-w64-mingw32/7.3-win32/../../../../x86_64-w64-mingw32/sys-include"
#include "..." search starts here:
#include <...> search starts here:
/usr/lib/gcc/x86_64-w64-mingw32/7.3-win32/include/c++
/usr/lib/gcc/x86_64-w64-mingw32/7.3-win32/include/c++/x86_64-w64-mingw32
/usr/lib/gcc/x86_64-w64-mingw32/7.3-win32/include/c++/backward
/usr/lib/gcc/x86_64-w64-mingw32/7.3-win32/include
/usr/lib/gcc/x86_64-w64-mingw32/7.3-win32/include-fixed
/usr/lib/gcc/x86_64-w64-mingw32/7.3-win32/../../../../x86_64-w64-mingw32/include
End of search list.
So now we know what to fix, but how? My naive first thought was to list all of
the search paths in the order I wanted them in CROSS_CPATH, like so (pretend the
newlines didn't exist):
CROSS_CPATH=
/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/include/c++
:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/include/c++/x86_64-w64-mingw32
:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/include/c++/backward
:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/lib/gcc/x86_64-w64-mingw32/7.4.0/include
:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/lib/gcc/x86_64-w64-mingw32/7.4.0/include-fixed
:/gnu/store/rabx878fijwgg0yl2bfsx0wvw7d7isnj-mingw-w64-x86_64-6.0.0/include
However, that doesn't work, what DID work was instead listing the order in
CROSS_CPLUS_INCLUDE_PATH and leaving CROSS_CPATH alone, resulting in something
like:
CROSS_CPLUS_INCLUDE_PATH=
/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/include/c++
:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/include/c++/x86_64-w64-mingw32
:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/include/c++/backward
:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/lib/gcc/x86_64-w64-mingw32/7.4.0/include
:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/lib/gcc/x86_64-w64-mingw32/7.4.0/include-fixed
:/gnu/store/rabx878fijwgg0yl2bfsx0wvw7d7isnj-mingw-w64-x86_64-6.0.0/include
CROSS_CPATH=/gnu/store/rabx878fijwgg0yl2bfsx0wvw7d7isnj-mingw-w64-x86_64-6.0.0/include
Note that I did try unsetting CROSS_CPATH, which did not work either.
Ultimately, this means that the final patch which works is the following:
diff --git a/gnu/packages/installers.scm b/gnu/packages/installers.scm
index c987254d61..794e9a758f 100644
--- a/gnu/packages/installers.scm
+++ b/gnu/packages/installers.scm
@@ -92,7 +92,20 @@
;; CROSS_-prefixed version of env vars
(setenv (string-append "CROSS_" env-name)
(filter-delimited-string env-val
mingw-path?))))
- '("CPLUS_INCLUDE_PATH" "LIBRARY_PATH"
"C_INCLUDE_PATH"))))
+ '("CPATH" "LIBRARY_PATH"))
+ (setenv "CROSS_CPLUS_INCLUDE_PATH"
+ (string-append
+ (string-join
+ (map
+ (lambda (x) (string-append (assoc-ref
%build-inputs "xgcc") x))
+ `("/include/c++"
+ ,(string-append "/include/c++" "/"
,triplet)
+ "/include/c++/backward"
+
"/lib/gcc/x86_64-w64-mingw32/7.4.0/include"
+
"/lib/gcc/x86_64-w64-mingw32/7.4.0/include-fixed"))
+ ":")
+ ":"
+ (getenv "CROSS_CPATH")))))
(add-before 'build 'fix-target-detection
(lambda _
;; NSIS target detection is screwed up, manually
I'd like some feedback on
1. Whether this hack is sane or not
2. Does this apply to other packages suffering from the long-standing
"important" issue #30756
3. Does this reveal something more fundamentally wrong with how we build our
search paths in the first place that should be addressed
4. How I might improve the readability of my final patch
Cheers,
Carl Dong
cont...@carldong.me
"I fight for the users"
