[gentoo-dev] [PATCH 0/3] Remove parallel run support from multilib & multibuild
Hello, everyone. Following a similar change in distutils-r1, I would like to remove the parallel run support in multilib eclasses, and effectively from multibuild completely. The goal is to make things simpler, and follow PMS properly. Right now the parallel runs imply running commands in a subshell. This has three important implications: 1. variable exports and environment changes inside the implicit subshell don't affect the outer scope -- developers are sometimes confused by that, 2. parallel runs can collide if temporary files aren't named uniquely -- think of ffmpeg, waf, 3. 'die' called in a subshell is unsupported by the PMS, and didn't make it into EAPI 6. The side advantages are that we can get rid of multiprocessing inherit and locking in multibuild_merge_root(). Possible issues: if people were relying on commands inside parallel not to affect the outer environment (i.e. doing 'cd' and expecting the next call to start in the initial location, or exporting variables and expecting them to disappear), the ebuilds will fail randomly. However, that's quite unlikely. Your thoughts? -- Best regards, Michał Górny
[gentoo-dev] [PATCH 1/3] multilib-minimal: do not run src_configure() in parallel
---
eclass/multilib-minimal.eclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/eclass/multilib-minimal.eclass b/eclass/multilib-minimal.eclass
index 4e1843e..3f6cfbd 100644
--- a/eclass/multilib-minimal.eclass
+++ b/eclass/multilib-minimal.eclass
@@ -51,7 +51,7 @@ multilib-minimal_src_configure() {
popd >/dev/null || die
}
- multilib_parallel_foreach_abi multilib-minimal_abi_src_configure
+ multilib_foreach_abi multilib-minimal_abi_src_configure
}
multilib-minimal_src_compile() {
--
2.2.0
[gentoo-dev] [PATCH 2/3] multilib-build: unparallelize multilib_parallel_foreach_abi
---
eclass/multilib-build.eclass | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/eclass/multilib-build.eclass b/eclass/multilib-build.eclass
index 2e35d47..b0a4ea1 100644
--- a/eclass/multilib-build.eclass
+++ b/eclass/multilib-build.eclass
@@ -223,18 +223,18 @@ multilib_foreach_abi() {
# @DESCRIPTION:
# If multilib support is enabled, sets the toolchain up for each
# supported ABI along with the ABI variable and correct BUILD_DIR,
-# and runs the given commands with them. The commands are run
-# in parallel with number of jobs being determined from MAKEOPTS.
+# and runs the given commands with them.
#
# If multilib support is disabled, it just runs the commands. No setup
# is done.
#
-# Useful for running configure scripts.
+# This function used to run multiple commands in parallel. Now it's just
+# a deprecated alias to multilib_foreach_abi.
multilib_parallel_foreach_abi() {
debug-print-function ${FUNCNAME} "${@}"
local MULTIBUILD_VARIANTS=( $(multilib_get_enabled_abi_pairs) )
- multibuild_parallel_foreach_variant _multilib_multibuild_wrapper "${@}"
+ multibuild_foreach_variant _multilib_multibuild_wrapper "${@}"
}
# @FUNCTION: multilib_for_best_abi
--
2.2.0
[gentoo-dev] [PATCH 3/3] multibuild: remove parallel run support
---
eclass/multibuild.eclass | 48 +++-
1 file changed, 3 insertions(+), 45 deletions(-)
diff --git a/eclass/multibuild.eclass b/eclass/multibuild.eclass
index 03e6280..bb17421 100644
--- a/eclass/multibuild.eclass
+++ b/eclass/multibuild.eclass
@@ -26,8 +26,6 @@ esac
if [[ ! ${_MULTIBUILD} ]]; then
-inherit multiprocessing
-
# @ECLASS-VARIABLE: MULTIBUILD_VARIANTS
# @DESCRIPTION:
# An array specifying all enabled variants which multibuild_foreach*
@@ -138,8 +136,8 @@ multibuild_foreach_variant() {
# @USAGE: [...]
# @DESCRIPTION:
# Run the passed command repeatedly for each of the enabled package
-# variants alike multibuild_foreach_variant. Multiple invocations of the
command
-# will be performed in parallel, up to MULTIBUILD_JOBS tasks.
+# variants. This used to run the commands in parallel but now it's
+# just a deprecated alias to multibuild_foreach_variant.
#
# The function returns 0 if all commands return 0, or the first non-zero
# exit status otherwise. However, it performs all the invocations
@@ -148,31 +146,7 @@ multibuild_foreach_variant() {
multibuild_parallel_foreach_variant() {
debug-print-function ${FUNCNAME} "${@}"
- local ret lret
-
- _multibuild_parallel() {
- (
- multijob_child_init
- "${@}"
- ) &
- multijob_post_fork
- }
-
- local opts
- if [[ ${MULTIBUILD_JOBS} ]]; then
- opts=-j${MULTIBUILD_JOBS}
- else
- opts=${MAKEOPTS}
- fi
-
- multijob_init "${opts}"
- multibuild_foreach_variant _multibuild_parallel "${@}"
- ret=${?}
- multijob_finish
- lret=${?}
-
- [[ ${ret} -eq 0 ]] && ret=${lret}
- return ${ret}
+ multibuild_foreach_variant "${@}"
}
# @FUNCTION: multibuild_for_best_variant
@@ -252,25 +226,12 @@ run_in_build_dir() {
# Merge the directory tree (fake root) from to
# (the real root). Both directories have to be real, absolute paths
# (i.e. including ${D}). Source root will be removed.
-#
-# This functions uses locking to support merging during parallel
-# installs.
multibuild_merge_root() {
local src=${1}
local dest=${2}
- local lockfile=${T}/.multibuild_merge_lock
- local lockfile_l=${lockfile}.${BASHPID}
local ret
- # Lock the install tree for merge. The touch+ln method ensures race
- # condition-free locking with maximum portability.
- touch "${lockfile_l}" || die
- until ln "${lockfile_l}" "${lockfile}" &>/dev/null; do
- sleep 1
- done
- rm "${lockfile_l}" || die
-
if use userland_BSD; then
# Most of BSD variants fail to copy broken symlinks, #447370
# also, they do not support --version
@@ -297,9 +258,6 @@ multibuild_merge_root() {
ret=${?}
fi
- # Remove the lock.
- rm "${lockfile}" || die
-
if [[ ${ret} -ne 0 ]]; then
die "${MULTIBUILD_VARIANT:-(unknown)}: merging image failed."
fi
--
2.2.0
Re: [gentoo-dev] [PATCH 0/3] Remove parallel run support from multilib & multibuild
On Thu, 11 Dec 2014 11:36:29 +0100 Michał Górny wrote: > Hello, everyone. > > Following a similar change in distutils-r1, I would like to remove > the parallel run support in multilib eclasses, and effectively from > multibuild completely. [...] > Your thoughts? +1 I hate parallel builds as I can't follow build execution properly; and, on a different level, there's emerge --jobs already. Alexis.
Re: [gentoo-dev] Re: metadata.xml un-ization, v2
On Thu, Dec 11, 2014 at 1:23 AM, Michael Palimaka wrote: > > This would be by far the easiest solution. Some herds already have an > alias like this eg. freedesktop -> freedesktop-bugs. Much easier than > mass-editing every single metadata.xml with what amounts to a cosmetic > change. > I think we're not all on the same page about what we're trying to accomplish. I think we need to agree on that before we can really agree on how to do it. There are many goals here that I can see: 1. Simplification of the xml schema so that we don't have so many different kinds of tags with different rules for each. 2. Simplification of how we track group (ie project/herd/alias/etc) member lists so that they're not in 5 different places with 5 different ways of determining who is in a group. 3. Avoiding having large groups of packages maintained by large groups of devs where the reality is that many packages aren't maintained by anybody but this is opaque. 4. If not all of the emails associated with metadata are considered true maintainers, making it easy to tell which ones are and aren't. I don't think all of these have equal support, which is why we end up debating different solutions (obviously a solution which addresses all of these is going to be more intrusive than a solution that only addresses some of these). I've been a proponent of solving all of these, but perhaps it would make more sense to start smaller than that. The only catch is that if we remove the distinction in metadata between maintainers/proxies/projects/herds/etc then if that distinction becomes more important in the future it becomes harder to tell which ones are which. Part of me is wondering if worrying about #3-4 is actually all that productive. Does it really matter if a package is maintained or not, when it all comes down to it? Does it make more sense to focus on whether packages have serious problems? Maybe if a package is completely unmaintained it makes it easier for developers to drop in and make changes without asking anybody about it first, versus logging a bug, waiting for the maintainer to drop the ball, and then making the change anyway. -- Rich
Re: [gentoo-dev] Lastrites: net-im/linpopup, app-office/teapot, net-irc/bitchx, sys-power/cpufrequtils, x11-plugins/gkrellm-cpufreq, media-sound/gnome-alsamixer, sys-devel/ac-archive, net-misc/emirror
On Tue, 09 Dec 2014 11:05:31 +0100
Pacho Ramos wrote:
>
> Regarding the gkrellm-plugins, looks like Fedora is supplying this
> one:
> http://pkgs.fedoraproject.org/cgit/gkrellm-freq.git/tree/gkrellm-freq.spec
>
>
I rewrote ebuild for gkrellm-gkfreq plugin a bit
https://github.com/cerebrum/dr/blob/master/x11-plugins/gkrellm-gkfreq/gkrellm-gkfreq-2.3.ebuild
And sent this patch upstream
https://github.com/cerebrum/dr/blob/master/x11-plugins/gkrellm-gkfreq/files/gkrellm-gkfreq-2.3-make.patch
https://sourceforge.net/p/gkrellm-gkfreq/feature-requests/2/
Why do we have a function tc-getCC for CC environment variable and why
we don't use CC environment variable automatically? With this patch
CFLAGS and LDFLAGS are substituted automatically without garbage in
ebuild. Why don't we use the same strategy for CC variable?
"Sometimes a package will try to use a bizarre compiler, or will need
to be told which compiler to use. In these situations, the tc-getCC()
function from toolchain-funcs.eclass should be used...
Note: It is not correct to use the ${CC} variable for this purpose."
https://devmanual.gentoo.org/ebuild-writing/functions/src_compile/building/index.html
And why CC variable gives me "cc" but not "x86_64-pc-linux-gnu-gcc" as
it should be? Am I right that it's a bit strange?
Re: [gentoo-dev] sys-devel/gcc::mgorny up for testing
On 12/09/2014 11:21, Michał Górny wrote: > Dnia 2014-12-08, o godz. 09:56:11 > Alexis Ballier napisał(a): > >> On Sun, 7 Dec 2014 11:37:57 +0100 >> Michał Górny wrote: >> >>> 1. No cross-compilation support. If the project proves being a success >>> it will be readded at some point. However, I will likely fork glibc >>> first and work on a sane crossdev alternative. >> >> Could you please elaborate on this ? >> How and why this broke, what is wrong in glibc, what would be a "sane >> crossdev alternative" and how crossdev is not. > > 1. eblits are broken in glibc. Another case of incorrect interpretation > of code sharing with love for breaking stable ebuilds. Plus I'd love to > add multilib flags to it but we'll likely doing that to main glibc as > well soon, if eblits don't get into our way. Originally, eblits were supposed to be ways of sharing common blocks of code between ebuilds in a specific package, but not anything that would warrant a new eclass (local package eclasses, kinda?). They never really took off for reasons lost to time. I use them in mips-sources, but version them so that if I change an eblit, it gets a new version and the old version is removed only when there are no more consumers. The idea of eblits itself isn't bad, but probably could use some re-thinking and/or re-implementation. Especially if a decent amount of kilobytes can be shaved off a given package's directory. > 2. crossdev is broken because it creates semi-random, unmaintained > ebuilds in randomly chosen overlay, those ebuilds relying on > toolchain.eclass behavior. I would argue this isn't necessarily "broken". crossdev produces working cross-toolchains and is a valuable tool for anyone running multiple arches, especially slower arches. How it produces said working cross-toolchains is an implementation detail. Don't be so hasty as to throw the baby out with the bathwater. -- Joshua Kinard Gentoo/MIPS ku...@gentoo.org 4096R/D25D95E3 2011-03-28 "The past tempts us, the present confuses us, the future frightens us. And our lives slip away, moment by moment, lost in that vast, terrible in-between." --Emperor Turhan, Centauri Republic
