Hello, This is the second and hopefully last version of python-r1 + distutils-r1 eclasses before committing. I would also like to shortly point out the goals and the differences between various python eclasses in Gentoo.
Both are attached. For those who prefer hosted form: https://bitbucket.org/mgorny/gx86-working-tree/src/5a2d39e69d6d/gx86/eclass/python-r1.eclass https://bitbucket.org/mgorny/gx86-working-tree/src/5a2d39e69d6d/gx86/eclass/distutils-r1.eclass Changes from the previous version: - support for DOCS and HTML_DOCS (alike EAPI 4); - minimal support for passing arguments to setup.py (through distutils-r1_python_compile & distutils-r1_python_install); - EPYTHON values adjusted to match python.eclass; - scripts are -${EPYTHON} suffixed -- i.e. foo-python2.7, foo-pypy-c1.8; - a wrapper script[1] is used to choose the correct script instead of constant symlink. Thus, EPYTHON & eselect-python are respected, - PYTHON_COMPAT can be an array only. What still needs considering: - a nicer way of passing setup.py arguments (mysetuppyargs=()?, esetuppy wrapper?); - Prefix support. [1]:https://bitbucket.org/mgorny/python-exec Now, for the differences part. Goals ----- python.eclass aims to support every single Python package out there, including rare corner cases supported in-eclass. It supports both packages supporting one and multiple Python implementations. distutils support is provided by second eclass, and both provide phase functions (python.eclass providing default ph.f. wrappers). python-distutils-ng aims to support majority of Python packages using distutils. With a bit of hackery it can support non-distutils packages, but the use is limited to the common cases. In an uncommon case, it's not flexible enough. It supports Python packages supporting multiple implementations only. python-r1 aims mostly to provide tools to support majority of Python packages. It tries to be simple & flexible, thus allowing handling various corner cases without special branches of code in the eclass. It doesn't export any phase functions, nor set dependencies by default (just provides a variable with them). Right now, it supports packages supporting multiple implementations only; if necessary, the support will be extended -- either through additional code if that wouldn't add too much complexity, or through additional eclass. It is accompanied by distutils-r1 which provides a simple overlay for the very common case of distutils packages. This eclass exports phase functions and sets dependencies implicitly. It also handles renaming the distutils-installed scripts and linking the python-exec wrapper. Python targets -------------- python.eclass uses implicit Python target specifications. It provides no ebuild-transparent way of enabling/disabling them. python-distutils-ng and python-r1 use USE flags to explicitly list enabled Python implementations. Both eclasses use the same values for PYTHON_TARGETS USE_EXPAND. Installed scripts ----------------- python-distutils-ng rework the installed scripts creating copies for Python implementations and changing the shebang. The created copies don't differ any other way. The old script name is then symlinked to the version for default implementation. python.eclass & python-r1 instead let distutils rework the scripts and just rename them before merging. A wrapper is used to choose the correct version, respecting EPYTHON & eselect-python. python.eclass installs the files to separate installation images and merges them. python-r1 installs them to the main image directly, renaming the installed scripts between successive implementations. python.eclass creates a wrapper script for each package. The script is written in python. python-r1 instead installs the compiled wrapper once (in an ebuild), and symlinks it for the packages. The wrapper is written in much simpler C. The implementation suffixes for all three eclasses are different: - python.eclass uses -2.6, -2.7 for Python, -2.7-pypy-1.8, -2.7-pypy-1.9 for pypy and -2.5-jython for jython (enjoy!); - p-d-ng uses Python target names (-python2_6, -python2_7, -pypy1_8), - p-r1 uses $EPYTHON values (-python2.6, -python2.7, -pypy-c1.8 (sic!)). Dependency on Python -------------------- python.eclass has a few variables necessary to set dependencies on Python implementation, including internal sub-syntax (and an even more complete generator sub-syntax in progress overlay). I suspect that most of the possible variants can be achieved using it, just please don't make me try learning it all. Additionally, packages supporting multiple Python implementations are required to specify their supported Python versions twice -- with PYTHON_DEPEND and RESTRICT_PYTHON_ABIS. USE flag dependencies are specified using three variables, and I'd like to avoid getting anywhere deeper. p-d-ng has a very simple dependency setup. It has a variable for listing supported Python versions and a variable to make Python dependencies conditional under the 'python' flag. Lately, it has been added a method of listing requested USE flags. p-d-ng does not provide any support for more complex dependency specifications, nor a way to disable adding default Python dependencies. python-r1 does not append any dependencies by default, and instead exports them in a variable. The variable can be used to easily express simple cases, and avoids polluting the ebuild in more complex cases. It also supports providing USE dependencies for the implementation. distutils-r1 adds the Python dependencies to RDEPEND & DEPEND; additionally, it adds a dependency on python-exec. Dependencies on other packages ------------------------------ As python.eclass (as of gx86 state) does not express enabled implementations explicitly, it is not possible to request those implementations from other packages. The dependencies on other Python packages are thus simple. p-d-ng does not provide any simpler way of writing dependencies on other Python packages using the eclass. Thus, the necessary USE dependencies have to be written by hand. python-r1 does provide a simple PYTHON_USEDEP variable which can be used in dependency atoms to request the same Python implementations being enabled. Phase functions for distutils ----------------------------- distutils.eclass allows specifying additional arguments to setup.py. src_install() installs default documentation files (*unconditionally*) and ${DOCS}, it also does installation image magic. p-d-ng has mixed unconditional and user-overridable stuff. Copying sources is done unconditionally; redoing scripts can be disabled using a separate variable. setup.py invocations are overridable. d-r1 splits each phase into 'global' and 'per-implementation' stages which are both overridable. The default stages apply PATCHES (and user patches), copy sources, run setup.py, install DOCS & HTML_DOCS and link python-exec wrappers. Thank you for reading up to this. I'd appreciate any comments, ideas, and especially any API suggestions while the eclass still ain't live. -- Best regards, Michał Górny
# Copyright 1999-2012 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $
# @ECLASS: python-r1
# @MAINTAINER:
# Python herd <pyt...@gentoo.org>
# @AUTHOR:
# Author: MichaŠGórny <mgo...@gentoo.org>
# Based on work of: Krzysztof Pawlik <nelch...@gentoo.org>
# @BLURB: A common, simple eclass for Python packages.
# @DESCRIPTION:
# A common eclass providing helper functions to build and install
# packages supporting being installed for multiple Python
# implementations.
#
# This eclass sets correct IUSE and REQUIRED_USE. It exports PYTHON_DEPS
# and PYTHON_USEDEP so you can create correct dependencies for your
# package easily. It also provides methods to easily run a command for
# each enabled Python implementation and duplicate the sources for them.
case "${EAPI}" in
0|1|2|3)
die "Unsupported EAPI=${EAPI} (too old) for ${ECLASS}"
;;
4)
# EAPI=4 needed for REQUIRED_USE
;;
*)
die "Unsupported EAPI=${EAPI} (unknown) for ${ECLASS}"
;;
esac
# @ECLASS-VARIABLE: _PYTHON_ALL_IMPLS
# @INTERNAL
# @DESCRIPTION:
# All supported Python implementations, most preferred last.
_PYTHON_ALL_IMPLS=(
jython2_5
pypy1_8 pypy1_9
python3_1 python3_2
python2_5 python2_6 python2_7
)
# @ECLASS-VARIABLE: PYTHON_COMPAT
# @DESCRIPTION:
# This variable contains a list of Python implementations the package
# supports. It must be set before the `inherit' call. The default is to
# enable all implementations. It has to be an array.
: ${PYTHON_COMPAT:=${_PYTHON_ALL_IMPLS[@]}}
# @ECLASS-VARIABLE: PYTHON_REQ_USE
# @DEFAULT_UNSET
# @DESCRIPTION:
# The list of USEflags required to be enabled on the chosen Python
# implementations, formed as a USE-dependency string. It should be valid
# for all implementations in PYTHON_COMPAT, so it may be necessary to
# use USE defaults.
#
# Example:
# @CODE
# PYTHON_REQ_USE="gdbm,ncurses(-)?"
# @CODE
#
# Will cause the Python dependencies to look like:
# @CODE
# python_targets_pythonX_Y? (
# dev-lang/python:X_Y[gdbm,ncurses(-)?] )
# @CODE
# @ECLASS-VARIABLE: PYTHON_DEPS
# @DESCRIPTION:
# This is an eclass-generated Python dependency string for all
# implementations listed in PYTHON_COMPAT. It should be used
# in RDEPEND and/or DEPEND like:
#
# @CODE
# RDEPEND="${PYTHON_DEPS}
# dev-foo/mydep"
# DEPEND="${RDEPEND}"
# @CODE
# @ECLASS-VARIABLE: PYTHON_USEDEP
# @DESCRIPTION:
# This is an eclass-generated USE-dependency string which can be used to
# depend on another Python package being built for the same Python
# implementations. It should be used like:
#
# @CODE
# RDEPEND="dev-python/foo[${PYTHON_USEDEP}]"
# @CODE
_python_set_globals() {
local flags=( "${PYTHON_COMPAT[@]/#/python_targets_}" )
local optflags=${flags[@]/%/?}
IUSE=${flags[*]}
REQUIRED_USE="|| ( ${flags[*]} )"
PYTHON_USEDEP=${optflags// /,}
PYTHON_DEPS=
local i
for i in ${PYTHON_COMPAT[@]}; do
local d
case ${i} in
python*)
d='dev-lang/python';;
jython*)
d='dev-java/jython';;
pypy*)
d='dev-python/pypy';;
*)
die "Invalid implementation: ${i}"
esac
local v=${i##*[a-z]}
local usestr
[[ ${PYTHON_REQ_USE} ]] && usestr="[${PYTHON_REQ_USE}]"
PYTHON_DEPS+=" python_targets_${i}? (
${d}:${v/_/.}${usestr} )"
done
}
_python_set_globals
# @FUNCTION: python_get_PYTHON
# @USAGE: <impl>
# @DESCRIPTION:
# Get the Python executable name for the given implementation. Please
# note that this this function returns the 'basename' only.
# If using it for a hashbang, please use #!/usr/bin/env.
python_get_PYTHON() {
debug-print-function ${FUNCNAME} "${@}"
local impl=${1/_/.}
local ret
case "${impl}" in
python*|jython*)
ret=${impl}
;;
pypy*)
ret=pypy-c${impl#pypy}
;;
*)
die "Invalid argument to python_get_PYTHON: ${1}"
;;
esac
debug-print "${FUNCNAME}: ${impl} -> ${ret}"
echo "${ret}"
}
# @FUNCTION: python_copy_sources
# @DESCRIPTION:
# Create a single copy of the package sources (${S}) for each enabled
# Python implementation.
python_copy_sources() {
debug-print-function ${FUNCNAME} "${@}"
local impl
local bdir=${BUILD_DIR:-${S}}
debug-print "${FUNCNAME}: bdir = ${bdir}"
einfo "Will copy sources from ${S}"
# the order is irrelevant here
for impl in ${PYTHON_COMPAT[@]}; do
if use python_targets_${impl}
then
local BUILD_DIR=${bdir%%/}-${impl}
einfo "${impl}: copying to ${BUILD_DIR}"
debug-print "${FUNCNAME}: [${impl}] cp ${S} =>
${BUILD_DIR}"
cp -pr "${S}" "${BUILD_DIR}" || die
fi
done
}
# @FUNCTION: python_foreach_impl
# @USAGE: <command> [<args>...]
# @DESCRIPTION:
# Run the given command for each of the enabled Python implementations.
# If additional parameters are passed, they will be passed through
# to the command. If the command fails, python_foreach_impl dies.
# If necessary, use ':' to force a successful return.
#
# Before the command is run, EPYTHON is set to the name of the current
# Python implementation, PYTHON is set to the correct Python executable
# name and exported, and BUILD_DIR is set to a 'default' build directory
# for given implementation (e.g. ${BUILD_DIR:-${S}}-python2_7).
#
# The command is run inside the build directory. If it doesn't exist
# yet, it is created (as an empty directory!). If your build system does
# not support out-of-source builds, you will likely want to use
# python_copy_sources first.
python_foreach_impl() {
debug-print-function ${FUNCNAME} "${@}"
local impl
local bdir=${BUILD_DIR:-${S}}
debug-print "${FUNCNAME}: bdir = ${bdir}"
for impl in ${_PYTHON_ALL_IMPLS[@]}; do
if has ${impl} ${PYTHON_COMPAT[@]} && use python_targets_${impl}
then
local PYTHON=$(python_get_PYTHON "${impl}")
local EPYTHON=${PYTHON}
local BUILD_DIR=${bdir%%/}-${impl}
export PYTHON
debug-print "${FUNCNAME}: [${impl}] build_dir =
${BUILD_DIR}"
mkdir -p "${BUILD_DIR}" || die
pushd "${BUILD_DIR}" &>/dev/null || die
einfo "${EPYTHON}: running ${@}"
"${@}" || die "${EPYTHON}: ${1} failed"
popd &>/dev/null || die
fi
done
}
# Copyright 1999-2012 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $
# @ECLASS: distutils-r1
# @MAINTAINER:
# Python herd <pyt...@gentoo.org>
# @AUTHOR:
# Author: MichaŠGórny <mgo...@gentoo.org>
# Based on the work of: Krzysztof Pawlik <nelch...@gentoo.org>
# @BLURB: A simple eclass to build Python packages using distutils.
# @DESCRIPTION:
# A simple eclass providing functions to build Python packages using
# the distutils build system. It exports phase functions for all
# the src_* phases. Each of the phases runs two pseudo-phases:
# python_..._all() (e.g. python_prepare_all()) once in ${S}, then
# python_...() (e.g. python_prepare()) for each implementation
# (see: python_foreach_impl() in python-r1).
#
# In distutils-r1_src_prepare(), the 'all' function is run before
# per-implementation ones (because it creates the implementations),
# per-implementation functions are run in a random order.
#
# In remaining phase functions, the per-implementation functions are run
# before the 'all' one, and they are ordered from the least to the most
# preferred implementation (so that 'better' files overwrite 'worse'
# ones).
#
# If the ebuild doesn't specify a particular pseudo-phase function,
# the default one will be used (distutils-r1_...). Defaults are provided
# for all per-implementation pseudo-phases, python_prepare_all()
# and python_install_all(); whenever writing your own pseudo-phase
# functions, you should consider calling the defaults (and especially
# distutils-r1_python_prepare_all).
#
# Please note that distutils-r1 sets RDEPEND and DEPEND for you.
case "${EAPI}" in
0|1|2|3)
die "Unsupported EAPI=${EAPI} (too old) for ${ECLASS}"
;;
4)
;;
*)
die "Unsupported EAPI=${EAPI} (unknown) for ${ECLASS}"
;;
esac
inherit eutils python-r1
EXPORT_FUNCTIONS src_prepare src_configure src_compile src_test src_install
RDEPEND="${PYTHON_DEPS}
dev-python/python-exec"
DEPEND=${PYTHON_DEPS}
# @ECLASS-VARIABLE: DOCS
# @DEFAULT_UNSET
# @DESCRIPTION:
# Array containing documents installed using dodoc.
#
# If unset, the default filename list (from PMS) will be used.
#
# Example:
# @CODE
# DOCS=( NEWS README )
# @CODE
# @ECLASS-VARIABLE: HTML_DOCS
# @DEFAULT_UNSET
# @DESCRIPTION:
# Array containing documents installed using dohtml.
#
# Example:
# @CODE
# HTML_DOCS=( doc/html/ )
# @CODE
# @FUNCTION: distutils-r1_python_prepare_all
# @DESCRIPTION:
# The default python_prepare_all(). It applies the patches from PATCHES
# array, then user patches and finally calls python_copy_sources to
# create copies of resulting sources for each Python implementation.
distutils-r1_python_prepare_all() {
debug-print-function ${FUNCNAME} "${@}"
[[ ${PATCHES} ]] && epatch "${PATCHES[@]}"
epatch_user
# create source copies for each implementation
python_copy_sources
}
# @FUNCTION: distutils-r1_python_prepare
# @DESCRIPTION:
# The default python_prepare(). Currently it is a no-op
# but in the future it may apply implementation-specific quirks
# to the build system.
distutils-r1_python_prepare() {
debug-print-function ${FUNCNAME} "${@}"
:
}
# @FUNCTION: distutils-r1_python_configure
# @DESCRIPTION:
# The default python_configure(). Currently a no-op.
distutils-r1_python_configure() {
debug-print-function ${FUNCNAME} "${@}"
:
}
# @FUNCTION: distutils-r1_python_compile
# @USAGE: [additional-args...]
# @DESCRIPTION:
# The default python_compile(). Runs 'setup.py build' using the correct
# Python implementation. Any parameters passed to this function will be
# passed to setup.py.
distutils-r1_python_compile() {
debug-print-function ${FUNCNAME} "${@}"
cd "${BUILD_DIR}" || die
set -- "${PYTHON}" setup.py build "${@}"
echo "${@}"
"${@}" || die
}
# @FUNCTION: distutils-r1_python_test
# @DESCRIPTION:
# The default python_test(). Currently a no-op.
distutils-r1_python_test() {
debug-print-function ${FUNCNAME} "${@}"
:
}
# @FUNCTION: distutils-r1_rename_scripts
# @DESCRIPTION:
# Renames installed Python scripts to be implementation-suffixed.
# ${PYTHON} has to be set to the expected Python executable (which
# hashbang will be grepped for), and ${EPYTHON} to the implementation
# name (for new name).
distutils-r1_rename_scripts() {
debug-print-function ${FUNCNAME} "${@}"
local f
# XXX: change this if we ever allow directories in bin/sbin
for f in "${D}"/{bin,sbin,usr/bin,usr/sbin}/*; do
if [[ -x ${f} ]]; then
debug-print "${FUNCNAME}: found executable at
${f#${D}/}"
if [[ "$(head -n 1 "${f}")" == '#!'*${PYTHON}* ]]
then
debug-print "${FUNCNAME}: matching shebang:
$(head -n 1 "${f}")"
local newf=${f}-${EPYTHON}
debug-print "${FUNCNAME}: renamed to
${newf#${D}/}"
mv "${f}" "${newf}" || die
fi
fi
done
}
# @FUNCTION: distutils-r1_python_install
# @USAGE: [additional-args...]
# @DESCRIPTION:
# The default python_install(). Runs 'setup.py install' using
# the correct Python implementation, and appending the optimization
# flags. Then calls distutils-r1_rename_scripts. Any parameters passed
# to this function will be passed to setup.py.
distutils-r1_python_install() {
debug-print-function ${FUNCNAME} "${@}"
local flags
case "${EPYTHON}" in
jython*)
flags='--compile';;
*)
flags='--compile -O2';;
esac
debug-print "${FUNCNAME}: [${EPYTHON}] flags: ${flags}"
unset PYTHONDONTWRITEBYTECODE
cd "${BUILD_DIR}" || die
set -- "${PYTHON}" setup.py install ${flags} --root="${D}" "${@}"
echo "${@}"
"${@}" || die
distutils-r1_rename_scripts
}
# @FUNCTION: distutils-r1_python_install_all
# @DESCRIPTION:
# The default python_install_all(). It symlinks wrappers
# for the implementation-suffixed executables and installs
# documentation.
distutils-r1_python_install_all() {
debug-print-function ${FUNCNAME} "${@}"
if declare -p DOCS &>/dev/null; then
dodoc "${DOCS[@]}" || die "dodoc failed"
else
local f
# same list as in PMS
for f in README* ChangeLog AUTHORS NEWS TODO CHANGES \
THANKS BUGS FAQ CREDITS CHANGELOG; do
if [[ -s ${f} ]]; then
dodoc "${f}" || die "(default) dodoc ${f}
failed"
fi
done
fi
if declare -p HTML_DOCS &>/dev/null; then
dohtml -r "${HTML_DOCS[@]}" || die "dohtml failed"
fi
# note: keep in sync with ...rename_scripts()
# also, we assume that each script is installed for all impls
local any_impl=$(python_get_PYTHON ${PYTHON_COMPAT[0]})
for f in "${D}"/{bin,sbin,usr/bin,usr/sbin}/*-${any_impl}; do
if [[ -x ${f} ]]; then
debug-print "${FUNCNAME}: found executable at
${f#${D}/}"
local wrapf=${f%-${any_impl}}
debug-print "${FUNCNAME}: will link wrapper to
${wrapf#${D}/}"
local wrapfrom
case "${f#${D}/}" in
usr/bin*)
wrapfrom=
;;
usr/sbin*)
wrapfrom=../bin/
;;
*)
wrapfrom=../usr/bin/
;;
esac
debug-print "${FUNCNAME}: (from ${wrapfrom}python-exec)"
ln -s ${wrapfrom}python-exec "${wrapf}" || die
fi
done
}
distutils-r1_src_prepare() {
debug-print-function ${FUNCNAME} "${@}"
# common preparations
if declare -f python_prepare_all >/dev/null; then
python_prepare_all
else
distutils-r1_python_prepare_all
fi
if declare -f python_prepare >/dev/null; then
python_foreach_impl python_prepare
else
distutils-r1_python_prepare
fi
}
distutils-r1_src_configure() {
if declare -f python_configure >/dev/null; then
python_foreach_impl python_configure
else
distutils-r1_python_configure
fi
if declare -f python_configure_all >/dev/null; then
python_configure_all
fi
}
distutils-r1_src_compile() {
debug-print-function ${FUNCNAME} "${@}"
if declare -f python_compile >/dev/null; then
python_foreach_impl python_compile
else
python_foreach_impl distutils-r1_python_compile
fi
if declare -f python_compile_all >/dev/null; then
python_compile_all
fi
}
distutils-r1_src_test() {
debug-print-function ${FUNCNAME} "${@}"
if declare -f python_test >/dev/null; then
python_foreach_impl python_test
else
distutils-r1_python_test
fi
if declare -f python_test_all >/dev/null; then
python_test_all
fi
}
distutils-r1_src_install() {
debug-print-function ${FUNCNAME} "${@}"
if declare -f python_install >/dev/null; then
python_foreach_impl python_install
else
python_foreach_impl distutils-r1_python_install
fi
if declare -f python_install_all >/dev/null; then
python_install_all
else
distutils-r1_python_install_all
fi
}
signature.asc
Description: PGP signature
