On Tue, 10 Oct 2023 at 16:59, David Pierret <david.pier...@smile.fr> wrote: > We've chosen to use a template configuration file rather than multiple > configuration files (1 per meta). The template file contains tags > between braces that is replaced in the script using simple `sed` calls. > The meta list to be analyzed is defined via the `meta_list` variable. > This variable matches all `meta-*` directories
I would suggest that AUH itself is modified to accept the dynamic layer_* parts as command line arguments. Tweaking things with sed is difficult to understand or maintain, and can be easily avoided here. Then the other commit for autobuilder-helper can as well be greatly simplified and can reuse existing scripts instead of copy-paste-tweaking them. Alex > Signed-off-by: David Pierret <david.pier...@smile.fr> > Reviewed-by: Yoann Congal <yoann.con...@smile.fr> > --- > upgrade-helper.in | 124 ++++++++++++++++++++++++++++++++++++++++++++++ > weeklyjob-oe.sh | 48 ++++++++++++++++++ > 2 files changed, 172 insertions(+) > create mode 100644 upgrade-helper.in > create mode 100644 weeklyjob-oe.sh > > diff --git a/upgrade-helper.in b/upgrade-helper.in > new file mode 100644 > index 0000000..4eedef1 > --- /dev/null > +++ b/upgrade-helper.in > @@ -0,0 +1,124 @@ > +# > +# This file is your upgrade-helper configuration files and is where all > local user > +# settings specifically for AUH are placed. The comments in this file > provide some > +# guidelines to the options a new user to AUH tool might want to change. > +# > +# This file should be placed inside your BUILDDIR/upgrade-helper after > initializing > +# the OE build environment (this location can be overriden with -c command > line option). > +# > +# Lines starting with the '#' character are commented out and in some cases > the > +# default values are provided as comments to show people example syntax. > Enabling > +# the option is a question of removing the # character and making any change > to the > +# variable as required. For boolean settings, anything not 'yes' is treated > as 'no'. > + > +[maintainer_override] > +# e-mail message for recipe upgrades will go to john.doe instead of > jane.doe, etc > +# see also the global_maintainer_override option > +#jane....@doe.com=john....@doe.com > +#johhny.br...@bravo.com=john....@doe.com > + > +[settings] > +# SMTP server that is used to send e-mails with patches and other > information. > +# If you are running AUH locally, you do not need to set this up, as AUH > +# saves everything to BUILDDIR/upgrade-helper/<timestamp>, and does not > attempt > +# to send email messages (unless explicitly asked with -e command line > option). > +# If no port is specified, port 25 is assumed. > +#smtp=smtp.my-server.com:25 > + > +# from whom should the e-mails be sent (mandatory if --send-emails is > passed). > +# Also sets the email address of the author of automated commits. > +#from=u...@not.set > + > +# If enabled, emails for all recipe upgrades will go to john.doe, > +# except when recipes are owned by specific maintainer_override entries > above. > +#global_maintainer_override=john....@doe.com > + > +# who should be CCd with all upgrade emails (optional) > +#cc_recipients=john....@doe.com > + > +# who should get the status mail with statistics, at the end (mandatory if > +# --send-emails is passed) > +#status_recipients=john....@doe.com > + > +# Only recipes belonging to maintainers in whitelist will be attempted > +# Insert your own e-mail address here together with any other maintainers' > addresses > +# whose recipes you wish to take care of. This restricts the set of recipes > that AUH > +# will attempt when it is run with 'all' option. > +#maintainers_whitelist=jane....@doe.com john....@doe.com > johhny.br...@bravo.com > + > +# recipes in blacklist will be skipped (applies only when 'all' or no recipe > is > +# passed; does not apply when layer_mode is enabled). > +#blacklist=python glibc gcc > + > +# specify the directory where work (patches) will be saved > +# (optional; default is BUILDDIR/upgrade-helper/) > +#workdir= > + > +# public url with AUH results to include in statistics summary (optional) > +#publish_work_url=http://auh.somehost.com/work > + > +# whether to attach a tarball with AUH artefacts (patches, log, > buildhistory, etc.) > +# to the summary email > +#summary_includes_tarball=True > + > +# clean sstate directory before upgrading > +# Generally not necessary, as bitbake can handle this automatically. > +#clean_sstate=no > + > +# clean tmp directory before upgrading > +# Generally not necessary as bitbake can handle this automatically. > +#clean_tmp=no > + > +# Machines to test build with. > +# Append _libc-name to test with alternative C library implementations > +# e.g. qemux86_musl. > +# > +# Buildhistory and testimages will be created only for the first > +# machine in the list, as otherwise it adds enormously to AUH run time. > +# > +# AUH has a reasonable default for this, so you do not need to set your own, > +# at least initially. > +# > +# Does not apply when layer_mode is enabled. > +#machines=qemux86 qemux86-64 qemuarm qemumips qemuppc qemux86_musl > + > +# Enables buildhistory feature; this is useful as it produces information > +# about what has changed in the resulting packages, compared to previous > version > +# > +# Requires 'buildhistory' to be present in INHERIT and BUILDHISTORY_COMMIT > to be set > +# in your conf/local.conf. > +#buildhistory=no > + > +# When AUH has built an upgraded recipe it then creates a commit with the > upgrade. > +# This setting specifies whether to also revert the commit. Possible values > are: > +# 'failed_to_build' (default) - revert if the recipe failed to build. This > is useful > +# to prevent further upgrades in the batch from failing, if they're > dependent on the > +# recipe. > +# 'all' - always revert. This is useful when running automatic, unattended > upgrades, > +# as it will effectively test each upgrade against the baseline, and will > avoid the > +# situation where one successful upgrade thwarts others due to newly > introduced > +# incompatibilities. > +# 'never' - never revert. Use in interactive upgrade sessions, where any > issues can be > +# manually fixed. > +#commit_revert_policy=failed_to_build > + > +# If enabled, build and boots a test image, and runs integration tests on it > +# If upgraded packages have ptest support those are run as well > +# > +# Requires 'testimage' in INHERIT in your conf/local.conf and 'ptest' in your > +# distro's DISTRO_FEATURES. > +#testimage=no > + > +# This can be used to change the name of the test image. > +# > +#testimage_name=core-image-sato > + > +# This can be used to upgrade recipes in a specific layer, > +# for example meta-intel, instead of upgrading oe-core recipes. > +# > +# When layer_mode is enabled, layer_name, layer_dir and layer_machines are > +# mandatory. 'blacklist' setting does not apply when in layer_mode. > +layer_mode=yes > +layer_name={meta} > +layer_dir={layerdir} > +layer_machines={machine} > diff --git a/weeklyjob-oe.sh b/weeklyjob-oe.sh > new file mode 100644 > index 0000000..1bad6da > --- /dev/null > +++ b/weeklyjob-oe.sh > @@ -0,0 +1,48 @@ > +#!/bin/bash > + > +# Cronjob which can be run weekly to run the upgrade helper script. > +# Add the job in /etc/crontab like below. > +# > +# It will execute weekly at the same hour (8 AM). > +# > +# 00 8 * * 6 auh /home/auh/bin/weeklyjob-oe.sh > + > +# Re-assign these to match your setup! > +auh_dir=~/auto-upgrade-helper > +poky_dir=~/poky > +openembedded_dir=~/meta-openembedded > +build_dir=~/build-tmp-auh-upgrades > +sstate_dir=~/sstate-cache > + > +meta_list="${openembedded_dir}/meta-*" > +machine_list="qemux86 qemux86-64 qemuarm qemumips qemuppc qemux86_musl" > + > +pushd $openembedded_dir || exit 1 > + > +source $poky_dir/oe-init-build-env $build_dir > + > +# Base the upgrades on meta_openembedded master > +git fetch origin > +git checkout -B tmp-auh-upgrades origin/main > +# upgrade helper need to use ?= for MACHINE > +sed -i.bak -E 's|^(MACHINE[[:space:]]*)[?+=]+([[:space:]]*.*)$|\1?=\2|' > $build_dir/conf/local.conf > + > +mkdir $build_dir/upgrade-helper/ > + > +for d in $meta_list; do > + meta=$(basename "$d") > + sed -E "s|\{meta\}|$meta| ; \ > + s|\{layerdir\}|$openembedded_dir/$meta| ; \ > + s|\{machine\}|$machine_list|" \ > + $auh_dir/upgrade-helper.in \ > + > $build_dir/upgrade-helper/upgrade-helper.conf > + python3 $auh_dir/upgrade-helper.py -e all > +done > + > +# clean up to avoid the disk filling up > +rm -rf $build_dir/tmp/ > +rm -rf $build_dir/workspace/sources/* > +find $sstate_dir -atime +10 -delete > + > +popd || exit 1 > + > -- > 2.39.2 >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61318): https://lists.yoctoproject.org/g/yocto/message/61318 Mute This Topic: https://lists.yoctoproject.org/mt/101876809/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
