Greetings,
this patch had a few errors that are fixed with this patch, checking for
NOT_MOVED and dont produce an error if its not there and also dont
delete packages that are in NOT_MOVED to build them again.
This Patch targets poudriere-devel 3.3.99.20200326_2
--- common.sh.1st 2021-01-27 15:44:15.000000000 +0100
+++ common.sh 2021-02-04 20:30:42.061211000 +0100
@@ -5040,7 +5040,10 @@
fi
if shash_get origin-moved "${origin}" new_origin; then
- if [ "${new_origin%% *}" = "EXPIRED" ]; then
+ # check also against NOT_MOVED from overlays so we dont
delete this package if its wanted
+ if [ -f ${MASTERMNT}${OVERLAYSDIR}/${OVERLAYS#
}/NOT_MOVED ] && (grep -q "${origin}"
${MASTERMNT}${OVERLAYSDIR}/${OVERLAYS# }/NOT_MOVED); then
+ msg_debug "${origin} is found in
${MASTERMNT}${OVERLAYSDIR}/${OVERLAYS# }/NOT_MOVED so dont delete this
package."
+ elif [ "${new_origin%% *}" = "EXPIRED" ]; then
msg "Deleting ${pkg##*/}:
${COLOR_PORT}${origin}${COLOR_RESET} ${new_origin#EXPIRED }"
else
msg "Deleting ${pkg##*/}:
${COLOR_PORT}${origin}${COLOR_RESET} moved to
${COLOR_PORT}${new_origin}${COLOR_RESET}"
@@ -6815,8 +6818,11 @@
continue
fi
origin_listed="${origin}"
- if shash_get origin-moved "${origin}" new_origin; then
- if [ "${new_origin%% *}" = "EXPIRED" ]; then
+ # check against the overlay list, skip port if its in there
+ if [ -f ${MASTERMNT}${OVERLAYSDIR}/${OVERLAYS#
}/NOT_MOVED ] && (grep -q "${origin}"
${MASTERMNT}${OVERLAYSDIR}/${OVERLAYS# }/NOT_MOVED); then
+ msg_debug "${origin} found in
${MASTERMNT}${OVERLAYSDIR}/${OVERLAYS# }/NOT_MOVED, skipped checking
MOVED File of master portstree"
+ elif shash_get origin-moved "${origin}" new_origin; then
+ if [ "${new_origin%% *}" = "EXPIRED" ] ; then
msg_error "MOVED: ${origin} ${new_origin}"
set_dep_fatal_error
continue
@@ -7085,6 +7091,10 @@
fi
[ -f ${MASTERMNT}${PORTSDIR}/MOVED ] || return 0
msg "Loading MOVED for ${MASTERMNT}${PORTSDIR}"
+ # Respect overlays, ${OVERLAYS} has a leading whitepace
+ if [ -f ${MASTERMNT}${OVERLAYSDIR}/${OVERLAYS# }/NOT_MOVED ]; then
+ msg "Loading NOT_MOVED from overlaydir:
${MASTERMNT}${OVERLAYSDIR}/${OVERLAYS# }"
+ fi
bset status "loading_moved:"
awk -f ${AWKPREFIX}/parse_MOVED.awk \
${MASTERMNT}${PORTSDIR}/MOVED | \
Kind regards,
Henrik Rosenke
Am 29.01.21 um 17:38 schrieb Henrik Rosenke:
Greetings,
i created a simple patch for this, this may not be the best solution
but my tests seems to be successful to get the desired result:
Without NOT_MOVED:
=>> Debug: Reading /usr/local/etc/poudriere.conf
[00:00:00] Creating the reference jail... done
[00:00:02] Mounting system devices for j12p64-default-server
[00:00:02] Mounting ports/packages/distfiles
[00:00:02] Using packages from previously failed build:
/home/poudriere/data/packages/j12p64-default-server/.building
[00:00:02] Mounting ccache from: /home/poudriere/ccache
[00:00:02] Mounting packages from:
/home/poudriere/data/packages/j12p64-default-server
[00:00:02] Copying /var/db/ports from:
/usr/local/etc/poudriere.d/j12p64-options
[00:00:02] Appending to make.conf: /usr/local/etc/poudriere.d/make.conf
[00:00:02] Appending to make.conf:
/usr/local/etc/poudriere.d/server-make.conf
[00:00:02] Appending to make.conf:
/usr/local/etc/poudriere.d/default-make.conf
[00:00:02] Appending to make.conf:
/usr/local/etc/poudriere.d/j12p64-make.conf
[00:00:02] Appending to make.conf:
/usr/local/etc/poudriere.d/default-server-make.conf
[00:00:02] Warning: Blacklisting (from
/usr/local/etc/poudriere.d/j12p64-blacklist): comms/chan_capi
[00:00:02] Warning: Blacklisting (from
/usr/local/etc/poudriere.d/j12p64-blacklist): comms/isdn4bsd-kmod
[00:00:02] Warning: Blacklisting (from
/usr/local/etc/poudriere.d/j12p64-blacklist): comms/isdn4bsd-utils
[00:00:02] Warning: Blacklisting (from
/usr/local/etc/poudriere.d/j12p64-blacklist): sysutils/env4801
[00:00:02] Warning: Blacklisting (from
/usr/local/etc/poudriere.d/j12p64-blacklist): security/ipsec-tools
/etc/resolv.conf ->
/home/poudriere/data/.m/j12p64-default-server/ref/etc/resolv.conf
[00:00:02] Starting jail j12p64-default-server
[00:00:02] Logs:
/home/poudriere/data/logs/bulk/j12p64-default-server/2021-01-29_11h14m59s
[00:00:02] WWW:
http://dsspkg.incore/poudriere//build.html?mastername=j12p64-default-server&build=2021-01-29_11h14m59s
[00:00:02] Loading MOVED for
/home/poudriere/data/.m/j12p64-default-server/ref/usr/ports
[00:00:03] Ports supports: FLAVORS SELECTED_OPTIONS
[00:00:03] Gathering ports metadata
[00:00:03] Error: MOVED: sysutils/sge62 EXPIRED 2013-06-05 Has
expired: Ancient and unsupported release
[00:00:03] Error: Fatal errors encountered gathering initial ports
metadata
[00:00:03] Cleaning up
[00:00:03] Unmounting file systems
With NOT_MOVED:
=== root@dsspkg (pts/1) /home/poudriere/ports/dss_overlay 35 -> cat
NOT_MOVED
sysutils/sge62
=== root@dsspkg (pts/1) /home/poudriere/ports/dss_overlay 97 ->
poudriere bulk -vv -O dss_overlay -j j12p64 -z server -p default
sysutils/sge62
=>> Debug: Reading /usr/local/etc/poudriere.conf
[00:00:00] Creating the reference jail... done
[00:00:02] Mounting system devices for j12p64-default-server
[00:00:02] Mounting ports/packages/distfiles
[00:00:02] Using packages from previously failed build:
/home/poudriere/data/packages/j12p64-default-server/.building
[00:00:02] Mounting ccache from: /home/poudriere/ccache
[00:00:02] Mounting packages from:
/home/poudriere/data/packages/j12p64-default-server
[00:00:02] Copying /var/db/ports from:
/usr/local/etc/poudriere.d/j12p64-options
[00:00:02] Appending to make.conf: /usr/local/etc/poudriere.d/make.conf
[00:00:02] Appending to make.conf:
/usr/local/etc/poudriere.d/server-make.conf
[00:00:02] Appending to make.conf:
/usr/local/etc/poudriere.d/default-make.conf
[00:00:02] Appending to make.conf:
/usr/local/etc/poudriere.d/j12p64-make.conf
[00:00:02] Appending to make.conf:
/usr/local/etc/poudriere.d/default-server-make.conf
[00:00:02] Warning: Blacklisting (from
/usr/local/etc/poudriere.d/j12p64-blacklist): comms/chan_capi
[00:00:02] Warning: Blacklisting (from
/usr/local/etc/poudriere.d/j12p64-blacklist): comms/isdn4bsd-kmod
[00:00:02] Warning: Blacklisting (from
/usr/local/etc/poudriere.d/j12p64-blacklist): comms/isdn4bsd-utils
[00:00:02] Warning: Blacklisting (from
/usr/local/etc/poudriere.d/j12p64-blacklist): sysutils/env4801
[00:00:02] Warning: Blacklisting (from
/usr/local/etc/poudriere.d/j12p64-blacklist): security/ipsec-tools
/etc/resolv.conf ->
/home/poudriere/data/.m/j12p64-default-server/ref/etc/resolv.conf
[00:00:02] Starting jail j12p64-default-server
[00:00:03] Logs:
/home/poudriere/data/logs/bulk/j12p64-default-server/2021-01-29_17h28m52s
[00:00:03] WWW:
http://dsspkg.incore/poudriere//build.html?mastername=j12p64-default-server&build=2021-01-29_17h28m52s
[00:00:03] Loading MOVED for
/home/poudriere/data/.m/j12p64-default-server/ref/usr/ports
[00:00:03] Loading NOT_MOVED from overlaydir:
/home/poudriere/data/.m/j12p64-default-server/ref/overlays/dss_overlay
[00:00:03] Ports supports: FLAVORS SELECTED_OPTIONS
[00:00:03] Gathering ports metadata
[00:00:03] Debug: sysutils/sge62 found in
/home/poudriere/data/.m/j12p64-default-server/ref/overlays/dss_overlay/NOT_MOVED,
skipped checking MOVED File of master portstree
[00:00:03] Debug: queueing sysutils/sge62 into gatherqueue (rdep=listed)
[00:00:03] Debug: Processing gatherqueue
[00:00:03] Warning: (sysutils/sge62): Debug: gather_port_vars_port
(sysutils/sge62): LOOKUP
[00:00:03] Warning: (sysutils/sge62): Debug: deps_fetch_vars:
discovered sysutils/sge62 is sge-6.2.2.1_6
[00:00:03] Warning: (sysutils/sge62): Debug: WILL BUILD sysutils/sge62
Patch leading to this:
--- /usr/local/share/poudriere/common.sh.1st 2021-01-27
15:44:15.000000000 +0100
+++ /usr/local/share/poudriere/common.sh 2021-01-29
17:27:14.810441000 +0100
@@ -6815,8 +6815,11 @@
continue
fi
origin_listed="${origin}"
- if shash_get origin-moved "${origin}" new_origin; then
- if [ "${new_origin%% *}" = "EXPIRED" ]; then
+ # check against the overlay list, skip port if its in
there
+ if (grep -q "${origin}"
${MASTERMNT}${OVERLAYSDIR}/${OVERLAYS# }/NOT_MOVED); then
+ msg_debug "${origin} found in
${MASTERMNT}${OVERLAYSDIR}/${OVERLAYS# }/NOT_MOVED, skipped checking
MOVED File of master portstree"
+ elif shash_get origin-moved "${origin}" new_origin; then
+ if [ "${new_origin%% *}" = "EXPIRED" ] ; then
msg_error "MOVED: ${origin}
${new_origin}"
set_dep_fatal_error
continue
@@ -7085,6 +7088,10 @@
fi
[ -f ${MASTERMNT}${PORTSDIR}/MOVED ] || return 0
msg "Loading MOVED for ${MASTERMNT}${PORTSDIR}"
+ # Respect overlays, ${OVERLAYS} has a leading whitepace
+ if [ -f ${MASTERMNT}${OVERLAYSDIR}/${OVERLAYS# }/NOT_MOVED ];
then
+ msg "Loading NOT_MOVED from overlaydir:
${MASTERMNT}${OVERLAYSDIR}/${OVERLAYS# }"
+ fi
bset status "loading_moved:"
awk -f ${AWKPREFIX}/parse_MOVED.awk \
${MASTERMNT}${PORTSDIR}/MOVED | \
Maybe someone else could test this, seems to work for me.
Kind regards,
Henrik Rosenke
Am 28.01.21 um 21:56 schrieb Henrik Rosenke:
Am 28.01.21 um 21:14 schrieb Miroslav Lachman:
On 28/01/2021 17:47, Henrik Rosenke wrote:
I tried this with a patched MOVED file, copied from the master
portstree, deleted the entry and tried to build but this File seems
to be ignored. I dont really understand where the MOVED file is
handled, it seems to be bsd.port.subdir.mk but i am not sure.
MOVED is handled by bsd.port.subdir.mk but overlays do not account
with it. Overlays are handled on more places
i thought this too but it seems poudriere is canceled before this, i
even tried to move the bsd.port.subdir.mk and commented out the
function load_moved in the common.sh of poudriere and can start a
build. Correct me if iam wrong but it seems that the common.sh from
poudriere needs to be patched to handle this rather than the Mk files
itself. I added set -x in the common.sh to get a better understanding
what is happening and it seems that shash_set sucks in the MOVED file
and this is checked later against the ports we want to build:
+ shash_set origin-moved sysutils/sge62 'EXPIRED 2013-06-05 Has
expired: Ancient and unsupported release'
later:
+ originspec_decode sysutils/sge62 origin '' flavor
+ local -
+ set +x -f
+ [ -n '' ]
+ origin_listed=sysutils/sge62
+ shash_get origin-moved sysutils/sge62 new_origin
+ local -
+ set +x
+ [ EXPIRED '=' EXPIRED ]
+ msg_error 'MOVED: sysutils/sge62 EXPIRED 2013-06-05 Has expired:
Ancient and unsupported release'
+ local -
+ set +x
[00:00:07] Error: MOVED: sysutils/sge62 EXPIRED 2013-06-05 Has
expired: Ancient and unsupported release
Kind regards,
Henrik Rosenke
Scripts/do-depends.sh
Scripts/depends-list.sh
Next is bsd.port.mk:
.if defined(USE_LOCAL_MK)
.include "${PORTSDIR}/Mk/bsd.local.mk"
.endif
.for odir in ${OVERLAYS}
.sinclude "${odir}/Mk/bsd.overlay.mk"
.endfor
But there is no bsd.overlay.mk and I don't know what should be in it
(file in the overlay ports tree)
The AWK code in bsd.port.subdir.mk can be changed to use MOVED from
overlay if it exists insead of the default one.
I think the best way to handle this would be a NOT_MOVED file in
the overlay portstree with the ports to ignore from the MOVED file
with entries like "sysutils/sge62".
This can be possible too. NOT_MOVED can be used as additional INDEXFILE
Or maybe start with PORTSEARCH_MOVED=0 (not tried yet)
Kind regards
Miroslav Lachman
_______________________________________________
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
_______________________________________________
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
