Jate, I don’t know when the binary is removed, but I see update-alternatives –remove for all the busybox links is in the prerm script and update-alternatives –install is in the postinst script. For opkg, I am witnessing the prerm stage run for busybox and other packages performing some stages of their upgrade prior to busybox completing its upgrade. It is possible that rpm operates differently.
I have a patch that I will be sending out shortly. I’ve tested on honister by forcing all package versions to increment. I’ve also tested upgrading from an image based on the morty release up to the latest on honister. Both ways I upgrade cleanly with my patch. -Bryan From: Jate Sujjavanich <jate...@gmail.com> Sent: Wednesday, February 2, 2022 10:59 AM To: Bryan Evenson <beven...@melinkcorp.com> Cc: Valek, Andrej <andrej.va...@siemens.com>; openembedded-core@lists.openembedded.org Subject: Re: [OE-core] [dunfell][PATCH RFC] busybox.inc: Create temporary busybox links during install I have performed a major distribution upgrade in the past. This was with rpm which requires bash. I did not have issues with utilities required by the upgrade disappearing. I believe rpm installs the new file, and then removes old files. Does opkg remove the old binary before installing the new version? -Jate On Fri, Jan 28, 2022 at 8:12 AM Bryan Evenson <beven...@melinkcorp.com<mailto:beven...@melinkcorp.com>> wrote: Andrej, > -----Original Message----- > From: Valek, Andrej > <andrej.va...@siemens.com<mailto:andrej.va...@siemens.com>> > Sent: Friday, January 28, 2022 6:39 AM > To: > openembedded-core@lists.openembedded.org<mailto:openembedded-core@lists.openembedded.org>; > Bryan Evenson > <beven...@melinkcorp.com<mailto:beven...@melinkcorp.com>> > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary busybox > links during install > > Hello Bryan, > > So looks like, there is some kind of problem. > > Was you able to run the busybox command after upgrade like, "busybox ls /" > ? In this particular case, after upgrade both "busybox ls" and "ls" worked. However, that was partly because for this particular case the packages were identical but I just forced a version number change. None of the files were actually different after the upgrade. Failure to replace files didn't affect operation. When I was upgrading my image from the morty release to the dunfell release, there was a change in util-linux that conflicted with busybox if they both didn't upgrade cleanly and the update-alternatives installation step would fail for both. In that case "busybox ls" would still work but "ls" would not. Repeated upgrade attempts didn't fix it. I had to manually delete certain files that were not deleted during the initial upgrade attempt and then upgrade packages manually in a particular order to get the system fully operational again. > - If yes, there is a problem, that update alternatives hasn't been processed > correctly. Try direct command after reboot, if possible > - If no, lets continue with patch creation based on the master branch I will continue work on a patch based on master. I'm going to try testing a few other cases so I don't anticipate having a patch ready until early next week. Thanks, Bryan > > Cheers, > Andrej > > On Thu, 2022-01-27 at 14:39 +0000, Bryan Evenson wrote: > > Andrej, > > > > > -----Original Message----- > > > From: Valek, Andrej > > > <andrej.va...@siemens.com<mailto:andrej.va...@siemens.com>> > > > Sent: Saturday, January 22, 2022 2:26 AM > > > To: > > > openembedded-core@lists.openembedded.org<mailto:openembedded-core@lists.openembedded.org>; > > > Bryan Evenson > > > <beven...@melinkcorp.com<mailto:beven...@melinkcorp.com>> > > > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > busybox links during install > > > > > > Hello again, > > > > > > Maybe a general question. Is it working in current master? I do not > > > want to brake dunfell, just applying something, which will create a > > > lot of divergence. > > > > > > > I wasn't able to build an image based on the latest master as of > > Monday. From the error messages I think it had something to do with > > the addition of setuptools3-base that wasn't complete yet. I was able > > to build and run an image based on the latest honister branch. > > The master branch has four commits for busybox that honsiter does not > > have. In those four commits, I see no changes to any of the > > installation steps. Based on this information, I'd say its safe to > > say that testing on the latest honister branch should confirm whether > > the problem is still present or not. > > > > To test, I first built my image and directly programmed it on my > > hardware. My image is based upon core-image-minimal, uses sysvinit > > for an init system and opkg for a package manager. I have also added > > bash, cronie, dhcpcd and util-linux to my image; I've added others but > > these seem to produce the most obvious errors. > > > > After programming the image on my hardware, I added PR="r1" to the gcc > > recipe. This forced all the packages to get rebuilt and increment the > > version number (if anyone knows a simpler way to accomplish this, let > > me know). I then created a package feed with this update and > > attempted an upgrade. > > > > To upgrade, I did: > > opkg update > > opkg --download-only upgrade > > opkg upgrade > full_upgrade.txt 2>&1 & > > > > I download all the packages to be upgraded first. I then sent the > > upgrade process to the background so I could check other things during > > the upgrade process. > > > > During the upgrade, I would occasionally execute "ls -l" to verify > > that the file full_upgrade.txt existed and was still growing. About a > > minute into the upgrade process when I executed "ls -l" I got the > > error message: > > > > ls: not found > > > > From that point on until upgrade was complete I had to execute > > "busybox ls -l" to see the current directory file list. After upgrade > > was complete, I checked full_upgrade.txt for error messages. > > I saw the following: > > > > /usr/bin/update-alternatives: line 1: sed: not found (occurred 102 > > times) > > /usr/sbin/update-rc.d: line 54: rm: not found (occurred 36 times) > > Stopping crond: /etc/init.d/crond: line 37: start-stop-daemon: not > > found > > /tmp/opkg-yBH1Ta/cronie-zHpym4/preinst: line 1: tr: not found > > /etc/init.d/udev: line 74: start-stop-daemon: not found > > /tmp/opkg-yBH1Ta/dhcpcd-c86ca9/preinst: line 1: tr: not found > > > > In this case, a lot of alternatives and init scripts may not be setup > > correctly because either the old ones were not removed or the new ones > > were not installed. Prior to my patch below, I saw similar errors > > when upgrading my image from a build based off the morty branch to one > > based off dunfell (and my system was not operational after the > > upgrade). After my patch, I performed the same upgrade with no errors > > and my system was fully operational. > > > > I'm up for any thoughts on a different approach that accomplishes the > > same goal. Otherwise, I'm going to work with what I have and update > > my changes so they patch cleanly on master. > > > > Thanks, > > Bryan > > > > > Cheers, > > > Andrej > > > > > > On Fri, 2022-01-21 at 15:02 +0000, Bryan Evenson wrote: > > > > Andrej, > > > > > > > > Thanks for the response. This is an attempt to fix a problem I am > > > > having with automated firmware upgrades for my system. I am using > > > > opkg for a package manager; not sure if the same problem exists > > > > with other package managers. I run into problems whenever busybox > > > > is one of the packages that needs to get updated. I enact my > > > > distribution firmware upgrade by calling "opkg --download-only > > > > upgrade; opkg upgrade". What I see happen is: > > > > > > > > 1. In the busybox pkg_prerm stage sets up some soft links for some > > > > common applets in a temporary directory and exports a path to that > > > > directory. It might also setup a temporary alternative to /bin/sh > > > > if it is the last shell. > > > > 2. After the remove stage, the busybox binary is gone. The > > > > softlinks created in the prerm stage are useless since they point > > > > to binary that no longer exists. > > > > 3. opkg continues with upgrade on other packages which may depend > > > > on a command provided by busybox in a prerm, postrm, preinst or > > > > postinst script. These upgrades then fail since the commands are > > > > no longer available. > > > > 4. The busybox upgrade completes, which may or may not complete > > > > successfully. For what I am attempting, I am upgrading my system > > > > from the morty branch to dunfell. I have util-linux on my system > > > > which shares some alternatives with busybox. The util-linux > > > > upgrade fails because it needs some busybox applets during its > > > > upgrade process. > > > > Then the busybox upgrade fails because the final update- > > > > alternatives doesn’t work; some files still exist that util-linux > > > > couldn't remove during its upgrade that clash with busybox's > > > > alternatives. > > > > > > > > After trying several ways to get my upgrade to work, I landed on > > > > the approach below. I'm creating a temporary directory and > > > > copying the busybox binary and the busybox.links files to that > > > > directory. I then install an alternative for every applet for > > > > busybox listed in all of its busybox.links files that points to > > > > the temporary copy of the busybox binary. This means that any > > > > package that uses a busybox applet during its install process > > > > should still work. Then during the postinst step I am removing > > > > all the temporary alternative links. > > > > I > > > > use the temporary busybox.links files for removing the alternative > > > > links in case the upgraded busybox is now configured with a > > > > different set of applets. > > > > > > > > This is a heavy handed approach, and it does extend the upgrade > > > > process for me by a few minutes since it runs through update- > > > > alternatives for busybox two more times. But, the approach works > > > > for me and I think would be more resilient than past approaches. > > > > I tried to mimic the existing code in my additions. If a more > > > > widescale rewrite makes sense than that works for me also. > > > > > > > > Thanks, > > > > Bryan > > > > > > > > > > > > > -----Original Message----- > > > > > From: Valek, Andrej > > > > > <andrej.va...@siemens.com<mailto:andrej.va...@siemens.com>> > > > > > Sent: Friday, January 21, 2022 9:01 AM > > > > > To: > > > > > openembedded-core@lists.openembedded.org<mailto:openembedded-core@lists.openembedded.org>; > > > > > Bryan Evenson > > > > > <beven...@melinkcorp.com<mailto:beven...@melinkcorp.com>> > > > > > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > > > busybox links during install > > > > > > > > > > Hi Bryan, > > > > > > > > > > Sorry, maybe I didn't fully understand the use-case. Are you > > > > > trying to upgrade the busybox on demand? If yes, that is not a > > > > > good idea. > > > > > > > > > > I'm little bit scary about doing "export > > > > > PATH=$busybox_rmdir:$PATH" > > > > > and > > > > > creating a custom locks is not a good at all. > > > > > > > > > > Cheers, > > > > > Andrej > > > > > > > > > > On Fri, 2022-01-21 at 13:29 +0000, Bryan Evenson wrote: > > > > > > All, > > > > > > > > > > > > Ping on this RFC. It works for me, but I have a feeling there > > > > > > is a better way to do this. It still seems a little messy and > > > > > > could probably be simplified for the same effect. > > > > > > > > > > > > Thanks, > > > > > > Bryan > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: Bryan Evenson > > > > > > > Sent: Thursday, December 23, 2021 9:50 AM > > > > > > > To: > > > > > > > openembedded-core@lists.openembedded.org<mailto:openembedded-core@lists.openembedded.org> > > > > > > > Subject: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > > > > > busybox links during install > > > > > > > > > > > > > > Busybox upgrades sometimes fail, especially if there is a > > > > > > > major > > > > > > > distribution upgrade and all packages need to be updated. > > > > > > > Success > > > > > > > is highly dependent on the package upgrade order. > > > > > > > > > > > > > > Commit [1] attempts to ensure a shell is still present by > > > > > > > adding > > > > > > > an alternative to /bin/sh if busybox is the only shell. > > > > > > > However, if busybox is not the only shell present and the > > > > > > > other > > > > > > > shells are upgrading, it may then be possible that all > > > > > > > shells > > > > > > > will be removed during the upgrade process. > > > > > > > > > > > > > > Commit [2] creates temporary symbolic links for all the > > > > > > > busybox > > > > > > > links during busybox's postinst step. However, this is too > > > > > > > late > > > > > > > in the process as some packages attempt to use 'rm' and > > > > > > > 'sed' > > > > > > > after > > > > > > > update-alternatives removes the old links and prior to when > > > > > > > busybox's postinst step runs. > > > > > > > > > > > > > > This fix is similar to [2] but runs during the preinst > > > > > > > step. For > > > > > > > opkg, this is the first step that is guaranteed to run from > > > > > > > the > > > > > > > new package (prerm is run from the old package) and will > > > > > > > therefore be a backwards-compatible fix for upgrading older > > > > > > > systems. > > > > > > > > > > > > > > Copies the existing busybox binary and the busybox.links > > > > > > > files > > > > > > > to a temporary directory and then creates alternative links > > > > > > > for > > > > > > > all installed busybox commands. The temporary links and > > > > > > > directory are cleaned up during the postinst step. > > > > > > > > > > > > > > RFC: This works for me, but there may be room for > > > > > > > improvement. I > > > > > > > don't know if the current pkg_prerm steps are necessary > > > > > > > anymore. > > > > > > > However, in > > > > > > > my testing I did need the links for update-alternatives to > > > > > > > work > > > > > > > in the preinst step. I am also not certain if the > > > > > > > populate_packages_updatealternatives_append > > > > > > > step is necessary anymore. I have also only tested this > > > > > > > fix on > > > > > > > dunfell, as I don't have a working image based on master > > > > > > > yet. > > > > > > > It > > > > > > > may be more appropriate for this to go to master and then > > > > > > > be > > > > > > > backported to dunfell, but I would need assistance in > > > > > > > testing. > > > > > > > > > > > > > > [1] https://git.openembedded.org/openembedded- > > > > > > > core/commit/meta/recipes- > > > > > > > > > > > > > > > > core/busybox/busybox.inc?id=a9d2af8f5b3da8239cf00a52883ca596a19ea23 > > > > > > > a > > > > > > > [2] https://git.openembedded.org/openembedded- > > > > > > > core/commit/meta/recipes- > > > > > > > > > > > > > > > > core/busybox/busybox.inc?id=3a035bd0a06a6ded4d0ce7e35a3bce42245727 > > > > > > > d2 > > > > > > > > > > > > > > Signed-off-by: Bryan Evenson > > > > > > > <beven...@melinkcorp.com<mailto:beven...@melinkcorp.com>> > > > > > > > --- > > > > > > > meta/recipes-core/busybox/busybox.inc | 57 > > > > > > > ++++++++++++++++++++++++++- > > > > > > > 1 file changed, 55 insertions(+), 2 deletions(-) > > > > > > > > > > > > > > diff --git a/meta/recipes-core/busybox/busybox.inc > > > > > > > b/meta/recipes- > > > > > > > core/busybox/busybox.inc index e0522be729..c85402411b > > > > > > > 100644 > > > > > > > --- a/meta/recipes-core/busybox/busybox.inc > > > > > > > +++ b/meta/recipes-core/busybox/busybox.inc > > > > > > > @@ -441,12 +441,28 @@ pkg_postinst_${PN}_prepend () { } > > > > > > > > > > > > > > pkg_postinst_${PN}_append () { > > > > > > > - # If busybox exists in the remove directory it is > > > > > > > because it was the only shell left. > > > > > > > if [ "x$D" = "x" ] ; then > > > > > > > + # If busybox exists in the remove directory it > > > > > > > is > > > > > > > because it was the only > > > > > > > shell left. > > > > > > > if [ "x$BUSYBOX" != "x" ] ; then > > > > > > > update-alternatives --remove sh $BUSYBOX > > > > > > > - rm -f $BUSYBOX > > > > > > > fi > > > > > > > + # Remove the temporary alternatives > > > > > > > + for busybox_preinstdir in /tmp/busyboxpreinst- > > > > > > > *; do > > > > > > > + if [ "$busybox_preinstdir" != > > > > > > > '/tmp/busyboxpreinst- > > > > > > > *' ] ; then > > > > > > > + > > > > > > > BUSYBOX_PREINST_DIR="$busybox_preinstdir" > > > > > > > + BUSYBOX="$BUSYBOX_PREINST_DIR/busybox" > > > > > > > + if [ -e $BUSYBOX ] ; then > > > > > > > + for suffix in "" ".nosuid" ".suid"; > > > > > > > do > > > > > > > + if [ -e > > > > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix ] ; then > > > > > > > + while read link; do > > > > > > > + update-alternatives -- > > > > > > > remove > > > > > > > $($BUSYBOX basename > > > > > > > $link) $BUSYBOX > > > > > > > + done < > > > > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix > > > > > > > + fi > > > > > > > + done > > > > > > > + fi > > > > > > > + rm -rf $BUSYBOX_PREINST_DIR > > > > > > > + fi > > > > > > > + done > > > > > > > fi > > > > > > > } > > > > > > > > > > > > > > @@ -480,6 +496,43 @@ pkg_prerm_${PN} () { > > > > > > > fi > > > > > > > } > > > > > > > > > > > > > > +pkg_preinst_${PN} () { > > > > > > > + # Create a temporary copy the busybox binary and > > > > > > > the > > > > > > > links > > > > > > > files. Then, > > > > > > > + # install an alternative link for all the links. > > > > > > > Other > > > > > > > packages use these > > > > > > > + # commands during their upgrade process. This > > > > > > > ensures > > > > > > > the > > > > > > > links are > > > > > > > available > > > > > > > + # to all the other packages. We do this in the > > > > > > > preinst > > > > > > > step because it is > > > > > > > + # the first step guaranteed to be used from the > > > > > > > new > > > > > > > package. The > > > > > > > prerm is > > > > > > > + # used from the old package. Placing this here > > > > > > > ensures it > > > > > > > runs on > > > > > > > upgrade even > > > > > > > + # on older systems. > > > > > > > + > > > > > > > + if [ "x$D" = "x" ] ; then > > > > > > > + # update-alternatives may need the links from > > > > > > > commands > > > > > > > added in > > > > > > > the prerm step > > > > > > > + # to operate. Make sure we can get to that > > > > > > > path. > > > > > > > + for busybox_rmdir in /tmp/busyboxrm-*; do > > > > > > > + if [ "$busybox_rmdir" != '/tmp/busyboxrm-*' > > > > > > > ] ; > > > > > > > then > > > > > > > + export PATH=$busybox_rmdir:$PATH > > > > > > > + fi > > > > > > > + done > > > > > > > + > > > > > > > + # Create a temporary directory for the busybox > > > > > > > binary > > > > > > > and the link lists > > > > > > > + BUSYBOX=${base_bindir}/busybox > > > > > > > + BUSYBOX_TMP_DIR=`$BUSYBOX mktemp -d > > > > > > > /tmp/busyboxpreinst- > > > > > > > XXXXXX` > > > > > > > + BUSYBOX_TMP_LOC="$BUSYBOX_TMP_DIR/busybox" > > > > > > > + $BUSYBOX cp $BUSYBOX $BUSYBOX_TMP_LOC > > > > > > > + > > > > > > > + # Go through all the links and install an > > > > > > > alternative > > > > > > > that points to the > > > > > > > temporary > > > > > > > + # busybox binary. > > > > > > > + for suffix in "" ".nosuid" ".suid"; do > > > > > > > + if [ -e ${sysconfdir}/busybox.links$suffix > > > > > > > ] ; > > > > > > > then > > > > > > > + $BUSYBOX cp > > > > > > > ${sysconfdir}/busybox.links$suffix > > > > > > > $BUSYBOX_TMP_DIR > > > > > > > + while read link; do > > > > > > > + update-alternatives --install $link > > > > > > > $($BUSYBOX basename $link) > > > > > > > $BUSYBOX_TMP_LOC 1 > > > > > > > + done < > > > > > > > $BUSYBOX_TMP_DIR/busybox.links$suffix > > > > > > > + fi > > > > > > > + done > > > > > > > + fi > > > > > > > +} > > > > > > > + > > > > > > > pkg_postrm_${PN} () { > > > > > > > # Add path to remove dir in case we removed our > > > > > > > only > > > > > > > grep > > > > > > > if [ "x$D" = "x" ] ; then > > > > > > > -- > > > > > > > 2.17.1 > > > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#161325): https://lists.openembedded.org/g/openembedded-core/message/161325 Mute This Topic: https://lists.openembedded.org/mt/87919018/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
