I have performed a major distribution upgrade in the past. This was with rpm which requires bash. I did not have issues with utilities required by the upgrade disappearing.
I believe rpm installs the new file, and then removes old files. Does opkg remove the old binary before installing the new version? -Jate On Fri, Jan 28, 2022 at 8:12 AM Bryan Evenson <beven...@melinkcorp.com> wrote: > Andrej, > > > -----Original Message----- > > From: Valek, Andrej <andrej.va...@siemens.com> > > Sent: Friday, January 28, 2022 6:39 AM > > To: openembedded-core@lists.openembedded.org; Bryan Evenson > > <beven...@melinkcorp.com> > > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary busybox > > links during install > > > > Hello Bryan, > > > > So looks like, there is some kind of problem. > > > > Was you able to run the busybox command after upgrade like, "busybox ls > /" > > ? > > In this particular case, after upgrade both "busybox ls" and "ls" worked. > However, that was partly because for this particular case the packages were > identical but I just forced a version number change. None of the files > were actually different after the upgrade. Failure to replace files didn't > affect operation. When I was upgrading my image from the morty release to > the dunfell release, there was a change in util-linux that conflicted with > busybox if they both didn't upgrade cleanly and the update-alternatives > installation step would fail for both. In that case "busybox ls" would > still work but "ls" would not. Repeated upgrade attempts didn't fix it. I > had to manually delete certain files that were not deleted during the > initial upgrade attempt and then upgrade packages manually in a particular > order to get the system fully operational again. > > > - If yes, there is a problem, that update alternatives hasn't been > processed > > correctly. Try direct command after reboot, if possible > > - If no, lets continue with patch creation based on the master branch > > I will continue work on a patch based on master. I'm going to try testing > a few other cases so I don't anticipate having a patch ready until early > next week. > > Thanks, > Bryan > > > > > Cheers, > > Andrej > > > > On Thu, 2022-01-27 at 14:39 +0000, Bryan Evenson wrote: > > > Andrej, > > > > > > > -----Original Message----- > > > > From: Valek, Andrej <andrej.va...@siemens.com> > > > > Sent: Saturday, January 22, 2022 2:26 AM > > > > To: openembedded-core@lists.openembedded.org; Bryan Evenson > > > > <beven...@melinkcorp.com> > > > > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > > busybox links during install > > > > > > > > Hello again, > > > > > > > > Maybe a general question. Is it working in current master? I do not > > > > want to brake dunfell, just applying something, which will create a > > > > lot of divergence. > > > > > > > > > > I wasn't able to build an image based on the latest master as of > > > Monday. From the error messages I think it had something to do with > > > the addition of setuptools3-base that wasn't complete yet. I was able > > > to build and run an image based on the latest honister branch. > > > The master branch has four commits for busybox that honsiter does not > > > have. In those four commits, I see no changes to any of the > > > installation steps. Based on this information, I'd say its safe to > > > say that testing on the latest honister branch should confirm whether > > > the problem is still present or not. > > > > > > To test, I first built my image and directly programmed it on my > > > hardware. My image is based upon core-image-minimal, uses sysvinit > > > for an init system and opkg for a package manager. I have also added > > > bash, cronie, dhcpcd and util-linux to my image; I've added others but > > > these seem to produce the most obvious errors. > > > > > > After programming the image on my hardware, I added PR="r1" to the gcc > > > recipe. This forced all the packages to get rebuilt and increment the > > > version number (if anyone knows a simpler way to accomplish this, let > > > me know). I then created a package feed with this update and > > > attempted an upgrade. > > > > > > To upgrade, I did: > > > opkg update > > > opkg --download-only upgrade > > > opkg upgrade > full_upgrade.txt 2>&1 & > > > > > > I download all the packages to be upgraded first. I then sent the > > > upgrade process to the background so I could check other things during > > > the upgrade process. > > > > > > During the upgrade, I would occasionally execute "ls -l" to verify > > > that the file full_upgrade.txt existed and was still growing. About a > > > minute into the upgrade process when I executed "ls -l" I got the > > > error message: > > > > > > ls: not found > > > > > > From that point on until upgrade was complete I had to execute > > > "busybox ls -l" to see the current directory file list. After upgrade > > > was complete, I checked full_upgrade.txt for error messages. > > > I saw the following: > > > > > > /usr/bin/update-alternatives: line 1: sed: not found (occurred 102 > > > times) > > > /usr/sbin/update-rc.d: line 54: rm: not found (occurred 36 times) > > > Stopping crond: /etc/init.d/crond: line 37: start-stop-daemon: not > > > found > > > /tmp/opkg-yBH1Ta/cronie-zHpym4/preinst: line 1: tr: not found > > > /etc/init.d/udev: line 74: start-stop-daemon: not found > > > /tmp/opkg-yBH1Ta/dhcpcd-c86ca9/preinst: line 1: tr: not found > > > > > > In this case, a lot of alternatives and init scripts may not be setup > > > correctly because either the old ones were not removed or the new ones > > > were not installed. Prior to my patch below, I saw similar errors > > > when upgrading my image from a build based off the morty branch to one > > > based off dunfell (and my system was not operational after the > > > upgrade). After my patch, I performed the same upgrade with no errors > > > and my system was fully operational. > > > > > > I'm up for any thoughts on a different approach that accomplishes the > > > same goal. Otherwise, I'm going to work with what I have and update > > > my changes so they patch cleanly on master. > > > > > > Thanks, > > > Bryan > > > > > > > Cheers, > > > > Andrej > > > > > > > > On Fri, 2022-01-21 at 15:02 +0000, Bryan Evenson wrote: > > > > > Andrej, > > > > > > > > > > Thanks for the response. This is an attempt to fix a problem I am > > > > > having with automated firmware upgrades for my system. I am using > > > > > opkg for a package manager; not sure if the same problem exists > > > > > with other package managers. I run into problems whenever busybox > > > > > is one of the packages that needs to get updated. I enact my > > > > > distribution firmware upgrade by calling "opkg --download-only > > > > > upgrade; opkg upgrade". What I see happen is: > > > > > > > > > > 1. In the busybox pkg_prerm stage sets up some soft links for some > > > > > common applets in a temporary directory and exports a path to that > > > > > directory. It might also setup a temporary alternative to /bin/sh > > > > > if it is the last shell. > > > > > 2. After the remove stage, the busybox binary is gone. The > > > > > softlinks created in the prerm stage are useless since they point > > > > > to binary that no longer exists. > > > > > 3. opkg continues with upgrade on other packages which may depend > > > > > on a command provided by busybox in a prerm, postrm, preinst or > > > > > postinst script. These upgrades then fail since the commands are > > > > > no longer available. > > > > > 4. The busybox upgrade completes, which may or may not complete > > > > > successfully. For what I am attempting, I am upgrading my system > > > > > from the morty branch to dunfell. I have util-linux on my system > > > > > which shares some alternatives with busybox. The util-linux > > > > > upgrade fails because it needs some busybox applets during its > > > > > upgrade process. > > > > > Then the busybox upgrade fails because the final update- > > > > > alternatives doesn’t work; some files still exist that util-linux > > > > > couldn't remove during its upgrade that clash with busybox's > > > > > alternatives. > > > > > > > > > > After trying several ways to get my upgrade to work, I landed on > > > > > the approach below. I'm creating a temporary directory and > > > > > copying the busybox binary and the busybox.links files to that > > > > > directory. I then install an alternative for every applet for > > > > > busybox listed in all of its busybox.links files that points to > > > > > the temporary copy of the busybox binary. This means that any > > > > > package that uses a busybox applet during its install process > > > > > should still work. Then during the postinst step I am removing > > > > > all the temporary alternative links. > > > > > I > > > > > use the temporary busybox.links files for removing the alternative > > > > > links in case the upgraded busybox is now configured with a > > > > > different set of applets. > > > > > > > > > > This is a heavy handed approach, and it does extend the upgrade > > > > > process for me by a few minutes since it runs through update- > > > > > alternatives for busybox two more times. But, the approach works > > > > > for me and I think would be more resilient than past approaches. > > > > > I tried to mimic the existing code in my additions. If a more > > > > > widescale rewrite makes sense than that works for me also. > > > > > > > > > > Thanks, > > > > > Bryan > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: Valek, Andrej <andrej.va...@siemens.com> > > > > > > Sent: Friday, January 21, 2022 9:01 AM > > > > > > To: openembedded-core@lists.openembedded.org; Bryan Evenson > > > > > > <beven...@melinkcorp.com> > > > > > > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > > > > busybox links during install > > > > > > > > > > > > Hi Bryan, > > > > > > > > > > > > Sorry, maybe I didn't fully understand the use-case. Are you > > > > > > trying to upgrade the busybox on demand? If yes, that is not a > > > > > > good idea. > > > > > > > > > > > > I'm little bit scary about doing "export > > > > > > PATH=$busybox_rmdir:$PATH" > > > > > > and > > > > > > creating a custom locks is not a good at all. > > > > > > > > > > > > Cheers, > > > > > > Andrej > > > > > > > > > > > > On Fri, 2022-01-21 at 13:29 +0000, Bryan Evenson wrote: > > > > > > > All, > > > > > > > > > > > > > > Ping on this RFC. It works for me, but I have a feeling there > > > > > > > is a better way to do this. It still seems a little messy and > > > > > > > could probably be simplified for the same effect. > > > > > > > > > > > > > > Thanks, > > > > > > > Bryan > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: Bryan Evenson > > > > > > > > Sent: Thursday, December 23, 2021 9:50 AM > > > > > > > > To: openembedded-core@lists.openembedded.org > > > > > > > > Subject: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > > > > > > busybox links during install > > > > > > > > > > > > > > > > Busybox upgrades sometimes fail, especially if there is a > > > > > > > > major > > > > > > > > distribution upgrade and all packages need to be updated. > > > > > > > > Success > > > > > > > > is highly dependent on the package upgrade order. > > > > > > > > > > > > > > > > Commit [1] attempts to ensure a shell is still present by > > > > > > > > adding > > > > > > > > an alternative to /bin/sh if busybox is the only shell. > > > > > > > > However, if busybox is not the only shell present and the > > > > > > > > other > > > > > > > > shells are upgrading, it may then be possible that all > > > > > > > > shells > > > > > > > > will be removed during the upgrade process. > > > > > > > > > > > > > > > > Commit [2] creates temporary symbolic links for all the > > > > > > > > busybox > > > > > > > > links during busybox's postinst step. However, this is too > > > > > > > > late > > > > > > > > in the process as some packages attempt to use 'rm' and > > > > > > > > 'sed' > > > > > > > > after > > > > > > > > update-alternatives removes the old links and prior to when > > > > > > > > busybox's postinst step runs. > > > > > > > > > > > > > > > > This fix is similar to [2] but runs during the preinst > > > > > > > > step. For > > > > > > > > opkg, this is the first step that is guaranteed to run from > > > > > > > > the > > > > > > > > new package (prerm is run from the old package) and will > > > > > > > > therefore be a backwards-compatible fix for upgrading older > > > > > > > > systems. > > > > > > > > > > > > > > > > Copies the existing busybox binary and the busybox.links > > > > > > > > files > > > > > > > > to a temporary directory and then creates alternative links > > > > > > > > for > > > > > > > > all installed busybox commands. The temporary links and > > > > > > > > directory are cleaned up during the postinst step. > > > > > > > > > > > > > > > > RFC: This works for me, but there may be room for > > > > > > > > improvement. I > > > > > > > > don't know if the current pkg_prerm steps are necessary > > > > > > > > anymore. > > > > > > > > However, in > > > > > > > > my testing I did need the links for update-alternatives to > > > > > > > > work > > > > > > > > in the preinst step. I am also not certain if the > > > > > > > > populate_packages_updatealternatives_append > > > > > > > > step is necessary anymore. I have also only tested this > > > > > > > > fix on > > > > > > > > dunfell, as I don't have a working image based on master > > > > > > > > yet. > > > > > > > > It > > > > > > > > may be more appropriate for this to go to master and then > > > > > > > > be > > > > > > > > backported to dunfell, but I would need assistance in > > > > > > > > testing. > > > > > > > > > > > > > > > > [1] https://git.openembedded.org/openembedded- > > > > > > > > core/commit/meta/recipes- > > > > > > > > > > > > > > > > > > > > core/busybox/busybox.inc?id=a9d2af8f5b3da8239cf00a52883ca596a19ea23 > > > > > > > > a > > > > > > > > [2] https://git.openembedded.org/openembedded- > > > > > > > > core/commit/meta/recipes- > > > > > > > > > > > > > > > > > > > > core/busybox/busybox.inc?id=3a035bd0a06a6ded4d0ce7e35a3bce42245727 > > > > > > > > d2 > > > > > > > > > > > > > > > > Signed-off-by: Bryan Evenson <beven...@melinkcorp.com> > > > > > > > > --- > > > > > > > > meta/recipes-core/busybox/busybox.inc | 57 > > > > > > > > ++++++++++++++++++++++++++- > > > > > > > > 1 file changed, 55 insertions(+), 2 deletions(-) > > > > > > > > > > > > > > > > diff --git a/meta/recipes-core/busybox/busybox.inc > > > > > > > > b/meta/recipes- > > > > > > > > core/busybox/busybox.inc index e0522be729..c85402411b > > > > > > > > 100644 > > > > > > > > --- a/meta/recipes-core/busybox/busybox.inc > > > > > > > > +++ b/meta/recipes-core/busybox/busybox.inc > > > > > > > > @@ -441,12 +441,28 @@ pkg_postinst_${PN}_prepend () { } > > > > > > > > > > > > > > > > pkg_postinst_${PN}_append () { > > > > > > > > - # If busybox exists in the remove directory it is > > > > > > > > because it was the only shell left. > > > > > > > > if [ "x$D" = "x" ] ; then > > > > > > > > + # If busybox exists in the remove directory it > > > > > > > > is > > > > > > > > because it was the only > > > > > > > > shell left. > > > > > > > > if [ "x$BUSYBOX" != "x" ] ; then > > > > > > > > update-alternatives --remove sh $BUSYBOX > > > > > > > > - rm -f $BUSYBOX > > > > > > > > fi > > > > > > > > + # Remove the temporary alternatives > > > > > > > > + for busybox_preinstdir in /tmp/busyboxpreinst- > > > > > > > > *; do > > > > > > > > + if [ "$busybox_preinstdir" != > > > > > > > > '/tmp/busyboxpreinst- > > > > > > > > *' ] ; then > > > > > > > > + > > > > > > > > BUSYBOX_PREINST_DIR="$busybox_preinstdir" > > > > > > > > + BUSYBOX="$BUSYBOX_PREINST_DIR/busybox" > > > > > > > > + if [ -e $BUSYBOX ] ; then > > > > > > > > + for suffix in "" ".nosuid" ".suid"; > > > > > > > > do > > > > > > > > + if [ -e > > > > > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix ] ; then > > > > > > > > + while read link; do > > > > > > > > + update-alternatives -- > > > > > > > > remove > > > > > > > > $($BUSYBOX basename > > > > > > > > $link) $BUSYBOX > > > > > > > > + done < > > > > > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix > > > > > > > > + fi > > > > > > > > + done > > > > > > > > + fi > > > > > > > > + rm -rf $BUSYBOX_PREINST_DIR > > > > > > > > + fi > > > > > > > > + done > > > > > > > > fi > > > > > > > > } > > > > > > > > > > > > > > > > @@ -480,6 +496,43 @@ pkg_prerm_${PN} () { > > > > > > > > fi > > > > > > > > } > > > > > > > > > > > > > > > > +pkg_preinst_${PN} () { > > > > > > > > + # Create a temporary copy the busybox binary and > > > > > > > > the > > > > > > > > links > > > > > > > > files. Then, > > > > > > > > + # install an alternative link for all the links. > > > > > > > > Other > > > > > > > > packages use these > > > > > > > > + # commands during their upgrade process. This > > > > > > > > ensures > > > > > > > > the > > > > > > > > links are > > > > > > > > available > > > > > > > > + # to all the other packages. We do this in the > > > > > > > > preinst > > > > > > > > step because it is > > > > > > > > + # the first step guaranteed to be used from the > > > > > > > > new > > > > > > > > package. The > > > > > > > > prerm is > > > > > > > > + # used from the old package. Placing this here > > > > > > > > ensures it > > > > > > > > runs on > > > > > > > > upgrade even > > > > > > > > + # on older systems. > > > > > > > > + > > > > > > > > + if [ "x$D" = "x" ] ; then > > > > > > > > + # update-alternatives may need the links from > > > > > > > > commands > > > > > > > > added in > > > > > > > > the prerm step > > > > > > > > + # to operate. Make sure we can get to that > > > > > > > > path. > > > > > > > > + for busybox_rmdir in /tmp/busyboxrm-*; do > > > > > > > > + if [ "$busybox_rmdir" != '/tmp/busyboxrm-*' > > > > > > > > ] ; > > > > > > > > then > > > > > > > > + export PATH=$busybox_rmdir:$PATH > > > > > > > > + fi > > > > > > > > + done > > > > > > > > + > > > > > > > > + # Create a temporary directory for the busybox > > > > > > > > binary > > > > > > > > and the link lists > > > > > > > > + BUSYBOX=${base_bindir}/busybox > > > > > > > > + BUSYBOX_TMP_DIR=`$BUSYBOX mktemp -d > > > > > > > > /tmp/busyboxpreinst- > > > > > > > > XXXXXX` > > > > > > > > + BUSYBOX_TMP_LOC="$BUSYBOX_TMP_DIR/busybox" > > > > > > > > + $BUSYBOX cp $BUSYBOX $BUSYBOX_TMP_LOC > > > > > > > > + > > > > > > > > + # Go through all the links and install an > > > > > > > > alternative > > > > > > > > that points to the > > > > > > > > temporary > > > > > > > > + # busybox binary. > > > > > > > > + for suffix in "" ".nosuid" ".suid"; do > > > > > > > > + if [ -e ${sysconfdir}/busybox.links$suffix > > > > > > > > ] ; > > > > > > > > then > > > > > > > > + $BUSYBOX cp > > > > > > > > ${sysconfdir}/busybox.links$suffix > > > > > > > > $BUSYBOX_TMP_DIR > > > > > > > > + while read link; do > > > > > > > > + update-alternatives --install $link > > > > > > > > $($BUSYBOX basename $link) > > > > > > > > $BUSYBOX_TMP_LOC 1 > > > > > > > > + done < > > > > > > > > $BUSYBOX_TMP_DIR/busybox.links$suffix > > > > > > > > + fi > > > > > > > > + done > > > > > > > > + fi > > > > > > > > +} > > > > > > > > + > > > > > > > > pkg_postrm_${PN} () { > > > > > > > > # Add path to remove dir in case we removed our > > > > > > > > only > > > > > > > > grep > > > > > > > > if [ "x$D" = "x" ] ; then > > > > > > > > -- > > > > > > > > 2.17.1 > > > > > > > > > > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#161218): https://lists.openembedded.org/g/openembedded-core/message/161218 Mute This Topic: https://lists.openembedded.org/mt/87919018/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
