I think no one actually anticipated all these corner cases when devtool was written. In general it's simply not designed to handle multiple sources in SRC_URI, it can do (tarball or git)+patches, and not any of the other options. No one is expecting a fix for all of them, perhaps it's best to keep the scope to plain gitsm://.
I don't have specific advice for the broader issue; if you have ideas, it would be good to at least document them. Alex On Fri, 10 Nov 2023 at 17:24, Julien Stephan <jstep...@baylibre.com> wrote: > > Hi all, > > Sorry for the long message here, but I would really appreciate > feedback from the community :) > > I am currently working on bug #14141 > (https://bugzilla.yoctoproject.org/show_bug.cgi?id=14141). The main > goal of this bug is to correctly support git submodules with devtool. > > I currently have an almost ready branch here: > https://git.yoctoproject.org/poky-contrib/log/?h=jstephan/devtool-submodule-fix > > I asked Jon Mason (reporter of the bug) to test the branch, he > confirmed that its working for git submodules (i.e recipes using gitsm > only) such as vulkan-samples or hafnium (meta-arm), but he also > pointed out a new issue highlighted by the recipe edk2-firmware: this > recipe is using gitsm for the primary source then extract another git > repository inside S. > > This issue is not related only to gitsm + git recipes, I reproduced > this kind of error with cross-localedef-native recipe, using git + > git or https + git. But I think the issue is even more global: as of > today devtool doesn't properly handle recipes that unpack secondary > sources inside S. Let suppose we have: > > SRC_URI = " \ > http://<some_url>.tar.gz \ > http://<another_url>.tar.gz;subdir=mysubdir \ > " > If I need to patch something inside mysubdir I would expect to have > something like: > SRC_URI = " \ > http://<some_url>.tar.gz \ > http://<another_url>.tar.gz;subdir=mysubdir \ > file://mypatch.patch;patchdir=mysubdir \ > " > Of course this is working, but if I run devtool modify, create my > patch and do a devtool finish I would have something like > SRC_URI = " \ > http://<some_url>.tar.gz \ > http://<another_url>.tar.gz;subdir=mysubdir \ > file://mypatch.patch \ > " > where mypatch.patch will patch files inside mysubdir as if they were > originally part of the primary source. > That might be acceptable (but I personally would prefer to have a > patchir parameter matching the subdir parameter). > Besides that, there is actually a bug here: trying to modify inside > devtool a file that was added using file:// will add a new patch AND > modify the original patch... (Tested on master branch with bzip2 with > the following: devtool modify bzip2, modify Makefile.am, commit the > modification, devtool finish bzip2 will create a new patch file AND > also update the originale Makefile.am inside the recipe space) > > But things get really bad when using a git source as secondary source: > SRC_URI = " \ > http://<some_url>.tar.gz \ > git ://<another_url>;destsuffix=mysubdir \ > " > mysubdir gets automatically committed inside the primary source > ("Commiting changes from do patch"). This is an issue because doing > "git add" on a git directory adds it as a submodule (in a wrong way, > the correct way of adding it would be "git submodule add url path"). > Here we end up with a submodule which is not populated into > .gitmodules then "git submodule foreach" commands just fail. > > My hack is to detect that git directories were added after unpack and > properly add them as submodules (" wip: support git tree extracted > inside S" in my branch) but I am not 100% satisfied with this because > it implies that devtool is managing secondary source differently if > they are git directories or not.. > > So the long story made short: > * current implementation of devtool is buggy (at least with the > scenario described above with bzip2), I may try to work on this latter > * is the current behaviour of devtool acceptable ? (i.e not adding > patchir parameter on patches) > * is my "hack" acceptable to handle git secondary sources as a real > git submodule? > > This is giving me headaches and I would really appreciate the feedback > of the community on this. > > Cheers > Julien > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61678): https://lists.yoctoproject.org/g/yocto/message/61678 Mute This Topic: https://lists.yoctoproject.org/mt/102509834/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
