Am 12.02.2025 um 00:14 schrieb Bruce Ashfield via lists.openembedded.org:
In message: [bitbake-devel] [RFC PATCH 00/30] Add vendor support for go, npm and rust on 11/02/2025 Stefan Herbrechtsmeier via lists.openembedded.org wrote:From: Stefan Herbrechtsmeier<stefan.herbrechtsme...@weidmueller.com> The series adds on-the-fly support for package manager specific dependencies and vendor directories. It contains the following changes: 1. Adds an early fetch, unpack and patch task to unpack and patch source code with an embedded lock file for dependencies. 2. Parse the go.sum, Cargo.lock and package-lock.json lock files and resolve the dependencies to SRC_URIs. 3. Save the SRC_URIs in a file and adapt all SRC_URIs users to handle the SRC_URI files beside the SRC_URIs in the recipe.I made a few comments, and will have another / better look at the series tomorrow. There's a lot here, and it is hard to wrap my head around everything that is changing. I have one specific question below (from the point of view of go). I've been looking through the series, and can't pick out where #3 is done.
The files are created in the package manager specific vendor classes (17 - 19) in the do_vendor_resolve task. The parsing is implemented in the package manager specific modules in the vendor package (6-8).
I see patch 14 using SRC_URI_FILES, but where are those files written ? Is that in patch 18 (vendor_go_do_vendor_resolve ?) What is written to those files ?
The file is written into the vendor directory below WORKDIR. The approach is similar to other classes which need to share data between tasks.
The files contain individual SRC_URIs per line.
The concept that I'm not understanding (and that's just me not being familiar with things, I'll continue reading the series) is that when we suggested we'd like to have a mode where the dependencies could clearly be listed in the SRC_URI, at least I was just thinking about a way run the fetch/module elements that you were adding, write them to a file and then have the recipe include it.
The additional SRC_URIs are dynamically added to the recipe without the indirection over an include file. The generated SRC_URIs could be inspected via the SRC_URI_FILES and manipulated via patches for the lock files or an separated lock file.
I can't tell if in the series those files are written each time, and that there would be no way to edit those SRC_URI_FILES .. but I'll look again tomorrow.
The files are written if the do_vendor_resolve task need to be rerun. It isn't possible to edit the generated SRC_URIs. Instead the lock file could be patched.
That file would manipulate the standard SRC_URI. In other words still support a mode that is like the .inc files with crate://. So someone could either have the lockfile parsed and fetched, or have a way to run the parsing and fetching via a task, write a file and include the file in their recipe to short circuit the processing of the lockfile. (meaning the expanded and end fetches that are done once you've processed the file are simply listed as a series of fetches that are carried out without extra processing .. and "unrolled" dependency file pointing at the "sources" git, crate, mod, whatever)
What is the motivation "to short circuit the processing of the lockfile"? Like any task the processing and its output are cached via the sstate cache.
It is possible to use the vendor package to create a include file but we should have good reasons to support two solutions for the same problem.
If that just doesn't make sense, then if there was a way to copy the lockfile out of the recipe and have it overlayed onto the fetched one .. maybe breaking out the individual fetch lines isn't required, since they could be individually manipulated in that lockfile.
I haven't test it but this should be possible. You can place a lock file beside the recipe, add it to the SRC_URI and change the XXX_LOCK_PATH.
Regards Stefan
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#211212): https://lists.openembedded.org/g/openembedded-core/message/211212 Mute This Topic: https://lists.openembedded.org/mt/111133313/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
