In message: [bitbake-devel] [RFC PATCH 00/30] Add vendor support for go, npm and rust on 11/02/2025 Stefan Herbrechtsmeier via lists.openembedded.org wrote:
> From: Stefan Herbrechtsmeier <stefan.herbrechtsme...@weidmueller.com> > > The series adds on-the-fly support for package manager specific > dependencies and vendor directories. It contains the following changes: > 1. Adds an early fetch, unpack and patch task to unpack and patch source > code with an embedded lock file for dependencies. > 2. Parse the go.sum, Cargo.lock and package-lock.json lock files and > resolve the dependencies to SRC_URIs. > 3. Save the SRC_URIs in a file and adapt all SRC_URIs users to handle > the SRC_URI files beside the SRC_URIs in the recipe. I made a few comments, and will have another / better look at the series tomorrow. There's a lot here, and it is hard to wrap my head around everything that is changing. I have one specific question below (from the point of view of go). I've been looking through the series, and can't pick out where #3 is done. I see patch 14 using SRC_URI_FILES, but where are those files written ? Is that in patch 18 (vendor_go_do_vendor_resolve ?) What is written to those files ? The concept that I'm not understanding (and that's just me not being familiar with things, I'll continue reading the series) is that when we suggested we'd like to have a mode where the dependencies could clearly be listed in the SRC_URI, at least I was just thinking about a way run the fetch/module elements that you were adding, write them to a file and then have the recipe include it. I can't tell if in the series those files are written each time, and that there would be no way to edit those SRC_URI_FILES .. but I'll look again tomorrow. That file would manipulate the standard SRC_URI. In other words still support a mode that is like the .inc files with crate://. So someone could either have the lockfile parsed and fetched, or have a way to run the parsing and fetching via a task, write a file and include the file in their recipe to short circuit the processing of the lockfile. (meaning the expanded and end fetches that are done once you've processed the file are simply listed as a series of fetches that are carried out without extra processing .. and "unrolled" dependency file pointing at the "sources" git, crate, mod, whatever) If that just doesn't make sense, then if there was a way to copy the lockfile out of the recipe and have it overlayed onto the fetched one .. maybe breaking out the individual fetch lines isn't required, since they could be individually manipulated in that lockfile. Bruce > 4. Create a package manager specific vendor directory during unpack to > support additional patching of the dependencies. > 5. Add the dependency name and version to the SBOM. > 6. Simplify the npm support > > > Stefan Herbrechtsmeier (30): > classes: create-spdx-2.2: use expanded FetchData for downloaded > packages > lib: spdx30_tasks: use expanded FetchData for download files > classes: create-spdx-2.2: use name and version for download > dependencies > lib: bb: fetch2: add support to unpack .crate files > lib: oe: add vendor module > lib: oe: vendor: add cargo support > lib: oe: vendor: add go support > lib: oe: vendor: add npm support > oeqa: oelib: add vendor tests > conf: bitbake: add SRC_URI_FILES variable > classes: go: make source directory configurable > classes: go-mod: make class customizable > classes: add nodejs-arch class > classes: base: add get_src_uris and unpack_src_uris functions > classes: add early fetch, unpack and patch support > classes: add vendor class > classes: add vendor class for cargo > classes: add vendor class for go > classes: add vendor class for npm > classes: add vendor_npm_build class > python3-bcrypt: mirgrate to vendor cargo class > python3-cryptography: mirgrate to vendor cargo class > python3-maturin: mirgrate to vendor cargo class > python3-rpds-py: mirgrate to vendor cargo class > librsvg: mirgrate to vendor cargo class > librsvg: update dependecies to fix RUSTSEC-2024-0421 > [DO NOT MERGE] recipes: add crucible go demo > [DO NOT MERGE] recipes: add node-red npm demo > [DO NOT MERGE] recipes: add nucleoidai npm demo > [DO NOT MERGE] classes: spdx: use version 2.2 > > bitbake/lib/bb/fetch2/__init__.py | 2 +- > .../crucible/crucible2_2023.11.02.bb | 18 + > .../node-red/node-red/package-lock.json | 6096 +++++++++++++++++ > .../node-red/node-red_4.0.8.bb | 14 + > .../nucleoidai/nucleoidai_0.7.10.bb | 11 + > meta/classes-global/base.bbclass | 47 +- > meta/classes-global/patch.bbclass | 17 +- > meta/classes-recipe/early.bbclass | 61 + > meta/classes-recipe/go-mod.bbclass | 10 +- > meta/classes-recipe/go.bbclass | 22 +- > meta/classes-recipe/nodejs-arch.bbclass | 15 + > meta/classes-recipe/vendor.bbclass | 28 + > meta/classes-recipe/vendor_cargo.bbclass | 46 + > meta/classes-recipe/vendor_go.bbclass | 59 + > meta/classes-recipe/vendor_npm.bbclass | 115 + > meta/classes-recipe/vendor_npm_build.bbclass | 50 + > meta/classes/archiver.bbclass | 4 +- > meta/classes/buildhistory.bbclass | 4 +- > meta/classes/copyleft_compliance.bbclass | 2 +- > meta/classes/create-spdx-2.2.bbclass | 14 +- > meta/classes/create-spdx.bbclass | 2 +- > meta/classes/externalsrc.bbclass | 2 +- > meta/conf/bitbake.conf | 1 + > meta/lib/oe/patch.py | 10 +- > meta/lib/oe/spdx30_tasks.py | 5 +- > meta/lib/oe/vendor/__init__.py | 28 + > meta/lib/oe/vendor/cargo.py | 121 + > meta/lib/oe/vendor/go.py | 96 + > meta/lib/oe/vendor/npm.py | 141 + > meta/lib/oeqa/selftest/cases/oelib/vendor.py | 237 + > .../python/python3-bcrypt-crates.inc | 84 - > .../python/python3-bcrypt_4.2.1.bb | 4 +- > .../python/python3-cryptography-crates.inc | 76 - > .../python/python3-cryptography.bb | 4 +- > .../python/python3-maturin-crates.inc | 712 -- > .../python/python3-maturin_1.8.1.bb | 4 +- > .../python/python3-rpds-py-crates.inc | 54 - > .../python/python3-rpds-py_0.22.3.bb | 4 +- > meta/recipes-gnome/librsvg/librsvg-crates.inc | 590 -- > ...-to-get-an-updated-idna-rustsec-2024.patch | 398 ++ > meta/recipes-gnome/librsvg/librsvg_2.59.2.bb | 7 +- > 41 files changed, 7633 insertions(+), 1582 deletions(-) > create mode 100644 > meta-selftest/recipes-support/crucible/crucible2_2023.11.02.bb > create mode 100644 > meta-selftest/recipes-support/node-red/node-red/package-lock.json > create mode 100644 meta-selftest/recipes-support/node-red/node-red_4.0.8.bb > create mode 100644 > meta-selftest/recipes-support/nucleoidai/nucleoidai_0.7.10.bb > create mode 100644 meta/classes-recipe/early.bbclass > create mode 100644 meta/classes-recipe/nodejs-arch.bbclass > create mode 100644 meta/classes-recipe/vendor.bbclass > create mode 100644 meta/classes-recipe/vendor_cargo.bbclass > create mode 100644 meta/classes-recipe/vendor_go.bbclass > create mode 100644 meta/classes-recipe/vendor_npm.bbclass > create mode 100644 meta/classes-recipe/vendor_npm_build.bbclass > create mode 100644 meta/lib/oe/vendor/__init__.py > create mode 100644 meta/lib/oe/vendor/cargo.py > create mode 100644 meta/lib/oe/vendor/go.py > create mode 100644 meta/lib/oe/vendor/npm.py > create mode 100644 meta/lib/oeqa/selftest/cases/oelib/vendor.py > delete mode 100644 meta/recipes-devtools/python/python3-bcrypt-crates.inc > delete mode 100644 > meta/recipes-devtools/python/python3-cryptography-crates.inc > delete mode 100644 meta/recipes-devtools/python/python3-maturin-crates.inc > delete mode 100644 meta/recipes-devtools/python/python3-rpds-py-crates.inc > delete mode 100644 meta/recipes-gnome/librsvg/librsvg-crates.inc > create mode 100644 > meta/recipes-gnome/librsvg/librsvg/0001-update-url-crate-to-get-an-updated-idna-rustsec-2024.patch > > -- > 2.39.5 > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#211198): https://lists.openembedded.org/g/openembedded-core/message/211198 Mute This Topic: https://lists.openembedded.org/mt/111133313/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
