On Thu, 2021-09-23 at 21:41 +0000, Peter Kjellerstedt wrote:
> > -----Original Message-----
> > From: openembedded-core@lists.openembedded.org <openembedded-
> > c...@lists.openembedded.org> On Behalf Of Richard Purdie
> > Sent: den 20 september 2021 14:46
> > To: openembedded-core@lists.openembedded.org
> > Subject: [OE-core] [RFC PATCH 10/14] package_ipk/deb/rpm: Drop recursive
> > do_build task dependencies
> >
> > This is a controversial change which removes the recursive dependencies
> > from the do_build target of packaging tasks of recipes.
> >
> > Currently this means when you "bitbake <image>" or "bitbake <recipe>",
> > the packaging tasks run for all packaging backends enabled for all recipes
> > in the dependency chain. The same therefore then applies to images.
> >
> > We don't actually need that, it is a convinience thing. Removing it
> > massively simplifies the task graph and causes much fewer tasks to execute
> > in many common scenarios. It also means less sstate is fetched for
> > example when building an image.
> >
> > This means when building a recipe all package formats would be built
> > but when building an image, only the format used by the image would be
> > generated. This should be an improvement in most cases but some CI systems
> > may need to be explict about what they're building.
> >
> > Signed-off-by: Richard Purdie <richard.pur...@linuxfoundation.org>
> > ---
> > meta/classes/package_deb.bbclass | 5 +----
> > meta/classes/package_ipk.bbclass | 4 +---
> > meta/classes/package_rpm.bbclass | 4 +---
> > 3 files changed, 3 insertions(+), 10 deletions(-)
> >
> > diff --git a/meta/classes/package_deb.bbclass
> > b/meta/classes/package_deb.bbclass
> > index eca43e17876..77caaa55c8f 100644
> > --- a/meta/classes/package_deb.bbclass
> > +++ b/meta/classes/package_deb.bbclass
> > @@ -315,10 +315,7 @@ python do_package_write_deb () {
> > do_package_write_deb[dirs] = "${PKGWRITEDIRDEB}"
> > do_package_write_deb[cleandirs] = "${PKGWRITEDIRDEB}"
> > do_package_write_deb[depends] +=
> > "${@oe.utils.build_depends_string(d.getVar('PACKAGE_WRITE_DEPS'),
> > 'do_populate_sysroot')}"
> > -addtask package_write_deb after do_packagedata do_package
> > -
> > +addtask package_write_deb after do_packagedata do_package before do_build
> >
> > PACKAGEINDEXDEPS += "dpkg-native:do_populate_sysroot"
> > PACKAGEINDEXDEPS += "apt-native:do_populate_sysroot"
> > -
> > -do_build[recrdeptask] += "do_package_write_deb"
> > diff --git a/meta/classes/package_ipk.bbclass
> > b/meta/classes/package_ipk.bbclass
> > index c3b53854e8b..998e18ba6c1 100644
> > --- a/meta/classes/package_ipk.bbclass
> > +++ b/meta/classes/package_ipk.bbclass
> > @@ -274,9 +274,7 @@ python do_package_write_ipk () {
> > do_package_write_ipk[dirs] = "${PKGWRITEDIRIPK}"
> > do_package_write_ipk[cleandirs] = "${PKGWRITEDIRIPK}"
> > do_package_write_ipk[depends] +=
> > "${@oe.utils.build_depends_string(d.getVar('PACKAGE_WRITE_DEPS'),
> > 'do_populate_sysroot')}"
> > -addtask package_write_ipk after do_packagedata do_package
> > +addtask package_write_ipk after do_packagedata do_package before do_build
> >
> > PACKAGEINDEXDEPS += "opkg-utils-native:do_populate_sysroot"
> > PACKAGEINDEXDEPS += "opkg-native:do_populate_sysroot"
> > -
> > -do_build[recrdeptask] += "do_package_write_ipk"
> > diff --git a/meta/classes/package_rpm.bbclass
> > b/meta/classes/package_rpm.bbclass
> > index 88d861c0e75..60f8299d965 100644
> > --- a/meta/classes/package_rpm.bbclass
> > +++ b/meta/classes/package_rpm.bbclass
> > @@ -748,9 +748,7 @@ python do_package_write_rpm () {
> > do_package_write_rpm[dirs] = "${PKGWRITEDIRRPM}"
> > do_package_write_rpm[cleandirs] = "${PKGWRITEDIRRPM}"
> > do_package_write_rpm[depends] +=
> > "${@oe.utils.build_depends_string(d.getVar('PACKAGE_WRITE_DEPS'),
> > 'do_populate_sysroot')}"
> > -addtask package_write_rpm after do_packagedata do_package
> > +addtask package_write_rpm after do_packagedata do_package before do_build
> >
> > PACKAGEINDEXDEPS += "rpm-native:do_populate_sysroot"
> > PACKAGEINDEXDEPS += "createrepo-c-native:do_populate_sysroot"
> > -
> > -do_build[recrdeptask] += "do_package_write_rpm"
> > --
> > 2.32.0
>
> I just wanted to give some positive feedback now that this has been
> integrated. This patch alone reduced the number of edges in the task
> graph by 50% for one of our typical products (from 127721 to 64785).
> That is pretty impressive!
>
> And to add to this, the later patch that removes the use of the meta
> class reduced the number of nodes in the task graph by 10% (from 17122
> to 15313). Also pretty impressive for such a small change.
Thanks, it is nice to hear something working out well like that! :)
I've been working towards this for a while but we've never been quite ready for
the final pieces until now.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#156279):
https://lists.openembedded.org/g/openembedded-core/message/156279
Mute This Topic: https://lists.openembedded.org/mt/85739581/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
