On Wed, 18 Oct 2023 at 08:45, Richard Purdie <richard.pur...@linuxfoundation.org> wrote: > I'm torn on the targets to test as sato-sdk is a large image and world > is a lot of work. I'd be tempted to test sato, weston and full-cmdline? > World is a good test I guess and if from sstate, shouldn't have that > much of an issue. It does also prove things are working.
I ran '-S printdiff world' on a blank build directory. First, scalability isn't great: Initialising tasks: 100% |##########################################################################################################################################################################| Time: 0:24:19 Checking sstate mirror object availability: 100% |##################################################################################################################################################| Time: 0:12:14 So it's taking 36 minutes just preparing to fetch the objects, and 2/3 of that time goes into communicating with hash equivalence server (e.g. BB_HASHSERVE_UPSTREAM = "hashserv.yocto.io:8687"). Second, there are significant misses. I don't have a clear theory where they come from, just want to list them: The differences between the current build and any cached tasks start at the following tasks: /srv/work/alex/poky/meta/recipes-core/meta/meta-ide-support.bb:do_populate_ide_support /srv/work/alex/poky/meta/recipes-graphics/xorg-driver/xf86-input-synaptics_1.9.2.bb:do_collect_spdx_deps /srv/work/alex/poky/meta/recipes-devtools/bootchart2/bootchart2_0.14.9.bb:do_create_runtime_spdx /srv/work/alex/poky/meta/recipes-graphics/igt-gpu-tools/igt-gpu-tools_git.bb:do_collect_spdx_deps /srv/work/alex/poky/meta/recipes-devtools/python/python3-dbusmock_0.29.1.bb:do_create_runtime_spdx /srv/work/alex/poky/meta/recipes-devtools/dpkg/dpkg_1.22.0.bb:do_configure /srv/work/alex/poky/meta/recipes-graphics/xorg-driver/xf86-input-vmmouse_13.2.0.bb:do_collect_spdx_deps /srv/work/alex/poky/meta/recipes-graphics/glew/glew_2.2.0.bb:do_collect_spdx_deps /srv/work/alex/poky/meta/recipes-graphics/xorg-driver/xf86-input-evdev_2.10.6.bb:do_collect_spdx_deps /srv/work/alex/poky/meta/recipes-graphics/libva/libva_2.19.0.bb:do_collect_spdx_deps /srv/work/alex/poky/meta/recipes-sato/webkit/libwpe_1.14.1.bb:do_collect_spdx_deps /srv/work/alex/poky/meta/recipes-multimedia/gstreamer/gstreamer1.0-meta-base.bb:do_collect_spdx_deps /srv/work/alex/poky/meta/recipes-multimedia/gstreamer/gstreamer1.0-python_1.22.6.bb:do_collect_spdx_deps /srv/work/alex/poky/meta/recipes-extended/cups/cups_2.4.6.bb:do_collect_spdx_deps /srv/work/alex/poky/meta/recipes-gnome/libdazzle/libdazzle_3.44.0.bb:do_collect_spdx_deps /srv/work/alex/poky/meta/recipes-graphics/igt-gpu-tools/igt-gpu-tools_git.bb:do_package /srv/work/alex/poky/meta/recipes-multimedia/gstreamer/gst-devtools_1.22.6.bb:do_collect_spdx_deps /srv/work/alex/poky/meta/recipes-core/packagegroups/packagegroup-core-sdk.bb:do_package /srv/work/alex/poky/meta/recipes-graphics/xorg-driver/xf86-input-mouse_1.9.5.bb:do_collect_spdx_deps /srv/work/alex/poky/meta/recipes-graphics/waffle/waffle_1.7.2.bb:do_collect_spdx_deps /srv/work/alex/poky/meta/recipes-multimedia/alsa/alsa-tools_1.2.5.bb:do_collect_spdx_deps /srv/work/alex/poky/meta/recipes-multimedia/gstreamer/gstreamer1.0-rtsp-server_1.22.6.bb:do_collect_spdx_deps /srv/work/alex/poky/meta/recipes-devtools/apt/apt_2.6.1.bb:do_install /srv/work/alex/poky/meta/recipes-gnome/gtk+/gtk4_4.12.3.bb:do_collect_spdx_deps /srv/work/alex/poky/meta/recipes-devtools/devel-config/distcc-config.bb:do_create_runtime_spdx /srv/work/alex/poky/meta/recipes-gnome/libhandy/libhandy_1.8.2.bb:do_collect_spdx_deps /srv/work/alex/poky/meta/recipes-support/vim/vim_9.0.bb:do_collect_spdx_deps So I think we should start with specific images first - minimal, full-cmdline, weston, sato and sato-sdk are all much faster to check. On qemux86_64 none of them show misses, but on qemuarm64 there are problems with sato, sato-sdk and weston, i.e. sato-sdk shows: The differences between the current build and any cached tasks start at the following tasks: /srv/work/alex/poky/meta/recipes-gnome/gdk-pixbuf/gdk-pixbuf_2.42.10.bb:do_package_write_rpm /srv/work/alex/poky/meta/recipes-connectivity/connman/connman-gnome_0.7.bb:do_package_write_rpm /srv/work/alex/poky/meta/recipes-multimedia/gstreamer/gst-examples_1.18.6.bb:do_package_write_rpm /srv/work/alex/poky/meta/recipes-sato/images/core-image-sato-sdk.bb:do_deploy_source_date_epoch /srv/work/alex/poky/meta/recipes-graphics/xorg-font/font-util_1.4.1.bb:do_package_write_rpm /srv/work/alex/poky/meta/recipes-core/packagegroups/packagegroup-core-sdk.bb:do_package /srv/work/alex/poky/meta/recipes-gnome/librsvg/librsvg_2.56.3.bb:do_package_write_rpm /srv/work/alex/poky/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_1.22.6.bb:do_package_write_rpm I'm not sure if that's because cache for the current master needs to be populated properly first, or if there's a deeper issue. > Either we start logging what we've built so we get the last known > revisions, or we run the test as part of a-quick/a-full at the end of > the build? I don't really want to extend the build but I'm not sure we > may have much choice. I think it can be a selftest and bundled into trigger-build-posttrigger which already runs 'buildoptions.SourceMirroring.test_yocto_source_mirror': https://git.yoctoproject.org/yocto-autobuilder-helper/tree/config.json#n465 The various yocto mirrors tests could be tagged in a way similar to machine/toolchain selftests, so that they're excluded from regular oe-selftest, but run specifically here. Alex
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61398): https://lists.yoctoproject.org/g/yocto/message/61398 Mute This Topic: https://lists.yoctoproject.org/mt/101525879/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
