-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi Richard and folks,
> The only other way I could imagine this to work would require us to > keep the manifest file within the meta layer and do a somewhat > "hacky" > workaround: > > It is possible to run `repo init` on a local git folder. > As such, something like this could work: > > 1) create temp folder (e.g. /tmp/repo-XXXXXXXXX) > 2) run git init in temp folder > 3) install manifest file into temp folder > 4) add & commit > 5) run repo init -u file://$temp-folder today I discovered that repo also supports the --standalone-manifest flag, which makes the workaround described above obsolete. Thus, I believe supporting repo manifests from the layer repo only is the cleanest and easiest way to implement this and we wouldn't have to rely on cloning manifest repos during recipe parsing. However, this would be a breaking change to the current repo fetcher implementation, as it supports remote manifest repos. I am pretty confident however that no one uses the arguably already broken and unusable current repo fetcher implementation in a production workload so this shouldn't be a major issue. What are your thoughts on this? - -- With best regards Jasper Orschulko DevOps Engineer Tel. +49 30 58 58 14 265 Fax +49 30 58 58 14 999 jasper.orschu...@iris-sensing.com • • • • • • • • • • • • • • • • • • • • • • • • • • iris-GmbH infrared & intelligent sensors Schnellerstraße 1-5 | 12439 Berlin https://iris-sensing.com/ On Thu, 2021-11-11 at 12:42 +0100, Jasper Orschulko wrote: > Hi Richard, > > > Even ten recipes using this will show a degradation in parsing > > speed > > and I do > > get a lot of complaints when parsing slows down for any reason. The > > user doesn't > > expect this and it won't be visible what bitbake is doing (sitting > > at > > 99% parsed > > for a period). > > > > Also, the "tend to be small" implies someone will create a huge one > > at some > > point even if that is a bad idea for whatever reasons, I just know > > how these > > things end up going :(. > > I believe this is something that could be addressed in the > documentation of the repo fetcher? Something along the lines of > "Using > the repo fetcher can increase the time spent in the parsing phase, as > we need to clone and inspect the manifest repository. To avoid this, > keep the size of your manifest repository at a minimum." > > When this is done, the dalay is barely noticable. We have a testsetup > with 24 recipes and reasonable parsing speeds. > > > * Does it only clone a repo in the AUTOREV case? > > No, it also clones with fixed refspecs, as we want to ensure that > each > of the git repos referenced in the manifest also uses fixed refspecs. > Otherwise, the sources in the background might change, while the > recipe > still (correctly) references the same revision of the manifest repo. > > > * Could it only obtain the manifest file somehow without a clone of > > the repo? > > Unfortunately not really. git does not offer a way to list the > content > of a file without cloning it beforehand. > > The only other way I could imagine this to work would require us to > keep the manifest file within the meta layer and do a somewhat > "hacky" > workaround: > > It is possible to run `repo init` on a local git folder. > As such, something like this could work: > > 1) create temp folder (e.g. /tmp/repo-XXXXXXXXX) > 2) run git init in temp folder > 3) install manifest file into temp folder > 4) add & commit > 5) run repo init -u file://$temp-folder > > The repo manifest could even still be used outside of yocto, by using > the metalayer repo as basis for the repo init: `repo init -u > $metalayer-repo -m $path-to-manifest-file`. Not pretty, but it works. > > So if cloning within the parser is a dealbreaker for you, this option > might be worth looking into. > -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEE4WyPMIC5Ap4+Ooo1Ygqew07VMNUFAmGeYnwACgkQYgqew07V MNWXFAgAkZ7+CRZadUItM+Ay8uePffysuQhlwxHaAgzx1H+4Pk/HKK9QpEy3jFWz kODr3DPOWlf6eS+AcqpJXVYdMsTucvE34eLIWb8reFkrPQKMtxp0N6U5gqz7e+Bm Wf4lS04tqGZzC/Ld2DOs/zEnGKEwXBRHMX/9j6YvVMYxwKjxghW2LRloWpzghduH j0FTAp1aGKLhCQjqeEuNZRMT+rn0i8k6u8Lxel1wRHi2LNhyT3VD3NThjVL64acH FF83jhaxQ0JxpsSn9ouvJD3wMJHi72rdfGzznuQHmFT0lOxO5AjlcIOtK3KisiAI KQ/jEV/44WN0JO7OpmHrVyOSfPeVBA== =H0FZ -----END PGP SIGNATURE-----
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#158723): https://lists.openembedded.org/g/openembedded-core/message/158723 Mute This Topic: https://lists.openembedded.org/mt/86840389/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-