> Most of provisioning scripts fall into 2 categories: > 1) Download installer (or other binary package) of 3rd party stuff, verify > checksum, install. These scripts are quite simple. > 2) Build stuff from sources in provisioning time, sometimes in a > sophisticated way (e.g. > https://code.qt.io/cgit/qt/qt5.git/tree/coin/provisioning/common/windows/android-openssl.ps1). > These can be (and IMO should be) replaced with Conan, to reduce provisioning > time and complexity.
Of course, it's also possible to use conan packages for category (1), however this may require writing quite a few custom recipes (see manual [1]), possibly containing more boilerplate than existing ps1 snippets. However, this would allow to have a nice list of all provisioned packages with their versions in one place. [1] https://docs.conan.io/en/latest/devtools/create_installer_packages.html#create-installer-packages That is what the automatic documentation tools are asking. Might be stupid questions due to the knowledge. We would install all dependencies during the provisioning and during the builds conan only exposes dependencies listed in the conanfile? One thing that conflicts a little bit here is that we want to use cached stuff from our servers instead of downloading from the internet in the provisioning and also have the provisioning to be runnable by anyone, but i guess that if the conan caches all the packages then it equals pretty much the same. >> As for provisioning and why the CI images are different from what are >> documented, biggest problem there is that the images are used by other >> components also e.g. RTA testing. Those require some additional stuff and >> provisioning currently installs everything. AFAIK also all the optional >> dependencies are installed, this makes it hard to distinguish between them >> and required dependencies because both are always present. These make it >> easy that new dependencies etc. slip through unnoticed. > > Note that when Conan packages are used via conanfiles (i.e. as it's > originally intended by Conan developers), there cannot be any dependency > slips because Conan dependencies don't become visible to any project which > doesn't require them via conanfile (directly or indirectly). In this case each testing activity should have its own conanfile with a list of dependencies, as well as each Qt project which uses them for building. Ultimately where would all the conanfiles sit? Currently we only use qt5 repository during provisioning and with conan each repo would have its own conanfiles which needs to be read when running provisioning? How big maintenance burden those conanfiles become, are common files across repos possible? I do not see why it would be impossible to use conan in provisioning, the transition is quite big effort though, and some cherry-picks need to be translated to the old format afterwards. Theres also many other things I cannot answer now related to artifactory and any legal sides to distributing the packages etc. There seems to be a lot of research into conan by Iikka, but i cannot find any related to building all the qt modules with it. I am sure it requires developer attention also, related to the CMake changes that needs to be done. It has a low priority now. https://bugreports.qt.io/browse/QTBUG-74397 ________________________________ From: Konstantin Tokarev <[email protected]> Sent: Wednesday, October 7, 2020 6:44 PM To: Toni Saario <[email protected]>; Iikka Eklund <[email protected]>; Richard Weickelt <[email protected]>; [email protected] <[email protected]> Subject: Re: [Development] Building additional components with Conan for Qt 6 02.10.2020, 16:20, "Konstantin Tokarev" <[email protected]>: > 02.10.2020, 15:53, "Toni Saario" <[email protected]>: >> One thing to check is that how much the provisioning it is possible to >> translate to Conan, I would believe that in the end it does not work for >> everything. > > Most of provisioning scripts fall into 2 categories: > 1) Download installer (or other binary package) of 3rd party stuff, verify > checksum, install. These scripts are quite simple. > 2) Build stuff from sources in provisioning time, sometimes in a > sophisticated way (e.g. > https://code.qt.io/cgit/qt/qt5.git/tree/coin/provisioning/common/windows/android-openssl.ps1). > These can be (and IMO should be) replaced with Conan, to reduce provisioning > time and complexity. Of course, it's also possible to use conan packages for category (1), however this may require writing quite a few custom recipes (see manual [1]), possibly containing more boilerplate than existing ps1 snippets. However, this would allow to have a nice list of all provisioned packages with their versions in one place. [1] https://docs.conan.io/en/latest/devtools/create_installer_packages.html#create-installer-packages >> As for provisioning and why the CI images are different from what are >> documented, biggest problem there is that the images are used by other >> components also e.g. RTA testing. Those require some additional stuff and >> provisioning currently installs everything. AFAIK also all the optional >> dependencies are installed, this makes it hard to distinguish between them >> and required dependencies because both are always present. These make it >> easy that new dependencies etc. slip through unnoticed. > > Note that when Conan packages are used via conanfiles (i.e. as it's > originally intended by Conan developers), there cannot be any dependency > slips because Conan dependencies don't become visible to any project which > doesn't require them via conanfile (directly or indirectly). In this case each testing activity should have its own conanfile with a list of dependencies, as well as each Qt project which uses them for building. > > I can see another possible way to use Conan in CI - install packages into the > system via "deploy" generator so that Qt build system can find them without > any knowledge of Conan. I guess this way may be called "Ossi-friendly because > Ossi was adamantly opposed to any explicit support of Conan in build system, > see e.g. https://codereview.qt-project.org/c/qt/qtbase/+/184241, however this > way indeed may cause dependency slips. -- Regards, Konstantin
_______________________________________________ Development mailing list [email protected] https://lists.qt-project.org/listinfo/development
