CFT: repository for kernel modules
Hello everyone,
In our current release model the binary packages are built on the lowest
supported version of a given branch. Which means when 14.2, the packages it is
using are built on 14.1 until 14.1's EOL which is march 31st.
This works greate for everything but kernel modules, beside developers trying
hard to keep KBI stability in the life time of a branch, there are always plenty
of issues. It has been decided to provide a dedicated repository for 14.2 user
only containings kernel modules, built out of the ports tree.
I order to be able to do it quickly here is what have been done and the related
constraints.
First all the packages which are providing kernel modules now have automatically
the OSVERSION which is appended to the package version:
foo-kmod-1.3.4_3,1 now becomes foo-kmod-1.3.4.${OSVERSION}_3,1
The goal of this change is that the kernel module for 14.2 is at a higher
version from the kernel module from 14.1 which means will force reinstalling
considering this is an upgrade.
The other point of this change, it helps user on current and on stable to figure
out the need to rebuild thir kernel module at some point of their upgrades!
On the infrastructure side, all kernel modules are now fully build in an
unpriviledge clean chroot, and cross built for all possible architectures for
each new releases (currently only 14.2, 14.1 kernel modules being in the regular
repository).
This has the advantage of allowing us to get kernel modules for every single
architecture we do support quickly, the drawback being kernel modules with build
and/or run dependency cannot be built (note that for run dependency I am working
on a workaround, stay tuned).
in order to discover quickly which ports should be built, the following is being
done:
> git grep "^USES.*kmod" and ls */*-kmod
Anything mathing with the above is listed as to be built.
Because of the way we are building and because we want to be able to ship
quickly updates on those kernel modules, there are other limitation which have
been added:
Only kernel modules without any build and run dependencies are being built,
which means most of them but not all.
I plan to fix the missing ones.
Having kernel modules isolated in their own packages is anyway a good idea as it
helps people on stable and current quickly falling back on their feet when a
kernel update breaks: it is faster to rebuild, and require less ressources.
As a result you can see open-vm-tools has been split into open-vm-tools and
open-vm-kmod.
The day we have thing repository incorporated into poudriere we will be able to
revisit the way
To be able to use those new repositories:
this is the repository configuration for packages built our of the main branch
> https://people.freebsd.org/~bapt/kmods-latest.conf.txt
for completion we also started building on quarterly branches, but keep in mind
that all the above changes are only in main now and will be in quarterly only in
january
> https://people.freebsd.org/~bapt/kmods-quarterly.conf.txt
Best regards,
Bapt
Re: Unable to update repository FreeBSD-kmods
On Fri 13 Dec 13:35, Graham Perrin wrote:
> On 13/12/2024 08:09, Baptiste Daroussin wrote:
> > …
> >
> > > https://people.freebsd.org/~bapt/kmods-quarterly.conf.txt
> > Best regards,
> > Bapt
>
>
> Thank you!
>
>
> An early test result:
>
> root@fourteen-pkgbase:~ # ee /usr/local/etc/pkg/repos/FreeBSD-kmods.conf
>
> root@fourteen-pkgbase:~ # pkg -vv | grep -B 1 -e url -e priority
> FreeBSD-ports: {
> url :
> "pkg+https://pkg.freebsd.org/FreeBSD:14:amd64/quarterly";,
> enabled : yes,
> priority : 0,
> --
> FreeBSD-base: {
> url :
> "pkg+https://pkg.freebsd.org/FreeBSD:14:amd64/base_release_2";,
> enabled : yes,
> priority : 0,
> --
> FreeBSD-kmods: {
> url :
> "pkg+https://pkg.freebsd.org/FreeBSD:14:amd64/kmods_quarterly_${VERSION_VERSION}";,
> enabled : yes,
> priority : 0,
> --
> local-poudriere: {
> url :
> "file:///usr/local/poudriere/data/packages/fourteen-default",
> enabled : yes,
> priority : 3
> root@fourteen-pkgbase:~ # date ; uptime
> Fri Dec 13 12:12:50 GMT 2024
> 12:12PM up 7:52, 1 user, load averages: 0.15, 0.39, 0.43
> root@fourteen-pkgbase:~ # pkg update
> Updating FreeBSD-ports repository catalogue...
> FreeBSD-ports repository is up to date.
> Updating FreeBSD-base repository catalogue...
> FreeBSD-base repository is up to date.
> Updating FreeBSD-kmods repository catalogue...
> pkg: Repository FreeBSD-kmods has a wrong packagesite, need to re-create
> database
> Unable to update repository FreeBSD-kmods
> Updating local-poudriere repository catalogue...
> local-poudriere repository is up to date.
> Error updating repositories!
> root@fourteen-pkgbase:~ #
>
>
> Attached: debug output.
>
> Maybe relevant:
>
Unable to update repository FreeBSD-kmods
On 13/12/2024 08:09, Baptiste Daroussin wrote:
…
https://people.freebsd.org/~bapt/kmods-quarterly.conf.txt
Best regards,
Bapt
Thank you!
An early test result:
root@fourteen-pkgbase:~ # ee /usr/local/etc/pkg/repos/FreeBSD-kmods.conf
root@fourteen-pkgbase:~ # pkg -vv | grep -B 1 -e url -e priority
FreeBSD-ports: {
url :
"pkg+https://pkg.freebsd.org/FreeBSD:14:amd64/quarterly";,
enabled : yes,
priority : 0,
--
FreeBSD-base: {
url :
"pkg+https://pkg.freebsd.org/FreeBSD:14:amd64/base_release_2";,
enabled : yes,
priority : 0,
--
FreeBSD-kmods: {
url :
"pkg+https://pkg.freebsd.org/FreeBSD:14:amd64/kmods_quarterly_${VERSION_VERSION}";,
enabled : yes,
priority : 0,
--
local-poudriere: {
url :
"file:///usr/local/poudriere/data/packages/fourteen-default",
enabled : yes,
priority : 3
root@fourteen-pkgbase:~ # date ; uptime
Fri Dec 13 12:12:50 GMT 2024
12:12PM up 7:52, 1 user, load averages: 0.15, 0.39, 0.43
root@fourteen-pkgbase:~ # pkg update
Updating FreeBSD-ports repository catalogue...
FreeBSD-ports repository is up to date.
Updating FreeBSD-base repository catalogue...
FreeBSD-base repository is up to date.
Updating FreeBSD-kmods repository catalogue...
pkg: Repository FreeBSD-kmods has a wrong packagesite, need to re-create
database
Unable to update repository FreeBSD-kmods
Updating local-poudriere repository catalogue...
local-poudriere repository is up to date.
Error updating repositories!
root@fourteen-pkgbase:~ #
Attached: debug output.
Maybe relevant:
RE: CFT: repository for kernel modules [the kmods-*.conf.txt examples have VERSION_VERSION instead of VERSION_MINOR]
Baptiste Daroussin wrote on
Date: Fri, 13 Dec 2024 08:09:22 UTC:
. . .
> To be able to use those new repositories:
>
> this is the repository configuration for packages built our of the main branch
>
> > https://people.freebsd.org/~bapt/kmods-latest.conf.txt
that file has ${VERSION_VERSION} instead of using ${VERSION_MINOR}:
FreeBSD-kmods {
url: pkg+https://pkg.freebsd.org/${ABI}/kmods_latest_${VERSION_VERSION}
signature_type: "fingerprints"
fingerprints: "/usr/share/keys/pkg"
mirror_type: "srv"
enabled: yes
}
> for completion we also started building on quarterly branches, but keep in
> mind
> that all the above changes are only in main now and will be in quarterly only
> in
> january
>
> > https://people.freebsd.org/~bapt/kmods-quarterly.conf.txt
that file also has ${VERSION_VERSION} instead of using ${VERSION_MINOR}:
FreeBSD-kmods {
url:
pkg+https://pkg.freebsd.org/${ABI}/kmods_quarterly_${VERSION_VERSION}
signature_type: "fingerprints"
fingerprints: "/usr/share/keys/pkg"
mirror_type: "srv"
enabled: yes
}
===
Mark Millard
marklmi at yahoo.com
Re: CFT: repository for kernel modules
Success! With drm-61-kmod-6.1.92.1402000_3 I can kldload 915kms on FreeBSD 14.2. Before switching to this repo, kldload would hang. Also, in addition to kmods, there are a few other ports that must be rebuilt for every minor version. devel/py-libzfs is one. Could that be added to the new repository?
kmods_quarterly_${VERSION_MINOR} (was: Unable to update repository FreeBSD-kmods)
On 13/12/2024 13:39, Baptiste Daroussin wrote:
…
replace VERSION_VERSION with VERSION_MINOR
Best regards,
Bapt
Thank you.
Now: whilst the repository does update, successfully, a package
(drm-61-kmod) does not upgrade unless the one repository is specified
for the upgrade.
Should I experimentally raise the priority of FreeBSD-kmods?
(In a test that preceded your CFT, priority 5 seemed appropriate for my
case.)
root@fourteen-pkgbase:~ # history -S
root@fourteen-pkgbase:~ # ee /usr/local/etc/pkg/repos/FreeBSD-kmods.conf
root@fourteen-pkgbase:~ # pkg update -r FreeBSD-kmods
Updating FreeBSD-kmods repository catalogue...
pkg: Repository FreeBSD-kmods has a wrong packagesite, need to re-create
database
Fetching meta.conf: 100% 178 B 0.2kB/s 00:01
Fetching data.pkg: 100% 12 KiB 12.6kB/s 00:01
Processing entries: 100%
The provides database is up-to-date.
FreeBSD-kmods repository update completed. 43 packages processed.
All repositories are up to date.
root@fourteen-pkgbase:~ # pkg -vv | grep -B 1 -e url -e priority
FreeBSD-ports: {
url :
"pkg+https://pkg.freebsd.org/FreeBSD:14:amd64/quarterly";,
enabled : yes,
priority : 0,
--
FreeBSD-base: {
url :
"pkg+https://pkg.freebsd.org/FreeBSD:14:amd64/base_release_2";,
enabled : yes,
priority : 0,
--
FreeBSD-kmods: {
url :
"pkg+https://pkg.freebsd.org/FreeBSD:14:amd64/kmods_quarterly_2";,
enabled : yes,
priority : 0,
--
local-poudriere: {
url :
"file:///usr/local/poudriere/data/packages/fourteen-default",
enabled : yes,
priority : 3
root@fourteen-pkgbase:~ # grep url
/usr/local/etc/pkg/repos/FreeBSD-kmods.conf
url:
pkg+https://pkg.freebsd.org/${ABI}/kmods_quarterly_${VERSION_MINOR}
root@fourteen-pkgbase:~ # pkg delete -y drm-kmod ; pkg delete -y
drm-61-kmod
No packages matched for pattern 'drm-kmod'
Checking integrity... done (0 conflicting)
1 packages requested for removal: 0 locked, 1 missing
Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 1 packages (of 0
packages in the universe):
Installed packages to be REMOVED:
drm-61-kmod: 6.1.92.1402000_3
Number of packages to be removed: 1
The operation will free 17 MiB.
[1/1] Deinstalling drm-61-kmod-6.1.92.1402000_3...
[1/1] Deleting files for drm-61-kmod-6.1.92.1402000_3: 100%
root@fourteen-pkgbase:~ # pkg install --repo FreeBSD-ports -y --quiet
--no-repo-update drm-61-kmod
=
Message from drm-61-kmod-6.1.92:
--
The drm-61-kmod port can be enabled for amdgpu (for AMD
GPUs starting with the HD7000 series / Tahiti) or i915kms (for Intel
APUs starting with HD3000 / Sandy Bridge) through kld_list in
/etc/rc.conf. radeonkms for older AMD GPUs can be loaded and there are
some positive reports if EFI boot is NOT enabled.
For amdgpu: kld_list="amdgpu"
For Intel: kld_list="i915kms"
For radeonkms: kld_list="radeonkms"
Please ensure that all users requiring graphics are members of the
"video" group.
root@fourteen-pkgbase:~ # pkg upgrade -n
Updating FreeBSD-ports repository catalogue...
FreeBSD-ports repository is up to date.
Updating FreeBSD-base repository catalogue...
FreeBSD-base repository is up to date.
Updating FreeBSD-kmods repository catalogue...
FreeBSD-kmods repository is up to date.
Updating local-poudriere repository catalogue...
local-poudriere repository is up to date.
All repositories are up to date.
Checking for upgrades (4 candidates): 100%
Processing candidates (4 candidates): 100%
Checking integrity... done (0 conflicting)
Your packages are up to date.
root@fourteen-pkgbase:~ # pkg upgrade -n -r FreeBSD-kmods
Updating FreeBSD-kmods repository catalogue...
FreeBSD-kmods repository is up to date.
All repositories are up to date.
Checking for upgrades (1 candidates): 100%
Processing candidates (1 candidates): 100%
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):
Installed packages to be UPGRADED:
drm-61-kmod: 6.1.92 -> 6.1.92.1402000_3 [FreeBSD-kmods]
Number of packages to be upgraded: 1
root@fourteen-pkgbase:~ #
Re: CFT: repository for kernel modules
On Fri 13 Dec 07:24, Alan Somers wrote: > Success! With drm-61-kmod-6.1.92.1402000_3 I can kldload 915kms on > FreeBSD 14.2. Before switching to this repo, kldload would hang. > > Also, in addition to kmods, there are a few other ports that must be > rebuilt for every minor version. devel/py-libzfs is one. Could that > be added to the new repository? Right now and until we have a thin repository support in poudriere: no :(. One of the limitation is everything is cross build from amd64 so I cannot get much things in that repo considering that in 2024 perl is still not cross build friendly and last I checked python wasn't either. Best regards, Bapt
Re: CFT: repository for kernel modules
On Fri, Dec 13, 2024 at 6:28 AM Baptiste Daroussin wrote: > On Fri 13 Dec 07:24, Alan Somers wrote: > > Success! With drm-61-kmod-6.1.92.1402000_3 I can kldload 915kms on > > FreeBSD 14.2. Before switching to this repo, kldload would hang. > > > > Also, in addition to kmods, there are a few other ports that must be > > rebuilt for every minor version. devel/py-libzfs is one. Could that > > be added to the new repository? > > Right now and until we have a thin repository support in poudriere: no :(. > > One of the limitation is everything is cross build from amd64 so I cannot > get > much things in that repo considering that in 2024 perl is still not cross > build > friendly and last I checked python wasn't either. > > Best regards, > Bapt > What about 'pkg version'? I am running main from late October (soon to be updated), but 'pkg version' shows all three of my -kmod ports out of date (1500026 vs. 1500029), thigh "updating" them simply reinstalls the existing build (26), as it should. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkober...@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683
