buildworld on CPU-A, installworld on CPU-B ends up with SIGILL
Hello. I want to build kernel and world (of FreeBSD 13.0-CURRENT) on a fast virtual machine for other ones (all the virtual machines are hosted on VMware EXSi hypervisors, which have different physical CPUs). After `make -j16 buildworld` has finished successfully on the build machine, I get there, for example, /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin/install program having the shlxq instruction (one from the BMI2 instruction set extensions). This eventually causes make installkernel and make installworld to fail with SIGILL on a virtual machine which must consume built world and kernel, and which is hosted on another ESXi instance, with older physical CPU (read: with CPU not implementing shlxq). The build machine (FreeBSD 13.0-CURRENT r354802) builds (x)install using the following commands (a part of buildworld): $ cc -O2 -pipe -O2 -march=x86-64 -pipe -I/usr/src/contrib/mtree -I/usr/src/lib/libnetbsd -DHAVE_STRUCT_STAT_ST_FLAGS=1 -MD -MF.depend.xinstall.o -MTxinstall.o -std=gnu99 -Wno-format-zero-length -Qunused-arguments -I/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/include -c /usr/src/usr.bin/xinstall/xinstall.c -o xinstall.o $ cc -O2 -pipe -O2 -march=x86-64 -pipe -I/usr/src/contrib/mtree -I/usr/src/lib/libnetbsd -DHAVE_STRUCT_STAT_ST_FLAGS=1 -MD -MF.depend.getid.o -MTgetid.o -std=gnu99 -Wno-format-zero-length -Qunused-arguments -I/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/include -c /usr/src/contrib/mtree/getid.c -o getid.o $ cc -O2 -pipe -O2 -march=x86-64 -pipe -I/usr/src/contrib/mtree -I/usr/src/lib/libnetbsd -DHAVE_STRUCT_STAT_ST_FLAGS=1 -std=gnu99 -Wno-format-zero-length -Qunused-arguments -I/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/include -static -L/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/lib -o xinstall xinstall.o getid.o -L/usr/obj/usr/src/amd64.amd64/tmp/obj-tools/lib/libmd -lmd -legacy This produces xintstall with `shlxq`s: $ llvm-objdump --disassemble xinstall | grep -c shlxq 135 I believe statically linked /usr/lib/libmd.a is a stuff which brings `shlxq`s into the xinstall. I didn't examine it further, sorry... My question is: is it possible to buildworld without issuing instructions which are native for the host CPU? I have neither /etc/make.conf, nor /etc/src.conf on the build machine. Is it possible at all for FreeBSD CURRENT to be built outside a host and installed on the host later? Just as a reference: My build machine has Intel(R) Xeon(R) Gold 6150 CPU that supports BMI2: # cpucontrol -i 7 /dev/cpuctl0 cpuid level 0x7: 0x 0xd19f6ffb 0x0018 0xbc00 (Bit 08 in EBX is set) , and a consuming machine has Intel(R) Xeon(R) CPU E5-4617 CPU that doesn't support BMI2: # cpucontrol -i 7 /dev/cpuctl0 cpuid level 0x7: 0x 0x0002 0x 0xbc00 (Bit 08 in EBX is unset). Both machines install kernel and world without any problem, if they were built locally. Thanks! -- Yours truly, Ruslan Garipov . -- MinGW-w64 developer, with write after approval GEDKeeper collaborator ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: buildworld on CPU-A, installworld on CPU-B ends up with SIGILL
On 11/25/2019 10:30 PM, Miroslav Lachman wrote: > Ruslan Garipov wrote on 2019/11/25 15:06: >> Hello. >> >> I want to build kernel and world (of FreeBSD 13.0-CURRENT) on a fast >> virtual machine for other ones (all the virtual machines are hosted on >> VMware EXSi hypervisors, which have different physical CPUs). >> >> After `make -j16 buildworld` has finished successfully on the build >> machine, I get there, for example, >> /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin/install program having the >> shlxq instruction (one from the BMI2 instruction set extensions). This >> eventually causes make installkernel and make installworld to fail with >> SIGILL on a virtual machine which must consume built world and kernel, >> and which is hosted on another ESXi instance, with older physical CPU >> (read: with CPU not implementing shlxq). >> >> The build machine (FreeBSD 13.0-CURRENT r354802) builds (x)install using >> the following commands (a part of buildworld): >> >> $ cc -O2 -pipe -O2 -march=x86-64 -pipe -I/usr/src/contrib/mtree >> -I/usr/src/lib/libnetbsd -DHAVE_STRUCT_STAT_ST_FLAGS=1 -MD >> -MF.depend.xinstall.o -MTxinstall.o -std=gnu99 -Wno-format-zero-length >> -Qunused-arguments -I/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/include >> -c /usr/src/usr.bin/xinstall/xinstall.c -o xinstall.o >> $ cc -O2 -pipe -O2 -march=x86-64 -pipe -I/usr/src/contrib/mtree >> -I/usr/src/lib/libnetbsd -DHAVE_STRUCT_STAT_ST_FLAGS=1 -MD >> -MF.depend.getid.o -MTgetid.o -std=gnu99 -Wno-format-zero-length >> -Qunused-arguments -I/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/include >> -c /usr/src/contrib/mtree/getid.c -o getid.o >> $ cc -O2 -pipe -O2 -march=x86-64 -pipe -I/usr/src/contrib/mtree >> -I/usr/src/lib/libnetbsd -DHAVE_STRUCT_STAT_ST_FLAGS=1 -std=gnu99 >> -Wno-format-zero-length -Qunused-arguments >> -I/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/include -static >> -L/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/lib -o xinstall xinstall.o >> getid.o -L/usr/obj/usr/src/amd64.amd64/tmp/obj-tools/lib/libmd -lmd -legacy >> >> This produces xintstall with `shlxq`s: >> >> $ llvm-objdump --disassemble xinstall | grep -c shlxq >> 135 >> >> I believe statically linked /usr/lib/libmd.a is a stuff which brings >> `shlxq`s into the xinstall. I didn't examine it further, sorry... >> >> My question is: is it possible to buildworld without issuing >> instructions which are native for the host CPU? I have neither >> /etc/make.conf, nor /etc/src.conf on the build machine. Is it possible >> at all for FreeBSD CURRENT to be built outside a host and installed on >> the host later? >> >> Just as a reference: >> >> My build machine has Intel(R) Xeon(R) Gold 6150 CPU that supports BMI2: >> >> # cpucontrol -i 7 /dev/cpuctl0 >> cpuid level 0x7: 0x 0xd19f6ffb 0x0018 0xbc00 >> >> (Bit 08 in EBX is set) >> >> , and a consuming machine has Intel(R) Xeon(R) CPU E5-4617 CPU that >> doesn't support BMI2: >> >> # cpucontrol -i 7 /dev/cpuctl0 >> cpuid level 0x7: 0x 0x0002 0x 0xbc00 >> >> (Bit 08 in EBX is unset). >> >> Both machines install kernel and world without any problem, if they were >> built locally. > > I didn't tried this with current but I am using it with stable (11.3 at > this time). Building on Xeon E3-1240v3 and installing on many different > machines. Some of them are 10+ years old AMD Opteron, some Xeon E5649, > some 10 years old Intel Pentium. > So at least it worked in the past (11.3 amd64). Did you use this > workflow in the past / did it work? No, unfortunately I didn't. Always built world/kernel on target host. > I remember some issue in the past which was (accidentaly?) fixed by > running "make buildworld && make builkernel && make installkernel && > make installworld" on the build host (to some different DESTDIR) and > then "make installkernel && make installworld" on the target host (build > machine is shared via NFS) Therefore, this trick somehow "fixes" /usr/obj shared on the build machine? I'll try this later. Thanks! > > Miroslav Lachman > ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: buildworld on CPU-A, installworld on CPU-B ends up with SIGILL
On 11/26/2019 12:09 AM, Miroslav Lachman wrote: > Ruslan Garipov wrote on 2019/11/25 19:26: > > [...] > >>> I didn't tried this with current but I am using it with stable (11.3 at >>> this time). Building on Xeon E3-1240v3 and installing on many different >>> machines. Some of them are 10+ years old AMD Opteron, some Xeon E5649, >>> some 10 years old Intel Pentium. >>> So at least it worked in the past (11.3 amd64). Did you use this >>> workflow in the past / did it work? >> No, unfortunately I didn't. Always built world/kernel on target host. >> >>> I remember some issue in the past which was (accidentally?) fixed by >>> running "make buildworld && make builkernel && make installkernel && >>> make installworld" on the build host (to some different DESTDIR) and >>> then "make installkernel && make installworld" on the target host (build >>> machine is shared via NFS) >> Therefore, this trick somehow "fixes" /usr/obj shared on the build >> machine? I'll try this later. Thanks! > > Yes, I think so. But I am not a developer nor I know much about how > build process works. I've tried to installkernel/installworld with non-default DESTDIR, it doesn't help for GENERIC kernel. But... After I had failed with that DESTDIR, I decided to update kernel/world on the build machine to the revision I tried to deploy (r355085). I cleaned result of the previous build, restored make.conf and src.conf specifying sandybridge† there as value of CPUTYPE and explicit -march, build and install kernel and world. Then I cleaned result of the build again, run buildworld and buildkernel preparing to investigate the build logs. But before doing that, I decided to run `make installkernel` once again on a target machine, and ... bang! It completed successfully! mergemaster, installworld -- the same! Everything completed smoothly. I updated the source to r355105 on the build machine, buildworld and buildkernel there -- installkernel, mergemaster, installworld on a target machine completed with no errors. To be honest, I don't know what exactly was a reason for my previous failure. Because I've tried to build with sandybridge in the configs. May be r355085 helped me, I don't know. In any case, I should inspect the build log, I believe. Miroslav, thanks for support! † This is the oldest chip we have on our ESXi hosts. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
buildworld failed (usr.bin/kyua)
I failed to update FreeBSD 13.0-CURRENT amd64 r359231 to r359351. End of the build log: $ su root -c "make -j16 buildworld" ... ld: error: unable to find library -lkyua_cli_pie ld: error: unable to find library -lkyua_drivers_pie ld: error: unable to find library -lkyua_model_pie ld: error: unable to find library -llutok_pie ld: error: unable to find library -lkyua_engine_pie ld: error: unable to find library -llutok_pie ld: error: unable to find library -lkyua_utils_pie ld: error: unable to find library -llutok_pie ld: error: unable to find library -lkyua_store_pie ld: error: unable to find library -lkyua_model_pie ld: error: unable to find library -llutok_pie ld: error: unable to find library -lkyua_utils_pie ld: error: unable to find library -llutok_pie ld: error: unable to find library -lkyua_engine_pie ld: error: unable to find library -llutok_pie ld: error: unable to find library -lkyua_utils_pie ld: error: unable to find library -llutok_pie ld: error: unable to find library -lkyua_model_pie ld: error: unable to find library -llutok_pie ld: error: unable to find library -lkyua_store_pie ld: error: too many errors emitted, stopping now (use -error-limit=0 to see all errors) c++: error: linker command failed with exit code 1 (use -v to see invocation) --- all_subdir_lib --- --- cpuset_getdomain.po --- --- all_subdir_usr.bin --- *** [kyua] Error code 1 make[4]: stopped in /usr/src/usr.bin/kyua 1 error make[4]: stopped in /usr/src/usr.bin/kyua *** [all_subdir_usr.bin/kyua] Error code 2 ... May be it's related to r359260[1]. Therefore, here is my TEST-settings: $ fgrep TEST /etc/src.conf WITHOUT_GOOGLETEST= WITHOUT_TESTS= WITH_TESTS_SUPPORT= Also what has confused me: it's a virtual machine which failed to build. A physical one built userland and kernel just fine. Both the physical and virtual machines have almost the same (differ only by CPU "selection" options) make.conf and src.conf and different kernel configurations. Both the machines were FreeBSD 13.0-CURRENT amd64 r359231. I've started to build on clean systems (no /usr/obj at all). Can anyone help me to resolve this issue? I apologize in advance if I will delay my further replies, due to world wide health situation. Thanks! [1] https://lists.freebsd.org/pipermail/svn-src-head/2020-March/134800.html ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: buildworld failed (usr.bin/kyua)
On 3/28/2020 4:29 AM, Brooks Davis wrote:
> On Fri, Mar 27, 2020 at 08:07:08PM +, Brooks Davis wrote:
>> On Fri, Mar 27, 2020 at 06:40:18PM +0500, Ruslan Garipov wrote:
>>> I failed to update FreeBSD 13.0-CURRENT amd64 r359231 to r359351.
>>>
>>> End of the build log:
>>>
>>> $ su root -c "make -j16 buildworld"
>>> ...
>>> ld: error: unable to find library -lkyua_cli_pie
>>> ld: error: unable to find library -lkyua_drivers_pie
>>> ld: error: unable to find library -lkyua_model_pie
>>> ld: error: unable to find library -llutok_pie
>>> ld: error: unable to find library -lkyua_engine_pie
>>> ld: error: unable to find library -llutok_pie
>>> ld: error: unable to find library -lkyua_utils_pie
>>> ld: error: unable to find library -llutok_pie
>>> ld: error: unable to find library -lkyua_store_pie
>>> ld: error: unable to find library -lkyua_model_pie
>>> ld: error: unable to find library -llutok_pie
>>> ld: error: unable to find library -lkyua_utils_pie
>>> ld: error: unable to find library -llutok_pie
>>> ld: error: unable to find library -lkyua_engine_pie
>>> ld: error: unable to find library -llutok_pie
>>> ld: error: unable to find library -lkyua_utils_pie
>>> ld: error: unable to find library -llutok_pie
>>> ld: error: unable to find library -lkyua_model_pie
>>> ld: error: unable to find library -llutok_pie
>>> ld: error: unable to find library -lkyua_store_pie
>>> ld: error: too many errors emitted, stopping now (use -error-limit=0
>>> to see all errors)
>>> c++: error: linker command failed with exit code 1 (use -v to see
>>> invocation)
>>> --- all_subdir_lib ---
>>> --- cpuset_getdomain.po ---
>>> --- all_subdir_usr.bin ---
>>> *** [kyua] Error code 1
>>>
>>> make[4]: stopped in /usr/src/usr.bin/kyua
>>> 1 error
>>>
>>> make[4]: stopped in /usr/src/usr.bin/kyua
>>> *** [all_subdir_usr.bin/kyua] Error code 2
>>> ...
>>>
>>> May be it's related to r359260[1]. Therefore, here is my TEST-settings:
>>>
>>> $ fgrep TEST /etc/src.conf
>>> WITHOUT_GOOGLETEST=
>>> WITHOUT_TESTS=
>>> WITH_TESTS_SUPPORT=
>>>
>>> Also what has confused me: it's a virtual machine which failed to build.
>>> A physical one built userland and kernel just fine. Both the physical
>>> and virtual machines have almost the same (differ only by CPU
>>> "selection" options) make.conf and src.conf and different kernel
>>> configurations. Both the machines were FreeBSD 13.0-CURRENT amd64
>>> r359231. I've started to build on clean systems (no /usr/obj at all).
>>>
>>> Can anyone help me to resolve this issue?
>>
>> I've replicated this issue and it goes back the the way
>> WITH_TESTS_SUPPORT was implemented way back in r273449
>>
>> This patch fixes WITHOUT_TESTS=t WITH_TESTS_SUPPORT=t for me.
>>
>> Index: Makefile.inc1
>> ===
>> --- Makefile.inc1 (revision 359367)
>> +++ Makefile.inc1 (working copy)
>> @@ -1100,7 +1100,7 @@
>> @echo
>> "--"
>> ${_+_}cd ${.CURDIR}; \
>> ${WMAKE} -DNO_FSCHG MK_HTML=no -DNO_LINT MK_MAN=no \
>> -MK_PROFILE=no MK_TESTS=no MK_TESTS_SUPPORT=${MK_TESTS}
>> libraries
>> +MK_PROFILE=no MK_TESTS=no libraries
>> everything: .PHONY
>> @echo
>> @echo
>> "--
>>
>> I've not committed it yet because I'm trying to figure out why this was
>> needed. I simply don't see how there could be a race between lib/aft and
>> libexec/aft as described. I suspect this may have been an error.
>
> I've committed a different fix in r359382.
Thanks a lot! I'll try it latter.
>
> The above change fixed the case at hand, but broke the default.
>
> -- Brooks
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
New messages after updating from r365443 to r365738
Hi!
After I had updated my FreeBSD 13.0-CURRENT r365443 amd64 to r365738 the
following messages started to appear on the console/log:
$ egrep "not implemented|async_" /var/log/messages | fgrep "Sep 18"
Sep 18 08:19:05 {host_name} kernel: __pm_runtime_resume not
implemented -- see your local kernel hacker
Sep 18 08:19:05 {host_name} kernel: pm_runtime_mark_last_busy not
implemented -- see your local kernel hacker
Sep 18 08:19:05 {host_name} kernel: __pm_runtime_suspend not
implemented -- see your local kernel hacker
Sep 18 08:19:05 {host_name} kernel: pm_runtime_get_if_in_use not
implemented -- see your local kernel hacker
Sep 18 08:19:05 {host_name} kernel: kmem_cache_shrink not implemented
-- see your local kernel hacker
Sep 18 08:19:05 {host_name} kernel: register_oom_notifier not
implemented -- see your local kernel hacker
Sep 18 08:19:05 {host_name} kernel: register_acpi_notifier not
implemented -- see your local kernel hacker
Sep 18 08:19:05 {host_name} kernel: async_schedule is dodgy -- see
your local kernel hacker
Sep 18 08:19:05 {host_name} kernel: pm_runtime_set_autosuspend_delay
not implemented -- see your local kernel hacker
Sep 18 08:19:05 {host_name} kernel: __pm_runtime_use_autosuspend not
implemented -- see your local kernel hacker
Sep 18 08:19:05 {host_name} kernel: async_synchronize_cookie not
implemented -- see your local kernel hacker
What do they mean?
Should I provide more information on my system?
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
