Re: A reliable port cross-build failure (hangup) in my context (amd64->armv7 cross build, with native-tool speedup involved)

2018-12-22 Thread Mark Millard via freebsd-emulation
[I messed up the freebsd-emulation email address the first time I sent this. I also forgot to indicate the qemu-user-static vintage relationship.] I had been reporting intermittent hang-ups for my amd64->{aarch64,armv7} port cross builds in another message sequence. But it turns out that one thin

Re: A reliable port cross-build failure (hangup) in my context (amd64->armv7 cross build, with native-tool speedup involved)

2018-12-22 Thread Mark Millard via freebsd-emulation
[I found my E-mail records reporting successful builds using qemu-user-static from ports head -r484783 under FreeBSD head -r340287.] On 2018-Dec-22, at 00:10, Mark Millard wrote: > [I messed up the freebsd-emulation email address the first time I sent > this. I also forgot to indicate the qemu-u

Re: A reliable port cross-build failure (hangup) in my context (amd64->armv7 cross build, with native-tool speedup involved)

2018-12-23 Thread Mark Millard via freebsd-emulation
[I built a FreeBSD head -r340288 context and tried ports head -r484783 and the problem repeated.] On 2018-Dec-22, at 12:55, Mark Millard wrote: > [I found my E-mail records reporting successful builds using > qemu-user-static from ports head -r484783 under FreeBSD > head -r340287.] > > On 2018-

Re: A reliable port cross-build failure (hangup) in my context (amd64->armv7 cross build, with native-tool speedup involved)

2018-12-24 Thread Mark Millard via freebsd-emulation
[A native poudreire-devel based build of multimedia/gstreamer1-qt@qt5 did not hang-up and worked fine. Official package build history also provides some evidence.] On 2018-Dec-22, at 12:55, Mark Millard wrote: > [I found my E-mail records reporting successful builds using > qemu-user-static from

qemu-user-static: Unknown syscall 545 (i.e., ppoll)

2018-12-27 Thread Mark Millard via freebsd-emulation
In trying to gain evidence for a amd64->armv{7,6} ports cross-build hangup I enabled do_strace in qemu-arm-static . One thing that I ran into is notices of: Unknown syscall 545 sys/sys/syscall.h lists: #define SYS_ppoll 545 The man pages report that ppoll was added in FreeBSD 11. 10.x and

Re: A reliable port cross-build failure (hangup) in my context (amd64->armv7 cross build, with native-tool speedup involved)

2018-12-28 Thread Mark Millard via freebsd-emulation
[The historical notes are removed and replaced by partial trace information from example hang-ups, not that I've figured out what contributes yet.] I ran into the following while trying to get evidence about the hang-up for an amd64->armv7 cross-build of multimedia/gstreamer1-qt@qt5 . The followi

Re: A reliable port cross-build failure (hangup) in my context (amd64->armv7 cross build, with native-tool speedup involved)

2018-12-28 Thread Mark Millard via freebsd-emulation
On 2018-Dec-28, at 05:13, Michal Meloun wrote: > Mark, > this is known problem with qemu-user-static. > Emulation of every single interruptible syscall is broken by design (it > have signal related races). Theses races cannot be solved without major > rewrite of syscall emulation code. > Unfor

Re: A reliable port cross-build failure (hangup) in my context (amd64->armv7 cross build, with native-tool speedup involved)

2018-12-28 Thread Mark Millard via freebsd-emulation
[Using ktrace/kdump shows an apperent oddity in the kevent use that hang-up in cmake, not that I know it causes the hang-up.] On 2018-Dec-28, at 00:16, Mark Millard wrote: > [The historical notes are removed and replaced by partial trace > information from example hang-ups, not that I've figured

Re: A reliable port cross-build failure (hangup) in my context (amd64->armv7 cross build, with native-tool speedup involved)

2018-12-28 Thread Mark Millard via freebsd-emulation
On 2018-Dec-28, at 12:12, Mark Millard wrote: > On 2018-Dec-28, at 05:13, Michal Meloun wrote: > >> Mark, >> this is known problem with qemu-user-static. >> Emulation of every single interruptible syscall is broken by design (it >> have signal related races). Theses races cannot be solved wit

Re: A reliable port cross-build failure (hangup) in my context (amd64->armv7 cross build, with native-tool speedup involved) [details of a specific qemu-arm-static source code problem]

2018-12-30 Thread Mark Millard via freebsd-emulation
On 2018-Dec-28, at 12:12, Mark Millard wrote: > On 2018-Dec-28, at 05:13, Michal Meloun wrote: > >> Mark, >> this is known problem with qemu-user-static. >> Emulation of every single interruptible syscall is broken by design (it >> have signal related races). Theses races cannot be solved wi

Re: A reliable port cross-build failure (hangup) in my context (amd64->armv7 cross build, with native-tool speedup involved) [details of a specific qemu-arm-static source code problem]

2018-12-30 Thread Mark Millard via freebsd-emulation
[Removing __packed did make the size and offsets match armv7 and the build worked based on the reconstructed qemu-arm-static.] On 2018-Dec-30, at 16:38, Mark Millard wrote: > On 2018-Dec-28, at 12:12, Mark Millard wrote: > >> On 2018-Dec-28, at 05:13, Michal Meloun wrote: >> >>> Mark, >>> th

Re: A reliable port cross-build failure (hangup) in my context (amd64->armv7 cross build, with native-tool speedup involved) [details of a specific qemu-arm-static source code problem]

2018-12-31 Thread Mark Millard via freebsd-emulation
On 2018-Dec-30, at 21:01, Jonathan Chen wrote: > On Mon, 31 Dec 2018 at 14:34, Mark Millard via freebsd-ports > wrote: >> >> [Removing __packed did make the size and offsets match armv7 >> and the build worked based on the reconstructed qemu-arm-static.] > > Thanks for the analysis Mark! I've

Re: A reliable port cross-build failure (hangup) in my context (amd64->armv7 cross build, with native-tool speedup involved) [details of a specific qemu-arm-static source code problem]

2018-12-31 Thread Mark Millard via freebsd-emulation
On 2018-Dec-31, at 10:16, Jonathan Chen wrote: > On Mon, 31 Dec 2018 at 21:05, Mark Millard wrote: > [...] >> But if you have a form of hang-up that shows no sign of being tied >> to kevent or hangs-up only sometimes, I'd be surprised if the __packed >> change(s) would fix the issue. > > With t

Re: A reliable port cross-build failure (hangup) in my context (amd64->armv7 cross build, with native-tool speedup involved) [details of a specific qemu-arm-static source code problem]

2018-12-31 Thread Mark Millard via freebsd-emulation
[I listed my /usr/src svn veriosn information instead of /usr/ports . Correcting. . .] On 2018-Dec-31, at 12:05, Mark Millard wrote: > On 2018-Dec-31, at 10:16, Jonathan Chen wrote: > >> On Mon, 31 Dec 2018 at 21:05, Mark Millard wrote: >> [...] >>> But if you have a form of hang-up that show

qemu-arm-static under amd64: example of stuck looping atomic_cmpset_int while building graphics/poppler-qt5 (with backtrace, code, and value details)

2019-01-01 Thread Mark Millard via freebsd-emulation
The below showed up for poudiere-devel bulk getting stuck using one FreeBSD cpu while building graphics/poppler-qt5 . This is not a kevent hang-up, unlike the last hang-up that I analyzed. I do not yet know how repeatable this is but the original hang-up and the one experiment the below is from. >

Re: qemu-arm-static under amd64: example of stuck looping atomic_cmpset_int while building graphics/poppler-qt5 [problem possibly found]

2019-01-01 Thread Mark Millard via freebsd-emulation
[It looks to me like the assembler code has some code moved out of the loop that it should not be (by intent). It appears that calling tcmpset_32 does not prevent code motion to before the call and the variable involved was not declared volatile.] On 2019-Jan-1, at 18:43, Mark Millard wrote: > T

Re: qemu-arm-static under amd64: example of stuck looping atomic_cmpset_int while building graphics/poppler-qt5 [problem not found]

2019-01-01 Thread Mark Millard via freebsd-emulation
[I was wrong: it is code elimination, not motion. volatile is not a fix.] On 2019-Jan-1, at 19:37, Mark Millard wrote: > [It looks to me like the assembler code has some code moved out of the > loop that it should not be (by intent). It appears that calling > tcmpset_32 does not prevent code mot

Re: qemu-arm-static under amd64: example of stuck looping atomic_cmpset_int while building graphics/poppler-qt5 [a tested fix included]

2019-01-02 Thread Mark Millard via freebsd-emulation
On 2019-Jan-1, at 18:43, Mark Millard wrote: > The below showed up for poudiere-devel bulk getting stuck using one FreeBSD > cpu while building graphics/poppler-qt5 . This is not a kevent hang-up, unlike > the last hang-up that I analyzed. I do not yet know how repeatable this is > but the origin

Re: qemu-arm-static under amd64: example of stuck looping atomic_cmpset_int while building graphics/poppler-qt5 [a tested fix included]

2019-01-02 Thread Mark Millard via freebsd-emulation
On 2019-Jan-2, at 17:41, Kyle Evans wrote: > On Wed, Jan 2, 2019 at 3:38 AM Mark Millard via freebsd-ports > wrote: >> >>> . . . >> >> So (without old line numbers): >> >>} else if (TARGET_URWLOCK_READER_COUNT(state) != 0) { >>/* decrement reader count */ >>for (;;) { >>

Under qemu-aarch64-static "wc /dev/null" gets "Unsupported ancillary data: 1/0" from a sendmsg attempt: because of wrong cmsg_len type in target_cmsghdr

2019-01-03 Thread Mark Millard via freebsd-emulation
[This note follows the investigation sequence, ending with the important conclusions.] My test context here is a poudriere-devel bulk -i for a amd64->aarch64 context. wc /dev/null or wc //dev/null does: # wc /dev/null Unsupported ancillary data: 1/0 that then hangs-up until I ^C to get back to

Re: Under qemu-aarch64-static "wc /dev/null" gets "Unsupported ancillary data: 1/0" from a sendmsg attempt: because of wrong cmsg_len type in target_cmsghdr

2019-01-03 Thread Mark Millard via freebsd-emulation
[Adding a for-reference note.] On 2019-Jan-3, at 01:25, Mark Millard wrote: > [This note follows the investigation sequence, > ending with the important conclusions.] > > My test context here is a poudriere-devel bulk -i for a > amd64->aarch64 context. > > wc /dev/null or wc //dev/null does: >

qemu-x86_64-static has target_msghdr's msg_controllen field with the wrong size so its msg_flags is at the wrong offset and target_msghdr is too large

2019-01-04 Thread Mark Millard via freebsd-emulation
[qemu-aarch64-static has the same problem but qemu-armv7-sstatic does not. The context here is FreeBSD head -r341836 based and ports head -r488859 based.] Note: I assume that "struct target_msghdr" is meant to match the memory layout of the target's native "struct msghdr". Otherwise the reported

qemu-x86_64-static has target_freebsd_flock being too large (__packed use issue)

2019-01-04 Thread Mark Millard via freebsd-emulation
[qemu-aarch64-static has the same problem but qemu-armv7-sstatic does not. The context here is FreeBSD head -r341836 based and ports head -r488859 based.] Note: I assume that "struct target_freebsd_flock" is meant to match the memory layout of the target's native "struct flock". Otherwise the re

Re: qemu-x86_64-static has target_freebsd_flock being too small (__packed use issue) [subject correction: fixed to be "too small"]

2019-01-04 Thread Mark Millard via freebsd-emulation
[Just correcting the "larger" to be "smaller".] On 2019-Jan-4, at 19:29, Mark Millard wrote: [qemu-aarch64-static has the same problem but qemu-armv7-sstatic does not. The context here is FreeBSD head -r341836 based and ports head -r488859 based.] Note: I assume that "struct target_freebsd_flo

qemu-arm-static has target_sigframe missing sf_fvp field so target_sigframe is too small

2019-01-05 Thread Mark Millard via freebsd-emulation
[The context here is FreeBSD head -r341836 based and ports head -r488859 based.] Note: I assume that "struct target_sigframe" is meant to match the memory layout of the target's native "struct sigframe". Otherwise the reported differences below could be irrelevant. For armv7 (and likely armv6) th

qemu-arm-static has target_prpsinfo missing pr_pid field so target_prpsinfo is too small

2019-01-05 Thread Mark Millard via freebsd-emulation
[The context here is FreeBSD head -r341836 based and ports head -r488859 based.] Note: I assume that "struct target_prpsinfo" is meant to match the memory layout of the target's native "struct prpsinfo". Otherwise the reported differences below could be irrelevant. Note: struct target_prpsinfo is

qemu-*-static struct target_msgbuf is greatly mismatched with /usr/include/sys/msgbuf.h 's struct msgbuf

2019-01-05 Thread Mark Millard via freebsd-emulation
[The context here is FreeBSD head -r341836 based and ports head -r488859 based.] /usr/include/sys/msgbuf.h has: struct msgbuf { char *msg_ptr;/* pointer to buffer */ #define MSG_MAGIC 0x063062 u_int msg_magic; u_int msg_size;/*

qemu-arm-static has target_msqid_ds too small vs. arm natives msqid_ds

2019-01-05 Thread Mark Millard via freebsd-emulation
[The context here is FreeBSD head -r341836 based and ports head -r488859 based.] Note: I assume that "struct target_msqid_ds" is meant to match the memory layout of the target's native "struct msqid_ds". Otherwise the reported differences below could be irrelevant. For armv7 (and likely armv6) th

qemu-arm-static has target_semd_ds too small vs. arm natives semid_ds

2019-01-05 Thread Mark Millard via freebsd-emulation
[The context here is FreeBSD head -r341836 based and ports head -r488859 based.] Note: I assume that "struct target_semd_ds" is meant to match the memory layout of the target's native "struct semid_ds". Otherwise the reported differences below could be irrelevant. For armv7 (and likely armv6) the

qemu-arm-static has target_shmd_ds too small vs. arm native's shmid_ds

2019-01-05 Thread Mark Millard via freebsd-emulation
[The context here is FreeBSD head -r341836 based and ports head -r488859 based.] Note: I assume that "struct target_shmd_ds" is meant to match the memory layout of the target's native "struct shmid_ds". Otherwise the reported differences below could be irrelevant. For armv7 (and likely armv6) the

qemu-arm-static has target_freebsd11_nstat too small vs. arm native's struct nstat

2019-01-05 Thread Mark Millard via freebsd-emulation
[The context here is FreeBSD head -r341836 based and ports head -r488859 based.] Note: I assume that "struct target_shmd_ds" is meant to match the memory layout of the target's native "struct shmid_ds". Otherwise the reported differences below could be irrelevant. For armv7 (and likely armv6) the

under qemu-arch64-static in use via poudriere on amd64: cmake ended up with a defunct child

2019-02-08 Thread Mark Millard via freebsd-emulation
What happened ( from ps -auxd ): root 790180.0 0.0 12512 3088 0 I13:58 0:00.07 | | `-- sh: poudriere[FBSDFSSDjailCortexA57-default][06]: build_pkg (poppler-qt5-0.72.0_1) (sh) root 935770.0 0.0 12512 3080 0 I14:02 0:00.00 | |

Just a note in case others see something similar under Hyper-V

2019-08-15 Thread Mark Millard via freebsd-emulation
I run head versions of FreeBSD under Hyper-V under Windows 10 Pro (1903 in recent times), normally the primary use of the Windows 10 Pro machine is for running FreeBSD this way. (The same FreeBSD installation can native boot, depending on BIOS vintage.) I have had two Windows 10 Pro SYNTHETIC_WATC

Re: [Bug 246527] emulators/qemu-user-static: please add syscall 574 (__realpathat)

2020-05-18 Thread Mark Millard via freebsd-emulation
Kyle Evans wrote: QUOTE --- Comment #3 from Kyle Evans --- Yup- I should have mentioned, if you do or are willing to build your own qemu-user-static port, I have a patch available here that updates it to a version that implements most of the latest syscalls and seemingly fixes issues with multi-t