Re: Upstreaming the glibc Hurd port
On 04/02/2018 02:10 AM, Samuel Thibault wrote: - elf/check-execstack: We have nested functions which make the stack executable indeed. I looked at the Hurd situation last month. Hurd is always read-implies-exec, I think. So there is no need to make the stacks executable, it does not have any observable effect on execution and therefore cannot be tested. Thanks, Florian
Re: Upstreaming the glibc Hurd port
Florian Weimer, on lun. 02 avril 2018 09:51:00 +0200, wrote: > On 04/02/2018 02:10 AM, Samuel Thibault wrote: > > - elf/check-execstack: We have nested functions which make the stack > >executable indeed. > > I looked at the Hurd situation last month. Hurd is always > read-implies-exec, I think. Nobody has implemented NX in gnumach yet, but that doesn't mean nobody will. Samuel
Re: Upstreaming the glibc Hurd port
On Mon, 2 Apr 2018, Samuel Thibault wrote: > - header standard conformity issues: These will be hard to fix. What are the issues here? > - elf/check-localplt: There will always be PLTs to libhurd/machuser.so > anyway. If a library has *local* PLT entries - PLT entries for within-library calls to other functions in that shared library - that are hard to fix to use hidden aliases, it's expected that the localplt.data files will list those as expected (with a "?" if the PLT entry may or may not be present depending on compiler version etc., as is the case for some configurations where functions get exported from both libgcc and glibc). > - elf/check-execstack: We have nested functions which make the stack > executable indeed. That's pointers to nested functions involving creation of trampolines? Do those nested functions actually improve the code or would it be cleaner (have cleaner internal interfaces etc.) without them? Do all libraries have these or only some? > - check-abi-libmachuser, check-abi-libhurduser: These actually depend on > .defs files in gnumach and hurd, so we can't really define ABI files. Depend in what way? Are you saying they export different symbols depending on the versions of gnumach and hurd glibc is built with? How are symbol versions for such extra symbols determined - based on gnumach and hurd versions instead of glibc versions? It's clearly desirable to be able to make sure that old symbol versions don't change. But if the contents of those versions are determined by gnumach and hurd, maybe those packages need to install ABI baselines for the glibc tests (or something like that). The nearest analogue I see in glibc for systems using the Linux kernel (without more information on how the gnumach/hurd dependency works) is the syscall lists - where we have a list in glibc of all possible syscall names (sysdeps/unix/sysv/linux/syscall-names.list), and a test will fail if the kernel has additional syscalls and is not newer than the version listed in that list (so using a different kernel version does not result in test failures, but the test is fully effective if the listed kernel version or an older kernel version is used). -- Joseph S. Myers jos...@codesourcery.com
Re: Upstreaming the glibc Hurd port
On Mon, 2 Apr 2018, Samuel Thibault wrote: > This means that build-glibcs i686-gnu now builds fine. Among the > remaining TODOs, there are Thanks! I'd add: the "requires out-of-tree patches" statement in README needs to be removed, and a NEWS entry is needed. Will you be able to provide full execution test results for the testsuite on the per-release wiki pages for 2.28 and later releases, during each release freeze period? -- Joseph S. Myers jos...@codesourcery.com
Re: Upstreaming the glibc Hurd port
Hello, Joseph Myers, on lun. 02 avril 2018 14:22:28 +, wrote: > On Mon, 2 Apr 2018, Samuel Thibault wrote: > > > This means that build-glibcs i686-gnu now builds fine. Among the > > remaining TODOs, there are > > Thanks! I'd add: the "requires out-of-tree patches" statement in README > needs to be removed, Oh, I was unaware of that mention. > and a NEWS entry is needed. Right :) > Will you be able to provide full execution test results for the > testsuite on the per-release wiki pages for 2.28 and later releases, > during each release freeze period? Now that it does build and run quite fine, sure! :) Samuel
Re: Upstreaming the glibc Hurd port
Joseph Myers, on lun. 02 avril 2018 14:17:38 +, wrote: > On Mon, 2 Apr 2018, Samuel Thibault wrote: > > > - header standard conformity issues: These will be hard to fix. > > What are the issues here? Some of these are small, like ./bits/types/sigevent_t.h's sigev_notify_attributes not being pthread_attr_t* Others need some work, like the missing SA_SIGINFO, for which we have patches which need polishing and committing. Others need implementing, like SA_NOCLDWAIT. Others just need defining, like IUCLC, IXANY, etc. There are a few remaining namespace issues due to missing __ marking or spurious #includes. So those, we can fix them. Others really pose problem, like ./sysdeps/mach/hurd/bits/fcntl.h's l_type/l_whence being int instead of short. There is also sys/un.h which defines a sun_len field, which IIRC is to be expected on BSD systems, but not defined in Posix? Also, ioctl takes (int, unsigned long int, ...) but ./conform/XPG42/stropts.h/conform.out wants (int, int, ...)? > > - elf/check-localplt: There will always be PLTs to libhurd/machuser.so > > anyway. > > If a library has *local* PLT entries - Ah. I got confused by the presence of __vm_allocate which is an RPC, but we actually have a non-RPC version inside libc.so itself. There are a hundred of them mostly in libc.so, ld.so, libpthread.so > PLT entries for within-library calls to other functions in that shared > library - that are hard to fix to use hidden aliases, it's expected > that the localplt.data files will list those as expected Ok. > > - elf/check-execstack: We have nested functions which make the stack > > executable indeed. > > That's pointers to nested functions involving creation of trampolines? Yes. > Do those nested functions actually improve the code Yes. There are notably callbacks whose parameters don't permit to get the context parameters etc. > Do all libraries have these or only some? Only libc.so, ld.so and libpthread.so have them. > > - check-abi-libmachuser, check-abi-libhurduser: These actually depend on > > .defs files in gnumach and hurd, so we can't really define ABI files. > > Depend in what way? Are you saying they export different symbols > depending on the versions of gnumach and hurd glibc is built with? That is it, yes. > How are symbol versions for such extra symbols determined - based on > gnumach and hurd versions instead of glibc versions? That was not actually settled, but I guess it should be gnumach and hurd versions. > maybe those packages need to install ABI baselines for the glibc tests > (or something like that). Indeed. Samuel
Re: Upstreaming the glibc Hurd port
Samuel Thibault, on lun. 02 avril 2018 17:50:17 +0200, wrote: > There are a few remaining namespace issues due to missing __ marking or > spurious #includes. One issue is with struct sched_param. The __sched_param definition was removed in glibc while htl's pthread_attr uses it. For now I just made struct___pthread_attr.h include but that brings things unwanted for (which is supposed to pull pthread_attr). So I guess we should revert to defining __sched_param? Samuel
Re: Upstreaming the glibc Hurd port
H.J. Lu, on lun. 02 avril 2018 09:01:30 -0700, wrote: > On Mon, Apr 2, 2018 at 7:47 AM, Samuel Thibault > wrote: > > Hello, > > > > Joseph Myers, on lun. 02 avril 2018 14:22:28 +, wrote: > >> On Mon, 2 Apr 2018, Samuel Thibault wrote: > >> > >> > This means that build-glibcs i686-gnu now builds fine. Among the > >> > remaining TODOs, there are > >> > >> Thanks! I'd add: the "requires out-of-tree patches" statement in README > >> needs to be removed, > > > > Oh, I was unaware of that mention. > > mig master branch failed to build on Fedora 27: > > gcc -g -O2 -o migcom error.o global.o header.o lexxer.o migcom.o > parser.o routine.o server.o statement.o string.o type.o user.o utils.o > vprint.o > /usr/local/bin/ld: lexxer.o: in function `yylex': > /export/gnu/import/git/toolchain/build/compilers/i686-gnu/mig/lexxer.c:1834: > undefined reference to `yywrap' Could you post config.log? configure is supposed to detect this: checking lex library... -lfl Samuel
Re: Upstreaming the glibc Hurd port
On Mon, Apr 2, 2018 at 7:47 AM, Samuel Thibault wrote: > Hello, > > Joseph Myers, on lun. 02 avril 2018 14:22:28 +, wrote: >> On Mon, 2 Apr 2018, Samuel Thibault wrote: >> >> > This means that build-glibcs i686-gnu now builds fine. Among the >> > remaining TODOs, there are >> >> Thanks! I'd add: the "requires out-of-tree patches" statement in README >> needs to be removed, > > Oh, I was unaware of that mention. > > mig master branch failed to build on Fedora 27: gcc -g -O2 -o migcom error.o global.o header.o lexxer.o migcom.o parser.o routine.o server.o statement.o string.o type.o user.o utils.o vprint.o /usr/local/bin/ld: lexxer.o: in function `yylex': /export/gnu/import/git/toolchain/build/compilers/i686-gnu/mig/lexxer.c:1834: undefined reference to `yywrap' collect2: error: ld returned 1 exit status make[3]: *** [Makefile:495: migcom] Error 1 H.J.
Re: Upstreaming the glibc Hurd port
On Mon, Apr 2, 2018 at 9:16 AM, Samuel Thibault wrote: > H.J. Lu, on lun. 02 avril 2018 09:01:30 -0700, wrote: >> On Mon, Apr 2, 2018 at 7:47 AM, Samuel Thibault >> wrote: >> > Hello, >> > >> > Joseph Myers, on lun. 02 avril 2018 14:22:28 +, wrote: >> >> On Mon, 2 Apr 2018, Samuel Thibault wrote: >> >> >> >> > This means that build-glibcs i686-gnu now builds fine. Among the >> >> > remaining TODOs, there are >> >> >> >> Thanks! I'd add: the "requires out-of-tree patches" statement in README >> >> needs to be removed, >> > >> > Oh, I was unaware of that mention. >> >> mig master branch failed to build on Fedora 27: >> >> gcc -g -O2 -o migcom error.o global.o header.o lexxer.o migcom.o >> parser.o routine.o server.o statement.o string.o type.o user.o utils.o >> vprint.o >> /usr/local/bin/ld: lexxer.o: in function `yylex': >> /export/gnu/import/git/toolchain/build/compilers/i686-gnu/mig/lexxer.c:1834: >> undefined reference to `yywrap' > > Could you post config.log? configure is supposed to detect this: > > checking lex library... -lfl > checking lex library... none needed -- H.J.
Re: Upstreaming the glibc Hurd port
H.J. Lu, on lun. 02 avril 2018 10:06:14 -0700, wrote: > On Mon, Apr 2, 2018 at 9:16 AM, Samuel Thibault > wrote: > > H.J. Lu, on lun. 02 avril 2018 09:01:30 -0700, wrote: > >> On Mon, Apr 2, 2018 at 7:47 AM, Samuel Thibault > >> wrote: > >> > Hello, > >> > > >> > Joseph Myers, on lun. 02 avril 2018 14:22:28 +, wrote: > >> >> On Mon, 2 Apr 2018, Samuel Thibault wrote: > >> >> > >> >> > This means that build-glibcs i686-gnu now builds fine. Among the > >> >> > remaining TODOs, there are > >> >> > >> >> Thanks! I'd add: the "requires out-of-tree patches" statement in README > >> >> needs to be removed, > >> > > >> > Oh, I was unaware of that mention. > >> > >> mig master branch failed to build on Fedora 27: > >> > >> gcc -g -O2 -o migcom error.o global.o header.o lexxer.o migcom.o > >> parser.o routine.o server.o statement.o string.o type.o user.o utils.o > >> vprint.o > >> /usr/local/bin/ld: lexxer.o: in function `yylex': > >> /export/gnu/import/git/toolchain/build/compilers/i686-gnu/mig/lexxer.c:1834: > >> undefined reference to `yywrap' > > > > Could you post config.log? configure is supposed to detect this: > > > > checking lex library... -lfl > > checking lex library... none needed Please really config.log, not only the configure output. Samuel
Re: Upstreaming the glibc Hurd port
On Mon, Apr 2, 2018 at 10:15 AM, Samuel Thibault wrote: > H.J. Lu, on lun. 02 avril 2018 10:06:14 -0700, wrote: >> On Mon, Apr 2, 2018 at 9:16 AM, Samuel Thibault >> wrote: >> > H.J. Lu, on lun. 02 avril 2018 09:01:30 -0700, wrote: >> >> On Mon, Apr 2, 2018 at 7:47 AM, Samuel Thibault >> >> wrote: >> >> > Hello, >> >> > >> >> > Joseph Myers, on lun. 02 avril 2018 14:22:28 +, wrote: >> >> >> On Mon, 2 Apr 2018, Samuel Thibault wrote: >> >> >> >> >> >> > This means that build-glibcs i686-gnu now builds fine. Among the >> >> >> > remaining TODOs, there are >> >> >> >> >> >> Thanks! I'd add: the "requires out-of-tree patches" statement in >> >> >> README >> >> >> needs to be removed, >> >> > >> >> > Oh, I was unaware of that mention. >> >> >> >> mig master branch failed to build on Fedora 27: >> >> >> >> gcc -g -O2 -o migcom error.o global.o header.o lexxer.o migcom.o >> >> parser.o routine.o server.o statement.o string.o type.o user.o utils.o >> >> vprint.o >> >> /usr/local/bin/ld: lexxer.o: in function `yylex': >> >> /export/gnu/import/git/toolchain/build/compilers/i686-gnu/mig/lexxer.c:1834: >> >> undefined reference to `yywrap' >> > >> > Could you post config.log? configure is supposed to detect this: >> > >> > checking lex library... -lfl >> >> checking lex library... none needed > > Please really config.log, not only the configure output. > configure:4096: checking lex library configure:4110: gcc -o conftest -g -O2 conftest.c >&5 /usr/local/bin/ld: /tmp/ccDAnO2r.o: in function `input': /export/gnu/import/git/toolchain/build/compilers/i686-gnu/mig/lex.yy.c:1185: undefined reference to `yywrap' /usr/local/bin/ld: /tmp/ccDAnO2r.o: in function `yylex': /export/gnu/import/git/toolchain/build/compilers/i686-gnu/mig/lex.yy.c:879: undefined reference to `yywrap' /usr/local/bin/ld: /tmp/ccDAnO2r.o: in function `main': /export/gnu/import/git/toolchain/build/compilers/i686-gnu/mig/conftest.l:18: undefined reference to `yywrap' collect2: error: ld returned 1 exit status configure:4110: $? = 1 configure:4460: result: no ... configure: exit 0 -- H.J.
Re: Upstreaming the glibc Hurd port
H.J. Lu, on lun. 02 avril 2018 10:22:31 -0700, wrote: > On Mon, Apr 2, 2018 at 10:15 AM, Samuel Thibault > wrote: > > H.J. Lu, on lun. 02 avril 2018 10:06:14 -0700, wrote: > >> On Mon, Apr 2, 2018 at 9:16 AM, Samuel Thibault > >> wrote: > >> > H.J. Lu, on lun. 02 avril 2018 09:01:30 -0700, wrote: > >> >> On Mon, Apr 2, 2018 at 7:47 AM, Samuel Thibault > >> >> wrote: > >> >> > Hello, > >> >> > > >> >> > Joseph Myers, on lun. 02 avril 2018 14:22:28 +, wrote: > >> >> >> On Mon, 2 Apr 2018, Samuel Thibault wrote: > >> >> >> > >> >> >> > This means that build-glibcs i686-gnu now builds fine. Among the > >> >> >> > remaining TODOs, there are > >> >> >> > >> >> >> Thanks! I'd add: the "requires out-of-tree patches" statement in > >> >> >> README > >> >> >> needs to be removed, > >> >> > > >> >> > Oh, I was unaware of that mention. > >> >> > >> >> mig master branch failed to build on Fedora 27: > >> >> > >> >> gcc -g -O2 -o migcom error.o global.o header.o lexxer.o migcom.o > >> >> parser.o routine.o server.o statement.o string.o type.o user.o utils.o > >> >> vprint.o > >> >> /usr/local/bin/ld: lexxer.o: in function `yylex': > >> >> /export/gnu/import/git/toolchain/build/compilers/i686-gnu/mig/lexxer.c:1834: > >> >> undefined reference to `yywrap' > >> > > >> > Could you post config.log? configure is supposed to detect this: > >> > > >> > checking lex library... -lfl > >> > >> checking lex library... none needed > > > > Please really config.log, not only the configure output. > > > > configure:4096: checking lex library > configure:4110: gcc -o conftest -g -O2 conftest.c >&5 > /usr/local/bin/ld: /tmp/ccDAnO2r.o: in function `input': > /export/gnu/import/git/toolchain/build/compilers/i686-gnu/mig/lex.yy.c:1185: > undefined reference to `yywrap' > /usr/local/bin/ld: /tmp/ccDAnO2r.o: in function `yylex': > /export/gnu/import/git/toolchain/build/compilers/i686-gnu/mig/lex.yy.c:879: > undefined reference to `yywrap' > /usr/local/bin/ld: /tmp/ccDAnO2r.o: in function `main': > /export/gnu/import/git/toolchain/build/compilers/i686-gnu/mig/conftest.l:18: > undefined reference to `yywrap' > collect2: error: ld returned 1 exit status > configure:4110: $? = 1 > > configure:4460: result: no Please really post config.log (or send it privately). "result: no" here is not related to "checking lex library". Samuel
Re: Upstreaming the glibc Hurd port
Samuel Thibault, on lun. 02 avril 2018 19:35:11 +0200, wrote: > > configure:4096: checking lex library > > configure:4110: gcc -o conftest -g -O2 conftest.c >&5 > > /usr/local/bin/ld: /tmp/ccDAnO2r.o: in function `input': > > /export/gnu/import/git/toolchain/build/compilers/i686-gnu/mig/lex.yy.c:1185: > > undefined reference to `yywrap' > > /usr/local/bin/ld: /tmp/ccDAnO2r.o: in function `yylex': > > /export/gnu/import/git/toolchain/build/compilers/i686-gnu/mig/lex.yy.c:879: > > undefined reference to `yywrap' > > /usr/local/bin/ld: /tmp/ccDAnO2r.o: in function `main': > > /export/gnu/import/git/toolchain/build/compilers/i686-gnu/mig/conftest.l:18: > > undefined reference to `yywrap' > > collect2: error: ld returned 1 exit status > > configure:4110: $? = 1 > > > > configure:4460: result: no > > Please really post config.log (or send it privately). "result: no" here > is not related to "checking lex library". I guess you don't have libfl-devel on your system, and apparently autoconf's programs.m4 doesn't actually handle that case. Samuel
Re: Upstreaming the glibc Hurd port
On Mon, 2 Apr 2018, Samuel Thibault wrote:
> Others really pose problem, like ./sysdeps/mach/hurd/bits/fcntl.h's
> l_type/l_whence being int instead of short.
Where something is problematic to fix, because a fix would break the ABI
or needs some external feature, there is an xfail mechanism internal to
the conform tests. It involves a bug being filed in Bugzilla for the
issue in question, an addition to conformtest-xfail-conds (conditional on
ifeq ($(subdir),conform)) in an appropriate sysdeps Makefile, with a
comment referencing the bug, and xfail[cond]- on the relevant expectation
in the relevant -data file, again with a comment referencing the bug.
For example:
// Bug 17786: st_dev has wrong type.
xfail[mips-o32-linux]-element {struct stat} dev_t st_dev
Before doing any such XFAILing, you should check that the expectation is
actually backed up by the relevant standard and that a fix really does
need an ABI change or a new feature. Also, this XFAIL mechanism can only
be used for positive expectations that are failing - it can't be used for
cases where the header violates the expected namespace.
> There is also sys/un.h which defines a sun_len field, which IIRC is to
> be expected on BSD systems, but not defined in Posix?
Well, in fact POSIX reserves sun_* for sys/un.h (note the reservations are
in a separate part of the standard from the main definitions of header
contents), so there's a bug in the expectations not allowing for it. This
illustrates the need for checking such failures against the standards to
see where the bug is.
> Also, ioctl takes (int, unsigned long int, ...) but
> ./conform/XPG42/stropts.h/conform.out wants (int, int, ...)?
That's already generically XFAILed with reference to bug 14362.
> > Do those nested functions actually improve the code
>
> Yes. There are notably callbacks whose parameters don't permit to get
> the context parameters etc.
>
> > Do all libraries have these or only some?
>
> Only libc.so, ld.so and libpthread.so have them.
Then maybe some mechanism is needed for sysdeps Makefiles to define
libraries expected to have executable stacks.
--
Joseph S. Myers
jos...@codesourcery.com
Re: Upstreaming the glibc Hurd port
On Mon, 2 Apr 2018, Samuel Thibault wrote: > Samuel Thibault, on lun. 02 avril 2018 17:50:17 +0200, wrote: > > There are a few remaining namespace issues due to missing __ marking or > > spurious #includes. > > One issue is with struct sched_param. The __sched_param definition > was removed in glibc while htl's pthread_attr uses it. For now I just > made struct___pthread_attr.h include but that brings things > unwanted for (which is supposed to pull pthread_attr). > > So I guess we should revert to defining __sched_param? If you need, on Hurd, in installed headers, a type "struct __sched_param", that would most naturally go in a Hurd-specific bits/types/struct___sched_param.h. If that's not appropriate, please explain the issue in more detail. Anything restoring the old __need_* mechanism for any glibc-internal definitions is not appropriate (struct __sched_param was removed as part of removing __need_*). -- Joseph S. Myers jos...@codesourcery.com
