Re: DragonFly added support for ELF preinit, init, and fini arrays
On Wed, Feb 15, 2012 at 08:22:02PM -0500, Pedro Giffuni wrote: > Hello; > > FYI, this commit in DragonFly seems interesting: > > http://leaf.dragonflybsd.org/mailarchive/commits/2012-02/msg00146.html > > It appears like linux had them from a while and some years ago they started > using them for C++ in gcc: > > http://gcc.gnu.org/ml/gcc-patches/2005-03/msg01872.html > > cheers, If you read the commit message, you will notice something at the end. I have WIP patches to add support for the init arrays to our rtld. It is not tested yet, and I want to ponder more about the ABI changes in csu that I introduce. In fact, I am more interested in having answer to the question, what immediate and long-term use do we have for the feature. Also, I want non-artificial test cases for DT_INIT_ARRAY and DT_FINI_ARRAY. http://people.freebsd.org/~kib/misc/init_array.1.patch pgppHAgPPi5tB.pgp Description: PGP signature
Re: DragonFly added support for ELF preinit, init, and fini arrays
On Thu, Feb 16, 2012 at 05:51:21PM +0100, John Marino wrote: > One obvious case for the immediate use is the building of gold linker in > binutils 2.22. By default, it moves constructors into the init array, > so gold will segfault if it was linked with itself. (The workaround is > to patch option.h file to change the ctors-in-init-array to false before > building gold so it has the same behavior as gnu ld). Linking gold with gold is quite a work to setup a test case. Could you provide me with isolated test cases like you did for the exception handling ? pgpM19kDBmxFk.pgp Description: PGP signature
Re: DragonFly added support for ELF preinit, init, and fini arrays
On Sat, Feb 18, 2012 at 07:53:22AM +0100, John Marino wrote: > On 2/16/2012 9:27 PM, Konstantin Belousov wrote: > >On Thu, Feb 16, 2012 at 05:51:21PM +0100, John Marino wrote: > >>One obvious case for the immediate use is the building of gold linker in > >>binutils 2.22. By default, it moves constructors into the init array, > >>so gold will segfault if it was linked with itself. (The workaround is > >>to patch option.h file to change the ctors-in-init-array to false before > >>building gold so it has the same behavior as gnu ld). > >Linking gold with gold is quite a work to setup a test case. > >Could you provide me with isolated test cases like you did for the > >exception handling ? > Hi Konstantin, > I have attached 5 small tests, where 4 of them are from glibc. They > should be sufficient by themselves to check your patch. However, if you > feel you need more then I could extract a few tests from the gnu ld > testsuite. Thank you, it was very useful. It seems that test4 needed some adjustments to actually provide the required dso for tests. Now, the patch at http://people.freebsd.org/~kib/misc/init_array.2.patch passes your tests and allows to boot world built with pre-patched csu into multiuser mode. I decided to move the calls to initializers for main binary into rtld, and used the brand note to distinguish old and new csu. ATM patch is only for i386 and amd64, I will port to other architectures after some feedback. pgpg6TtnVLiOt.pgp Description: PGP signature
Re: DragonFly added support for ELF preinit, init, and fini arrays
On Sun, Feb 19, 2012 at 09:03:00AM +0100, John Marino wrote: > On 2/18/2012 10:38 PM, Konstantin Belousov wrote: > > > >Thank you, it was very useful. It seems that test4 needed some adjustments > >to actually provide the required dso for tests. > Yeah. I thought I gave you a version that worked out of the box but I > did modify this test as well. > > >Now, the patch at http://people.freebsd.org/~kib/misc/init_array.2.patch > >passes your tests and allows to boot world built with pre-patched csu > >into multiuser mode. > > > >I decided to move the calls to initializers for main binary into rtld, > >and used the brand note to distinguish old and new csu. > I assume the purpose of this is to have a clean split with static > initialization done in crt1 and dynamic initialization done entirely in > rtld for binaries built with version 19 and later. Does the note > digestion used to check osrel add significant overhead? I do not think that note parsing can add any measureable overhead. The goal there is not to have some theoretical cleaness, but to cause the least possible bug propagation. The crt code gets linked into the binary, be it static or dynamic. We cannot help the static case, but for dynamic one, less non-application code is stored in the binary, the bigger our possibilities to deliver fixes for OS bugs without requiring application recompilation. From this POV, having the preinit array/init/array init linked into the binary is quite bad. > > Is not this information already available via aux_vectors passed on from > imgact_elf.c? No. After thinking about the init_array.3 patch which uses __FreeBSD_version of the crt to distinguish old and new crt, I understood that this change is very hard to merge to stable branch. In essence, we would ship some rtld's in HEAD which will misbehave on stable binaries, if the change is merged back. So I decided to redo this once more, adding a vendor note that signifies that crt does not call init. See http://people.freebsd.org/~kib/misc/init_array.4.patch > > > > >ATM patch is only for i386 and amd64, I will port to other architectures > >after some feedback. > It's out of scope of your current work but related to notes, but there's > another test that DragonFly passes that FreeBSD fails. If memory > serves, it was caused by the PT_NOTE segment getting located outside of > the first page the majority of the time starting with binutils 2.21. > http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/9d35f29fb5dd669d4e04521b39296e5bdf14b2b0 > > A related problem was an interpret segment outside of the first page: > http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/99aed3e43ef052664173110d6db3ddf6a62e375b > > Both of these were handled within sys/kern/imgact_elf.c The fact that interpreter path and notes should be placed at the first page of ELF image is well-known ELF quirk, possibly not well documented. There is indeed an issue with crtbrand.c unable to create a proper section for the note, see the long comment in recent crtbrand.c. FreeBSD solution was to emit right section type. pgpDs0dghiU9m.pgp Description: PGP signature
init arrays patch
Hi, The latest version of the patch to add support for init and fini arrays for FreeBSD is available at http://people.freebsd.org/~kib/misc/init_array.7.patch Apparently, some variant of ARM ABI mandates the use of arrays, so there is a demand for the change. Also, it is another step to bring us closer to modern ELF platform. I decided to change ABI by no longer calling _init/_fini functions from the crt1 for dynamic binaries. To keep the backward compatibility, a new ELF note is added for binaries that have modern crt1. The rtld now parses notes and calls _init/_fini itsef. This is done to easier the implementation of the proper order between preinit array, _init and init array. The patch passes make universe, was tested by me on i386 and amd64, a previous version of the patch was tested by Andrew on arm. The sparc64, ia64, mips and both powerpc are not tested. Please note that ia64 also has old kind of crt1, which calls _init and does not provide note. The cause is that crt1.S for ia64 is written in assembly, which I do not know, and have no access to machine. Good test for the patch is to boot new world with old ports, which were linked with old crt1. Also, there is a test suite at http://people.freebsd.org/~kib/misc/rtld-init_array_tests.tbz Use top-level script run_tests.sh to run the tests, possibly changing the CC variable to point to your preferred compiler. Normal run shall produce no output. Looking for feedback, test results, esp. on the listed architectures. pgpvMDJIHbqn0.pgp Description: PGP signature
Re: DragonFly added DT_GNU_HASH support to rtld
On Sun, Mar 11, 2012 at 10:59:00AM +0100, John Marino wrote: > Hi Konstantin, > > It seems that no BSD supported DT_GNU_HASH despite this option being > available on the base binutils (FreeBSD's 2.17.50 binutils supports it). > This gnu extension is a big performance improvement over the specified > SysV hash. > > The guy porting libreoffice to pkgsrc was finding -Wl,--hash-style=gnu > to be the default build for that package. Indeed, using the standard > hash results in very long startup times for something like Writer (> 8 > seconds launched from a SSD) > > The result is that we brought in DT_GNU_HASH support to our real-time > linker this weekend. We're still waiting to see how that improves > libreoffice startup times. > > full commit: > >http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/7629c6317998f850ebca23c296822ba08af09e5b > > > Modification to base compiler so all system libs and binaries can take > advantage of it: > >http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/4687ecd9561d76f3d02ccb4f7adeecd5e3afdd8f > > > For a while, binaries with dynamic symbol tables will have both types of > hashes embedded. At some point in the future, we may switch over to > only generating the GNU hash. This will break forward compatibility, > but that happens rather frequently for other reasons anyway. > > I was a bit surprised FreeBSD didn't already have this functionality > given the performance benefits, so hopefully these DragonFly commits > will be interesting for you. > I finally ported the Dragonfly commit to FreeBSD. There were several changes reverted in dragonfly version of the extracted matched_symbol() function which were restored. I also blindly converted all non-x86 arches. The matched_symbol() extraction is the good opportunity to apply the style(9) formatting to the large chunk of rtld code. Any testers, esp. on non-x86 architectures, are welcome. You would need to modify gcc spec file for you architecture, see corresponding x86 changes in contrib/gcc/config/i386. For me, patch successfully worked on the machine were I disabled sysv hashes at all. http://people.freebsd.org/~kib/misc/rtld-gnu_hash.1.patch pgpzIUBZ4SqUl.pgp Description: PGP signature
Re: gdb support for follow-fork
On Thu, Apr 12, 2012 at 01:15:37PM -0700, Dmitry Mikulin wrote: > Hi, > > I implemented support for follow-fork mode in gdb and would like to submit > it for a review. The feature allows users to specify which process to > follow when the inferior does a fork(). The default mode is to stay with > the parent. A gdb command is used to switch between following the > child/parent: > > set follow-fork-mode child > set follow-fork-mode parent > > When gdb is set to follow the child, it detaches from the parent at the > fork() point and continues to trace the newly created child. > > Gdb uses support that has recently been implemented in the FreeBSD kernel. > A new ptrace() request, PT_FOLLOW_FORK was added to generate a SIGTRAP at > the point where a child process is created. This allows gdb switch from > parent to child. PT_FOLLOW_EXEC was added to SIGTRAP on return from exit() > to allow gdb to load a new binary for debugging. > > I attached the patch here. > Please let me know if can provide additional info. Yes, could you, please, split the patch somewhat ? It would be much easier to review if you were able to provide relatively small pieces with descriptions. pgpzrr4bYW1ft.pgp Description: PGP signature
[RFC] Un-staticise the toolchain
I think it is time to stop building the toolchain static. I was told that
original reasoning for static linking was the fear of loosing the ability
to recompile if some problem appears with rtld and any required dynamic
library. Apparently, current dependencies are much more spread, e.g. /bin/sh
is dynamically linked, and statically linked make does not solve anything.
Patch below makes the dynamically linked toolchain a default, adding an
WITHOUT_SHARED_TOOLCHAIN build-time option for real conservators.
I did not looked in details why including bsd.own.mk makes NO_MAN
non-functional. Please see the diffs for gnu/usr.bin/cc1*/Makefile.
gnu/usr.bin/binutils/ar/Makefile |3 +++
gnu/usr.bin/binutils/as/Makefile |3 +++
gnu/usr.bin/binutils/ld/Makefile |3 +++
gnu/usr.bin/binutils/ranlib/Makefile |3 +++
gnu/usr.bin/cc/cc/Makefile|2 ++
gnu/usr.bin/cc/cc1/Makefile |5 -
gnu/usr.bin/cc/cc1plus/Makefile |5 -
share/mk/bsd.own.mk |3 ++-
tools/build/options/WITH_STATIC_TOOLCHAIN |6 ++
usr.bin/ar/Makefile |4
usr.bin/make/Makefile |4
11 files changed, 38 insertions(+), 3 deletions(-)
diff --git a/gnu/usr.bin/binutils/ar/Makefile b/gnu/usr.bin/binutils/ar/Makefile
index 464445e..e26be85 100644
--- a/gnu/usr.bin/binutils/ar/Makefile
+++ b/gnu/usr.bin/binutils/ar/Makefile
@@ -1,6 +1,7 @@
# $FreeBSD$
.include "../Makefile.inc0"
+.include
.PATH: ${SRCDIR}/binutils ${SRCDIR}/binutils/doc
@@ -16,7 +17,9 @@ CFLAGS+= -D_GNU_SOURCE
CFLAGS+= -I${.CURDIR}/${RELTOP}/libbinutils
CFLAGS+= -I${SRCDIR}/binutils
CFLAGS+= -I${SRCDIR}/bfd
+.if ${MK_SHARED_TOOLCHAIN} != "no"
NO_SHARED?= yes
+.endif
DPADD= ${RELTOP}/libbinutils/libbinutils.a
DPADD+=${RELTOP}/libbfd/libbfd.a
DPADD+=${RELTOP}/libiberty/libiberty.a
diff --git a/gnu/usr.bin/binutils/as/Makefile b/gnu/usr.bin/binutils/as/Makefile
index bf8df81..a39918c 100644
--- a/gnu/usr.bin/binutils/as/Makefile
+++ b/gnu/usr.bin/binutils/as/Makefile
@@ -4,6 +4,7 @@
# BINDIR
.include "${.CURDIR}/../../Makefile.inc"
.include "${.CURDIR}/../Makefile.inc0"
+.include
.PATH: ${SRCDIR}/gas ${SRCDIR}/gas/config
@@ -79,7 +80,9 @@ CFLAGS+= -D_GNU_SOURCE
CFLAGS+= -I${SRCDIR}/gas -I${SRCDIR}/bfd -I${SRCDIR}/gas/config -I${SRCDIR}
CFLAGS+= -I${.CURDIR} -I${.CURDIR}/${TARGET_CPUARCH}-freebsd
+.if ${MK_SHARED_TOOLCHAIN} != "no"
NO_SHARED?=yes
+.endif
DPADD= ${RELTOP}/libbfd/libbfd.a
DPADD+=${RELTOP}/libiberty/libiberty.a
diff --git a/gnu/usr.bin/binutils/ld/Makefile b/gnu/usr.bin/binutils/ld/Makefile
index d4420ed..50d88b5 100644
--- a/gnu/usr.bin/binutils/ld/Makefile
+++ b/gnu/usr.bin/binutils/ld/Makefile
@@ -1,6 +1,7 @@
# $FreeBSD$
.include "../Makefile.inc0"
+.include
.PATH: ${SRCDIR}/ld
@@ -34,7 +35,9 @@ CFLAGS+= -DBINDIR=\"${BINDIR}\"
-DTARGET_SYSTEM_ROOT=\"${TOOLS_PREFIX}\"
CFLAGS+= -DTOOLBINDIR=\"${TOOLS_PREFIX}/${BINDIR}/libexec\"
CFLAGS+= -D_GNU_SOURCE
CFLAGS+= -I${SRCDIR}/ld -I${SRCDIR}/bfd
+.if ${MK_SHARED_TOOLCHAIN} != "no"
NO_SHARED?= yes
+.endif
DPADD= ${RELTOP}/libbfd/libbfd.a
DPADD+=${RELTOP}/libiberty/libiberty.a
LDADD= ${DPADD}
diff --git a/gnu/usr.bin/binutils/ranlib/Makefile
b/gnu/usr.bin/binutils/ranlib/Makefile
index 8679375..6d4b13e 100644
--- a/gnu/usr.bin/binutils/ranlib/Makefile
+++ b/gnu/usr.bin/binutils/ranlib/Makefile
@@ -1,6 +1,7 @@
# $FreeBSD$
.include "../Makefile.inc0"
+.include
.PATH: ${SRCDIR}/binutils ${SRCDIR}/binutils/doc
@@ -16,7 +17,9 @@ CFLAGS+= -D_GNU_SOURCE
CFLAGS+= -I${.CURDIR}/${RELTOP}/libbinutils
CFLAGS+= -I${SRCDIR}/binutils
CFLAGS+= -I${SRCDIR}/bfd
+.if ${MK_SHARED_TOOLCHAIN} != "no"
NO_SHARED?= yes
+.endif
DPADD= ${RELTOP}/libbinutils/libbinutils.a
DPADD+=${RELTOP}/libbfd/libbfd.a
DPADD+=${RELTOP}/libiberty/libiberty.a
diff --git a/gnu/usr.bin/cc/cc/Makefile b/gnu/usr.bin/cc/cc/Makefile
index 78c83a5..6a65d69 100644
--- a/gnu/usr.bin/cc/cc/Makefile
+++ b/gnu/usr.bin/cc/cc/Makefile
@@ -9,7 +9,9 @@ PROG= gcc
MAN= gcc.1
SRCS+= gccspec.c
+.if ${MK_SHARED_TOOLCHAIN} != "no"
NO_SHARED?=yes
+.endif
MLINKS=gcc.1 g++.1
.if ${MK_CLANG_IS_CC} == "no"
diff --git a/gnu/usr.bin/cc/cc1/Makefile b/gnu/usr.bin/cc/cc1/Makefile
index c65acd2..bb8fe19 100644
--- a/gnu/usr.bin/cc/cc1/Makefile
+++ b/gnu/usr.bin/cc/cc1/Makefile
@@ -1,14 +1,17 @@
# $FreeBSD$
.include "../Makefile.inc"
+.include
.PATH: ${GCCDIR}
PROG= cc1
SRCS= main.c c-parser.c c-lang.c
BINDIR=/usr/libexec
-NO_MAN=
+MAN=
+.if ${MK_SHARED_TOOLCHAIN} != "no"
NO_SHARED?=yes
+.endif
OBJS+= ${PROG}-checksum.o
DPADD= ${LIBBACKEND} ${LIBCPP} ${LIBDECNUMBER} ${LIBIBERTY}
diff --git a/gnu/usr.bin/cc/cc1plus/Makefile b/gnu/usr.bin/cc/cc1plus/Makefile
index 964d20f..0bcbe61 100644
--- a/gnu/usr.bin/cc/cc1plus/Makefile
+++ b/gnu/usr.bin/cc/cc1plu
Re: [RFC] Un-staticise the toolchain
On Thu, Apr 26, 2012 at 05:41:40PM +0400, Ruslan Ermilov wrote: > On Thu, Apr 26, 2012 at 12:35:48PM +0300, Konstantin Belousov wrote: > > I think it is time to stop building the toolchain static. I was told that > > original reasoning for static linking was the fear of loosing the ability > > to recompile if some problem appears with rtld and any required dynamic > > library. Apparently, current dependencies are much more spread, e.g. /bin/sh > > is dynamically linked, and statically linked make does not solve anything. > > > r76801 | sobomax | 2001-05-18 13:05:56 +0400 (Fri, 18 May 2001) | 6 lines > > By default build make(1) as a static binary. It costs only 100k of additional > disk space, buf provides measureable speed increase for make-intensive > operations, such as pkg_version(1), `make world' and so on. > > MFC after:1 week > > > > Have things changed enough that the above is not true anymore? > > > Patch below makes the dynamically linked toolchain a default, adding an > > WITHOUT_SHARED_TOOLCHAIN build-time option for real conservators. > > > > I did not looked in details why including bsd.own.mk makes NO_MAN > > non-functional. Please see the diffs for gnu/usr.bin/cc1*/Makefile. > > Because you include bsd.own.mk before NO_MAN is defined, and the way > how .if works in make(1). What is the 'right' thing to do then ? Postpone the inclusion of bsd.own.mk after NO_MAN definition ? This makes the .if $MK_SHARED_TOOLCHAIN to not work. Or, continue to do what I have done, using 'MAN=' instead ? N.B. I will commit the change, with defaults kept to build toolchain static, just to avoid bikeshed. pgpvnaU9s5ATe.pgp Description: PGP signature
Re: [RFC] Un-staticise the toolchain
On Fri, Apr 27, 2012 at 02:58:06PM +0400, Ruslan Ermilov wrote:
> Regarding your patch...
>
> By placing SHARED_TOOLCHAIN to __DEFAULT_NO_OPTIONS list in
> bsd.own.mk, you already had MK_SHARED_TOOLCHAIN set to "no" by
> default, which preserves the current status quo of building
> toolchain static. But you misspelled
> tools/build/options/WITH_STATIC_TOOLCHAIN, probably as the result
> of iteratively modifying your change. The option and this file
> should be named WITH_SHARED_TOOLCHAIN, the opposite of the
> default. Anyway, checking that the resulting src.conf(5) manpage
> sounds sensible is a good idea. As for the contents of this file,
> I wouldn't call "ar/ranlib" a "librarian" but rather a "library
> archives manager", as per POSIX. Your diff also suggests that it
> misses a newline at EOF.
Thank you for the suggestions.
Below is the variant that should handle all the commentary.
diff --git a/gnu/usr.bin/binutils/ar/Makefile b/gnu/usr.bin/binutils/ar/Makefile
index 464445e..6fe22c8 100644
--- a/gnu/usr.bin/binutils/ar/Makefile
+++ b/gnu/usr.bin/binutils/ar/Makefile
@@ -1,6 +1,7 @@
# $FreeBSD$
.include "../Makefile.inc0"
+.include
.PATH: ${SRCDIR}/binutils ${SRCDIR}/binutils/doc
@@ -16,7 +17,9 @@ CFLAGS+= -D_GNU_SOURCE
CFLAGS+= -I${.CURDIR}/${RELTOP}/libbinutils
CFLAGS+= -I${SRCDIR}/binutils
CFLAGS+= -I${SRCDIR}/bfd
+.if ${MK_SHARED_TOOLCHAIN} == "no"
NO_SHARED?= yes
+.endif
DPADD= ${RELTOP}/libbinutils/libbinutils.a
DPADD+=${RELTOP}/libbfd/libbfd.a
DPADD+=${RELTOP}/libiberty/libiberty.a
diff --git a/gnu/usr.bin/binutils/as/Makefile b/gnu/usr.bin/binutils/as/Makefile
index bf8df81..5fef1f3 100644
--- a/gnu/usr.bin/binutils/as/Makefile
+++ b/gnu/usr.bin/binutils/as/Makefile
@@ -4,6 +4,7 @@
# BINDIR
.include "${.CURDIR}/../../Makefile.inc"
.include "${.CURDIR}/../Makefile.inc0"
+.include
.PATH: ${SRCDIR}/gas ${SRCDIR}/gas/config
@@ -79,7 +80,9 @@ CFLAGS+= -D_GNU_SOURCE
CFLAGS+= -I${SRCDIR}/gas -I${SRCDIR}/bfd -I${SRCDIR}/gas/config -I${SRCDIR}
CFLAGS+= -I${.CURDIR} -I${.CURDIR}/${TARGET_CPUARCH}-freebsd
+.if ${MK_SHARED_TOOLCHAIN} == "no"
NO_SHARED?=yes
+.endif
DPADD= ${RELTOP}/libbfd/libbfd.a
DPADD+=${RELTOP}/libiberty/libiberty.a
diff --git a/gnu/usr.bin/binutils/ld/Makefile b/gnu/usr.bin/binutils/ld/Makefile
index d4420ed..ef19afa 100644
--- a/gnu/usr.bin/binutils/ld/Makefile
+++ b/gnu/usr.bin/binutils/ld/Makefile
@@ -1,6 +1,7 @@
# $FreeBSD$
.include "../Makefile.inc0"
+.include
.PATH: ${SRCDIR}/ld
@@ -34,7 +35,9 @@ CFLAGS+= -DBINDIR=\"${BINDIR}\"
-DTARGET_SYSTEM_ROOT=\"${TOOLS_PREFIX}\"
CFLAGS+= -DTOOLBINDIR=\"${TOOLS_PREFIX}/${BINDIR}/libexec\"
CFLAGS+= -D_GNU_SOURCE
CFLAGS+= -I${SRCDIR}/ld -I${SRCDIR}/bfd
+.if ${MK_SHARED_TOOLCHAIN} == "no"
NO_SHARED?= yes
+.endif
DPADD= ${RELTOP}/libbfd/libbfd.a
DPADD+=${RELTOP}/libiberty/libiberty.a
LDADD= ${DPADD}
diff --git a/gnu/usr.bin/binutils/ranlib/Makefile
b/gnu/usr.bin/binutils/ranlib/Makefile
index 8679375..052f9fe 100644
--- a/gnu/usr.bin/binutils/ranlib/Makefile
+++ b/gnu/usr.bin/binutils/ranlib/Makefile
@@ -1,6 +1,7 @@
# $FreeBSD$
.include "../Makefile.inc0"
+.include
.PATH: ${SRCDIR}/binutils ${SRCDIR}/binutils/doc
@@ -16,7 +17,9 @@ CFLAGS+= -D_GNU_SOURCE
CFLAGS+= -I${.CURDIR}/${RELTOP}/libbinutils
CFLAGS+= -I${SRCDIR}/binutils
CFLAGS+= -I${SRCDIR}/bfd
+.if ${MK_SHARED_TOOLCHAIN} == "no"
NO_SHARED?= yes
+.endif
DPADD= ${RELTOP}/libbinutils/libbinutils.a
DPADD+=${RELTOP}/libbfd/libbfd.a
DPADD+=${RELTOP}/libiberty/libiberty.a
diff --git a/gnu/usr.bin/cc/cc/Makefile b/gnu/usr.bin/cc/cc/Makefile
index 78c83a5..ba53565 100644
--- a/gnu/usr.bin/cc/cc/Makefile
+++ b/gnu/usr.bin/cc/cc/Makefile
@@ -9,7 +9,9 @@ PROG= gcc
MAN= gcc.1
SRCS+= gccspec.c
+.if ${MK_SHARED_TOOLCHAIN} == "no"
NO_SHARED?=yes
+.endif
MLINKS=gcc.1 g++.1
.if ${MK_CLANG_IS_CC} == "no"
diff --git a/gnu/usr.bin/cc/cc1/Makefile b/gnu/usr.bin/cc/cc1/Makefile
index c65acd2..7b1e343 100644
--- a/gnu/usr.bin/cc/cc1/Makefile
+++ b/gnu/usr.bin/cc/cc1/Makefile
@@ -1,14 +1,17 @@
# $FreeBSD$
.include "../Makefile.inc"
+NO_MAN=
+.include
.PATH: ${GCCDIR}
PROG= cc1
SRCS= main.c c-parser.c c-lang.c
BINDIR=/usr/libexec
-NO_MAN=
+.if ${MK_SHARED_TOOLCHAIN} == "no"
NO_SHARED?=yes
+.endif
OBJS+= ${PROG}-checksum.o
DPADD= ${LIBBACKEND} ${LIBCPP} ${LIBDECNUMBER} ${LIBIBERTY}
diff --git a/gnu/usr.bin/cc/cc1plus/Makefile b/gnu/usr.bin/cc/cc1plus/Makefile
index 964d20f..dd3d524 100644
--- a/gnu/usr.bin/cc/cc1plus/Makefile
+++ b/gnu/usr.bin/cc/cc1plus/Makefile
@@ -1,6 +1,8 @@
# $FreeBSD$
.include "../Makefile.inc"
+NO_MAN=
+.include
.PATH: ${GCCDIR}/cp ${GCCDIR}
@@ -13,8 +15,9 @@ SRCS+=main.c cp-lang.c c-opts.c call.c class.c cvt.c
cxx-pretty-print.c \
cp-objcp-common.c cp-gimplify.c tree-mudflap.c
BINDIR=/usr/libexec
-NO_MAN=
+.if ${MK_SHARED_TOOLCHAIN} == "no"
NO_SHARED?=yes
+.endif
CFLAGS+
Re: [GDB follow-fork] behavior change for wait()
On Fri, Apr 27, 2012 at 03:33:49PM -0700, Marcel Moolenaar wrote:
> Hi Dmitry,
>
> I've been testing the follow-fork changes in GDB and ran into some weird
> behavior. Without gdb, my test program (attached) prints something like:
>
> fbsdvm% ./fe
> fe(41042): initial process. Doing fork & exec...
> fe(41043): child after fork. Doing exec...
> fe(41043): child after exec. Exiting...
> fe(41042): child 41043 exited with status 0
>
> In particular: the parent (pid=41042) calls wait(2) to reap the child and
> the child exits with 0.
>
> Under gdb, I see this:
>
> fbsdvm% gdb ./fe
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB. Type "show warranty" for details.
> This GDB was configured as "i386-marcel-freebsd"...
> (gdb) br main
> Breakpoint 1 at 0x80487b0: file fe.c, line 14.
> (gdb) run
> Starting program: /usr/home/marcel/fe
>
> Breakpoint 1, main (argc=Error accessing memory address 0x1: Bad address.
> ) at fe.c:14
> 14{
> (gdb) n
> main (argc=1, argv=0xbfbfebb4) at fe.c:19
> 19if (getenv("__FE_FORKED__") != NULL) {
> (gdb) c
> Continuing.
> fe(41141): initial process. Doing fork & exec...
> [New process 41143]
> fe(41143): child after fork. Doing exec...
> fe(41143): child after exec. Exiting...
> fe(41141): wait(2) failed with error 10 (No child processes)
>
> Program exited normally.
> (gdb)
>
> When stepping at least once, the inferior will not be able to properly wait(2)
> for its child as it seems to have been reaped already. I suspect this is done
> by the debugger -- unintentionally at least.
>
> Have you seen this before?
How old is your kernel ? In particular, do you have
r232048+r232104+r232947+r232975+r233808+r233809 ?
I cannot reproduce your issue on recent kernels and patched gdb.
pgpmPeTknz4Ee.pgp
Description: PGP signature
libunwind-based pstack(1)
Hi, I reimplemented pstack(1) using libunwind. The source is available at the git repository at http://people.freebsd.org/~kib/git/pstacku.git/ . To use it, you should also use git HEAD of the libunwind from http://libunwind.nongnu.org, I do not think that version from ports will work. Due to libunwind use, this pstack works on i386 and amd64. When libunwind/FreeBSD is ported to other arches, adding corresponding support to pstack is quite easy. So far, I tried to implement most of the features supported by original pstack, but there are limitations due to use of libunwind. Only libthr supported as the threading library, you probably get some funny results for libc_r and kse-based libpthread. The big unimplemented feature is coredump stack dumping, but libunwind only got support for Linux coredump backtracing a day ago, and I did not yet looked at porting this to FreeBSD. Lesser implemented but not properly working feature is the arguments printing. I might fix this later. Enjoy. pgpChGnHCvNXf.pgp Description: PGP signature
Re: libunwind-based pstack(1)
On Tue, Jun 05, 2012 at 04:34:17PM +0200, Jeremie Le Hen wrote: > Hi Kostik, > > On Thu, May 24, 2012 at 03:25:18PM +0300, Konstantin Belousov wrote: > > Hi, > > I reimplemented pstack(1) using libunwind. The source is available at > > the git repository at http://people.freebsd.org/~kib/git/pstacku.git/ . > > To use it, you should also use git HEAD of the libunwind from > > http://libunwind.nongnu.org, I do not think that version from ports > > will work. Due to libunwind use, this pstack works on i386 and amd64. > > When libunwind/FreeBSD is ported to other arches, adding corresponding > > support to pstack is quite easy. > > > > So far, I tried to implement most of the features supported by original > > pstack, but there are limitations due to use of libunwind. Only libthr > > supported as the threading library, you probably get some funny results > > for libc_r and kse-based libpthread. > > > > The big unimplemented feature is coredump stack dumping, but libunwind > > only got support for Linux coredump backtracing a day ago, and I did not > > yet looked at porting this to FreeBSD. > > > > Lesser implemented but not properly working feature is the arguments > > printing. I might fix this later. > > I tried it on RELENG_9/amd64 it works mostly, but most of the time I only get > the stack address and an offset from the _init symbol when the address > points into the binary text or "" when it is a shared library. > Supposedly this is because there is no known symbol for the function, > but having used Solaris' pstack quite extensively, I was a little bit > disappointed. > > % obiwan:/opt/bin# ./pstack -O 2237 > % 2237: /usr/local/libexec/postfix/nqmgr (osrel 0) > % Thread 101037: > % 0x80455153c in /lib/libcrypto.so.6 > % 0x42118a event_loop+0x11a in /usr/local/libexec/postfix/nqmgr > % 0x40b217 trigger_server_main+0xbd7 in /usr/local/libexec/postfix/nqmgr > % 0x403089 main+0xd9 in /usr/local/libexec/postfix/nqmgr > % 0x402f1c _start+0x9c in /usr/local/libexec/postfix/nqmgr > % 0x800d85000 in /usr/local/lib/libpcre.so.1 Can you show me the kdump of the run when '???' is reported for dso ip's ? pgp5Uu626Zjp8.pgp Description: PGP signature
Re: BSD ld (was Re: MCLinker and llvm-config)
On Sat, Jul 28, 2012 at 06:15:29PM +0200, Dimitry Andric wrote: > On 2012-07-28 16:15, Pedro Giffuni wrote: > > The Elftoolchain project has been developing a BSD ld: > > > > http://sourceforge.net/apps/trac/elftoolchain/ > > > > > > http://sourceforge.net/apps/trac/elftoolchain/browser/trunk/ld > > > > > > I thought that would be the official FreeBSD implementation. > > Let's just say there really isn't any consensus yet, except that GNU ld > should be binned. :) > > At EuroBSDCon 2011 there was a toolchain WG, where a few ideas and > requirements for a good system linker were shuffled around. See > Brooks' report here: > > http://wiki.freebsd.org/201110DevSummit/Toolchain?action=AttachFile&do=view&target=toolchain-wg-report.pdf This looks very futile and disorganized. When I tried to overlook only the basic features of modern linker required to build base, I ended up with approximately twenty-pages document, consisting mostly of enumerations of features. Consider this fully consumed by 'GNU ld compAtibility" in the referenced document. Other items looks mis-placed, e.g. multiple hash tables support is very technical and mundane, comparing with LTO/IR/incremental support, which ties the linker to compiler. And, the list of architectures provided shall be exploded since many architectures listed support different ABIs, which has direct impact on the linker functionality. My impression is that the list provided has no relation to 'good system linker requirements'. pgpFreCsBlF32.pgp Description: PGP signature
Re: Clang as default compiler November 4th
On Mon, Sep 10, 2012 at 04:12:07PM -0500, Brooks Davis wrote: > [Please confine your replies to toolch...@freebsd.org to keep the thread > on the most relevant list.] I do not see how removing current@ can be done, toolchain@ is not relevant for this discussion. Proposed is not a local change in the toolchain itself, but a far reaching and IMO premature change. > > For the past several years we've been working towards migrating from > GCC to Clang/LLVM as our default compiler. We intend to ship FreeBSD > 10.0 with Clang as the default compiler on i386 and amd64 platforms. To > this end, we will make WITH_CLANG_IS_CC the default on i386 and amd64 > platforms on November 4th. > > What does the mean to you? > > * When you build world after the default is changed /usr/bin/cc, cpp, and >c++ will be links to clang. > > * This means the initial phase of buildworld and "old style" kernel >compilation will use clang instead of gcc. This is known to work. > > * It also means that ports will build with clang by default. A major >of ports work, but a significant number are broken or blocked by >broken ports. For more information see: > http://wiki.freebsd.org/PortsAndClang > > What issues remain? > > * The gcc->clang transition currently requires setting CC, CXX, and CPP >in addition to WITH_CLANG_IS_CC. I will post a patch to toolchain@ >to address this shortly. > > * Ports compiler selection infrastructure is still under development. > > * Some ports could build with clang with appropriate tweaks. > > What can you do to help? > > * Switch (some of) your systems. Early adoption can help us find bugs. > > * Fix ports to build with clang. If you don't have a clang system, you >can use the CLANG/amd64 or CLANG/i386 build environments on >redports.org. > > tl;dr: Clang will become the default compiler for x86 architectures on > 2012-11-04 There was a chorus of voices talking about ports already. My POV is that suggesting to 'fix remaining ports to work with clang' is just a nonsense. You are proposing to fork the development of all the programs which do not compile with clang. Often, upstream developers do not care about clang at all since it not being default compiler in Debian/Fedora/Whatever Linux. The project simply do not have resources to maintain the fork of 20K programs. Looking from less amiable angle, you propose to knowingly break significant and important piece of the project work. My belief is that switch cannot be done before ports switch to the port-provided compiler. Another issue with the switch, which seems to be not only not addressed, but even not talked about, is the performance impact of the change. I do not remember any measurements, whatever silly they could be, of the performance change by the compiler switch. We often have serious and argumented push-back for kernel changes that give as low as 2-3% of the speed hit. What are the numbers for clang change, any numbers ? And, some small but annoying things left with clang, like ABI change requiring 16-byte stack alignment on i386, but lets ignore this until two big issues are resolved. pgpNKrF89TmKL.pgp Description: PGP signature
Re: Clang as default compiler November 4th
On Tue, Sep 11, 2012 at 02:06:49PM +0200, Roman Divacky wrote: > > > tl;dr: Clang will become the default compiler for x86 architectures on > > > 2012-11-04 > > > > There was a chorus of voices talking about ports already. My POV > > is that suggesting to 'fix remaining ports to work with clang' is > > just a nonsense. You are proposing to fork the development of all the > > programs which do not compile with clang. Often, upstream developers > > do not care about clang at all since it not being default compiler in > > Debian/Fedora/Whatever Linux. The project simply do not have resources > > to maintain the fork of 20K programs. > > We currently dont compile 4680 ports (out of 23857). Top 10 ports that prevent > the most other ports from compiling together prevent ports from > compilation. So if we fixed those 10 ports we could be at around 2500 ports > not compiling. Thats quite far from your claim of forking 20k programs. Sorry, I cannot buy the argument. How many patches there are already in the ports tree to cope with clang incompatibility with gcc ? You may declare that all of them are application bugs, but it completely misses the point. > > > Looking from less amiable angle, you propose to knowingly break significant > > and important piece of the project work. My belief is that switch cannot > > be done before ports switch to the port-provided compiler. > > I believe majority of the broken ports is broken because their maintainer > never saw them being broken with clang just because it's not the default > compiler. Thus by making it the default majority of the problems would just > go away. Can you, please, read what I wrote ? Fixing _ports_ to compile with clang is plain wrong. Upstream developers use gcc almost always for development and testing. Establishing another constant cost on the porting work puts burden on the ports submitters, maintainers and even ports users. I do strongly oppose the attempt to drain the freebsd resources by forcing porters to port third-party code to other compiler. > > Note that the work needed to make ports compiling with clang is largely shared > with the work to make them compiliable with newer GCCs (as they are stricter > etc.) > > > Another issue with the switch, which seems to be not only not addressed, > > but even not talked about, is the performance impact of the change. I > > do not remember any measurements, whatever silly they could be, of the > > performance change by the compiler switch. We often have serious and > > argumented push-back for kernel changes that give as low as 2-3% of > > the speed hit. What are the numbers for clang change, any numbers ? > > Agreed. We should provide numbers. At least how buildworld times change > with clang compiled kernel/workd and gcc compiler kernel/world. For this 'silly' benchmark, I would suggest to benchmark the identical buildworlds with only different kernels, compiled with in-tree gcc and clang. > > > And, some small but annoying things left with clang, like ABI change > > requiring 16-byte stack alignment on i386, but lets ignore this until > > two big issues are resolved. > > Thank you for pointing this out. This is, in my opinion, one of the strongest > reasons to switch to clang. We can go, fix issues or report we find upstream > and then import newer clang. Unlike with gcc. We do not want to hunt for the compiler bugs at all, goal of FreeBSD is to develop the OS and not a compiler. pgpYw2YTW2pBD.pgp Description: PGP signature
Re: Clang as default compiler November 4th
On Tue, Sep 11, 2012 at 09:27:07AM -0700, Pedro Giffuni wrote: > Hello; > > Just my $0.02. > > - Original Message - > ... > > Can you, please, read what I wrote ? Fixing _ports_ to compile with > > clang is plain wrong. Upstream developers use gcc almost always for > > development and testing. Establishing another constant cost on the > > porting work puts burden on the ports submitters, maintainers and even > > ports users. > > > > I do strongly oppose the attempt to drain the freebsd resources by > > forcing porters to port third-party code to other compiler. > > > > I can only speak for Apache OpenOffice but since Apple did the switch > already we are feeling a growing pressure to port OpenOffice to clang. > > For the time being we need gcc but we would really prefer something > more up to date than gcc 4.2.1 + fixes. In other words, yes making > clang the default may sound drastic but I am OK with killing base > gcc and if clang is what is left I can live with it. But allowing ports to select the compiler is the main point of my response, at least in the port part of it. I mean global configuration, and not referenced the existing per-port knobs (USE_GCC/WANT_GCC whatever). I would expect the portmgr to select some gcc (or clang or pcc or anything they find suitable) version and use it for a moment for ports. I do not claim that portmgr would consider 4.2.1 as the base for the switch but this is probably the least intrusive road right now. I do expect that selection shall be based on some measurement of the most supported compiler, and my gut feeling is that it ends as a version of gcc. Definitely, FreeBSD project is not a suitable place to make an efforts to port all existing OSS to clang, despite the opposite claims of the clang proponents. pgpsY89OtFlg3.pgp Description: PGP signature
Re: Compiler performance tests on FreeBSD 10.0-CURRENT
On Sun, Sep 16, 2012 at 12:34:45AM +0200, Dimitry Andric wrote: > Hi all, > > By request, I performed a series of kernel performance tests on FreeBSD > 10.0-CURRENT, particularly comparing the runtime performance of GENERIC > kernels compiled by gcc 4.2.1 and by clang 3.2. > > The attached text file[1] contains more information about the tests, > some semi-cooked performance data, and my conclusions. Any errors and > omissions are also my fault, so if you notice them, please let me know. > > The executive summary: GENERIC kernels compiled with clang 3.2 are > slightly faster than those compiled by gcc 4.2.1, though the difference > will not very noticeable in practice. > > Last but not least, thanks to Gavin Atkinson for providing the required > hardware. Thank you very much for doing this. I tried to map the CPUID into more human-friendly family moniker, and it seems that these are Pentium-4 class CPUs. Am I right ? If yes, could you, please, rerun the tests on anything more recent than Core2, i.e. any Core i7-whatever class of Xeons ? Thank again. pgptGb3ujM41w.pgp Description: PGP signature
Re: More kernel performance tests on FreeBSD 10.0-CURRENT
On Fri, Sep 21, 2012 at 11:39:40PM +0200, Dimitry Andric wrote: > Hi all, > > As a followup to my previous post about the performance of FreeBSD 10.0 > kernels compiled with different compilers (clang and gcc), I did another > series of tests, now on a more modern machine (Core i5-based). I also > tested the performance with different compiler optimization settings. > > The attached text file[1] contains more information about these tests, > performance data, and my conclusions. Any errors and omissions are also > my fault, so if you notice them, please let me know. > > The executive summary: GENERIC kernels compiled with clang 3.2 are again > a little faster than those compiled with gcc 4.2.1. For gcc, compiling > with -O2 also gives a slightly faster kernel than with -O1, but for > clang there is no measurable difference between those flags. > > Again, many thanks to Gavin Atkinson for providing the required > hardware. ... > Conclusion: > --- > Kernels compiled with clang are a little faster in real time for building > world, > and in system time the difference is even larger, roughly 10%. For clang, the > difference between -O1 and -O2 is not measurable, but for gcc, -O2 is slightly > faster than -O1. > Thank you very much for finishing the initial assessment. In my opinion, this positively closes the issue of the uncertainicity of the performance impact of the proposed clang use by default for the base system. Now, if only ports were handled. pgpugWddMiPW0.pgp Description: PGP signature
Re: LLVM Image Activator
On Mon, Jan 07, 2013 at 09:36:38AM -0500, John Baldwin wrote: > On Sunday, January 06, 2013 01:02:21 PM Nathan Whitehorn wrote: > > Having LLVM/clang in the base system lets us do some interesting things > > that we couldn't do with GCC. One is that LLVM ships with a JIT for LLVM > > IR as well as components of a toolchain for it (this is what Google's > > pNACL uses) and that you can end up producing binary files that are in > > IR instead of native code. The IR isn't really cross-platform, but does > > let you do CPU-specific optimizations when executed by the JIT, etc. > > > > The attached patch causes the LLVM JIT (lli) to be built by default > > (adding ~20 seconds to buildworld on my five-year-old laptop) and adds a > > kernel image activator that invokes it when passed LLVM bitcode files. > > It's not completely finished (see the XXX comment in the middle), but it > > does work, as follows: > > > > $ clang -emit-llvm -c -o hw.ll hw.c > > $ file hw.ll > > hw.ll: LLVM bitcode > > $ lli hw.ll > > Hello world! > > $ chmod a+x hw.ll > > $ ./hw.ll > > Hello world! > > $ > > > > Is there any interest in having features like this? It seems like this > > could provides some interesting possibilities for us and nice > > integration from having imported clang into base. > > -Nathan > > This sounds neat indeed. Does the IR format provide any sort of notation for > encoding the path to the interpreter (similar to ELF)? If not, you might > want > to at least make the path to 'lli' be configurable via a tunable and/or > sysctl > (e.g. if using a newer version of clang in /usr/local). I do not like one-purpose write-once never-used image activators. I still do remember the buzz about the binary format 0xCAFEBABE, which AFAIR gained image activator support on several OSes, to be garbage collected. For one thing, such hacks are very system-specific and thus are never used, causing quick bit rot. Second, more important, is that parsing of yet another format opens an attack vector on kernel, and due to the nature of the activators, on the user space. Look at the PE image activator history, including at least one SA, to see what I mean. And PE is real image format instead of hack. Even for ELF, there were recent panic-level bugs. Note that several language runtimes are happy to use the shebang for bytecode. For examples, look at the ocaml and lisps. If adding despite these arguments, please do not make the thing 'standard'. Provide the option to compile it in, and may be make it a module. pgpdPIA0aUmpv.pgp Description: PGP signature
Fast sigblock (AKA rtld speedup)
Below is the forward of the patch for which I failed to obtain a private
review. Might be, the list generates more responses.
Our rtld has a performance bootleneck, typically exposed by the images
with the lot of the run-time relocation processing, and by the C++
exception handling. We block the signals delivery during the rtld
performing the relocations, as well as for the dl_iterate_phdr(3) (the
later is used for finding the dwarf unwinding tables).
The signal blocking is needed to allow the rtld to resolve the symbols
for the signal handlers in the safe way, but also causes 2 syscalls
overhead per each rtld entry.
The proposed approach allows to shave off those two syscalls, doubling
the FreeBSD performance for the (silly) throw/catch C++ microbenchmark.
Date: Mon, 13 Aug 2012 15:26:00 +0300
From: Konstantin Belousov
...
The basic idea is to implement sigprocmask() as single write into usermode
address. If kernel needs to calculate the signal mask for current thread,
it takes into the consideration non-zero value of the word at the agreed
address. Also, usermode is informed about signals which were put on hold
due to fast sigblock active.
As I said, on my measurements in microbenchmark that did throw/catch in
a loop, I see equal user and system time spent for unpatched system, and
same user time with zero system time on patched system.
Patch can be improved further, e.g. it would be nice to allow rtld to fall
back to sigprocmask(2) if kernel does not support fast sigblock, to prevent
flag day. Also, the mask enforced by fast sigblock can be made configurable.
Note that libthr already blocks signals by catching them, and not using rtld
service in the first line handler. I tried to make the change in the spirit
of libthr interceptors, but handoff to libthr appears too complicated to
work. In fact, libthr can be changed to start using fast sigblock instead
of wrapping sigaction, but this is out of scope of the proposal right now.
Please comment.
diff --git a/lib/libc/sys/Symbol.map b/lib/libc/sys/Symbol.map
index 6888ea0..3b75539 100644
--- a/lib/libc/sys/Symbol.map
+++ b/lib/libc/sys/Symbol.map
@@ -546,6 +546,7 @@ FBSDprivate_1.0 {
__sys_extattr_set_link;
_extattrctl;
__sys_extattrctl;
+ __sys_fast_sigblock;
_fchdir;
__sys_fchdir;
_fchflags;
diff --git a/libexec/rtld-elf/rtld_lock.c b/libexec/rtld-elf/rtld_lock.c
index d1563e5..50c52c6 100644
--- a/libexec/rtld-elf/rtld_lock.c
+++ b/libexec/rtld-elf/rtld_lock.c
@@ -43,6 +43,7 @@
*/
#include
+#include
#include
#include
#include
@@ -59,8 +60,8 @@ typedef struct Struct_Lock {
void *base;
} Lock;
-static sigset_t fullsigmask, oldsigmask;
static int thread_flag;
+static uint32_t fsigblock;
static void *
def_lock_create()
@@ -111,18 +112,26 @@ def_rlock_acquire(void *lock)
}
static void
+sig_fastunblock(void)
+{
+ uint32_t oldval;
+
+ oldval = atomic_fetchadd_32(&fsigblock, -FAST_SIGBLOCK_INC);
+ if (oldval == (FAST_SIGBLOCK_PEND | FAST_SIGBLOCK_INC))
+ __sys_fast_sigblock(FAST_SIGBLOCK_UNBLOCK, NULL);
+}
+
+static void
def_wlock_acquire(void *lock)
{
Lock *l = (Lock *)lock;
-sigset_t tmp_oldsigmask;
for ( ; ; ) {
- sigprocmask(SIG_BLOCK, &fullsigmask, &tmp_oldsigmask);
+ atomic_add_rel_32(&fsigblock, FAST_SIGBLOCK_INC);
if (atomic_cmpset_acq_int(&l->lock, 0, WAFLAG))
break;
- sigprocmask(SIG_SETMASK, &tmp_oldsigmask, NULL);
+ sig_fastunblock();
}
-oldsigmask = tmp_oldsigmask;
}
static void
@@ -134,7 +143,7 @@ def_lock_release(void *lock)
atomic_add_rel_int(&l->lock, -RC_INCR);
else {
atomic_add_rel_int(&l->lock, -WAFLAG);
- sigprocmask(SIG_SETMASK, &oldsigmask, NULL);
+ sig_fastunblock();
}
}
@@ -286,19 +295,7 @@ lockdflt_init()
memcpy(&lockinfo, &deflockinfo, sizeof(lockinfo));
_rtld_thread_init(NULL);
-/*
- * Construct a mask to block all signals except traps which might
- * conceivably be generated within the dynamic linker itself.
- */
-sigfillset(&fullsigmask);
-sigdelset(&fullsigmask, SIGILL);
-sigdelset(&fullsigmask, SIGTRAP);
-sigdelset(&fullsigmask, SIGABRT);
-sigdelset(&fullsigmask, SIGEMT);
-sigdelset(&fullsigmask, SIGFPE);
-sigdelset(&fullsigmask, SIGBUS);
-sigdelset(&fullsigmask, SIGSEGV);
-sigdelset(&fullsigmask, SIGSYS);
+__sys_fast_sigblock(FAST_SIGBLOCK_SETPTR, &fsigblock);
}
/*
@@ -319,7 +316,10 @@ _rtld_thread_init(struct RtldLockInfo *pli)
if (pli == NULL)
pli = &deflockinfo;
-
+ else {
+ fsigblock = 0;
+ __sys_fast_sigblock(FAST_SIGBLOCK_UNSETPTR, NULL);
+ }
for (i = 0; i < RTLD_LOCK_CNT; i++)
if ((locks[i] = pli->lock_create()) =
Re: Fast sigblock (AKA rtld speedup)
On Mon, Jan 07, 2013 at 12:18:41PM -0800, Julian Elischer wrote: > On 1/7/13 10:22 AM, Konstantin Belousov wrote: > > Below is the forward of the patch for which I failed to obtain a private > > review. Might be, the list generates more responses. > > > > Our rtld has a performance bootleneck, typically exposed by the images > > with the lot of the run-time relocation processing, and by the C++ > > exception handling. We block the signals delivery during the rtld > > performing the relocations, as well as for the dl_iterate_phdr(3) (the > > later is used for finding the dwarf unwinding tables). > > > > The signal blocking is needed to allow the rtld to resolve the symbols > > for the signal handlers in the safe way, but also causes 2 syscalls > > overhead per each rtld entry. > > > > The proposed approach allows to shave off those two syscalls, doubling > > the FreeBSD performance for the (silly) throw/catch C++ microbenchmark. > > > > Date: Mon, 13 Aug 2012 15:26:00 +0300 > > From: Konstantin Belousov > > > > ... > > > > The basic idea is to implement sigprocmask() as single write into usermode > > address. If kernel needs to calculate the signal mask for current thread, > > it takes into the consideration non-zero value of the word at the agreed > > address. Also, usermode is informed about signals which were put on hold > > due to fast sigblock active. > > > > As I said, on my measurements in microbenchmark that did throw/catch in > > a loop, I see equal user and system time spent for unpatched system, and > > same user time with zero system time on patched system. > > > > Patch can be improved further, e.g. it would be nice to allow rtld to fall > > back to sigprocmask(2) if kernel does not support fast sigblock, to prevent > > flag day. Also, the mask enforced by fast sigblock can be made configurable. > > > > Note that libthr already blocks signals by catching them, and not using rtld > > service in the first line handler. I tried to make the change in the spirit > > of libthr interceptors, but handoff to libthr appears too complicated to > > work. In fact, libthr can be changed to start using fast sigblock instead > > of wrapping sigaction, but this is out of scope of the proposal right now. > Is there any danger that an upriveliged user program can trick the kernel > into doing anything it shouldn't by manipulating either the agreed upon > address or the contents of the mask at the address? (even reading > something by seeing what sigs get masked) I do not see how would anything like this possible. > > This was an issue with the M:N threading package and resulted in extra > syscalls > to get around the problem. (I forget the details). Would be great to dig up the details indeed. I cannot comment otherwise. pgpY8Fi8Hqf9Z.pgp Description: PGP signature
Re: Fast sigblock (AKA rtld speedup)
On Mon, Jan 07, 2013 at 03:42:01PM -0500, Alfred Perlstein wrote: > On 1/7/13 1:22 PM, Konstantin Belousov wrote: > > Below is the forward of the patch for which I failed to obtain a private > > review. Might be, the list generates more responses. > Very cool. > > Sorry for being rusty here, but is it safe to call fuword in the middle > of issignal()? > > The reason I ask is because it looks like proc lock is required for this > block, and I forget it fuword(9) can fault. > > If fuword(9) can fault then I don't think you can do this. > > You'll need to use something like fuswintr(9) or wire the page down > using vslock(9) or something. > > I am probably missing something? No, you are completely right. This is a serious bug. Thank you. I was careful to code the td_sigblock_val prefetch etc, but failed to do the update properly. There is no reason to execute the update in the issignal() at all, the setting of the pending bit can be postponed to the moment after the postsig() loop was executed. I uploaded the updated but not yet tested patch at http://people.freebsd.org/~kib/misc/rtld-sigblock.1.patch pgpcwNRftu9Ru.pgp Description: PGP signature
Re: Fast sigblock (AKA rtld speedup)
On Tue, Jan 08, 2013 at 01:09:51PM +0800, David Xu wrote:
> On 2013/01/08 02:22, Konstantin Belousov wrote:
> > Below is the forward of the patch for which I failed to obtain a private
> > review. Might be, the list generates more responses.
> >
> > Our rtld has a performance bootleneck, typically exposed by the images
> > with the lot of the run-time relocation processing, and by the C++
> > exception handling. We block the signals delivery during the rtld
> > performing the relocations, as well as for the dl_iterate_phdr(3) (the
> > later is used for finding the dwarf unwinding tables).
> >
> > The signal blocking is needed to allow the rtld to resolve the symbols
> > for the signal handlers in the safe way, but also causes 2 syscalls
> > overhead per each rtld entry.
> >
> > The proposed approach allows to shave off those two syscalls, doubling
> > the FreeBSD performance for the (silly) throw/catch C++ microbenchmark.
> >
> > Date: Mon, 13 Aug 2012 15:26:00 +0300
> > From: Konstantin Belousov
> >
> > ...
> >
> > The basic idea is to implement sigprocmask() as single write into usermode
> > address. If kernel needs to calculate the signal mask for current thread,
> > it takes into the consideration non-zero value of the word at the agreed
> > address. Also, usermode is informed about signals which were put on hold
> > due to fast sigblock active.
> >
> > As I said, on my measurements in microbenchmark that did throw/catch in
> > a loop, I see equal user and system time spent for unpatched system, and
> > same user time with zero system time on patched system.
> >
> > Patch can be improved further, e.g. it would be nice to allow rtld to fall
> > back to sigprocmask(2) if kernel does not support fast sigblock, to prevent
> > flag day. Also, the mask enforced by fast sigblock can be made configurable.
> >
> > Note that libthr already blocks signals by catching them, and not using rtld
> > service in the first line handler. I tried to make the change in the spirit
> > of libthr interceptors, but handoff to libthr appears too complicated to
> > work. In fact, libthr can be changed to start using fast sigblock instead
> > of wrapping sigaction, but this is out of scope of the proposal right now.
> >
> > Please comment.
> >
> > diff --git a/lib/libc/sys/Symbol.map b/lib/libc/sys/Symbol.map
> > index 6888ea0..3b75539 100644
> > --- a/lib/libc/sys/Symbol.map
> > +++ b/lib/libc/sys/Symbol.map
> > @@ -546,6 +546,7 @@ FBSDprivate_1.0 {
> > __sys_extattr_set_link;
> > _extattrctl;
> > __sys_extattrctl;
> > + __sys_fast_sigblock;
> > _fchdir;
> > __sys_fchdir;
> > _fchflags;
> > diff --git a/libexec/rtld-elf/rtld_lock.c b/libexec/rtld-elf/rtld_lock.c
> > index d1563e5..50c52c6 100644
> > --- a/libexec/rtld-elf/rtld_lock.c
> > +++ b/libexec/rtld-elf/rtld_lock.c
> > @@ -43,6 +43,7 @@
> >*/
> >
> > #include
> > +#include
> > #include
> > #include
> > #include
> > @@ -59,8 +60,8 @@ typedef struct Struct_Lock {
> > void *base;
> > } Lock;
> >
> > -static sigset_t fullsigmask, oldsigmask;
> > static int thread_flag;
> > +static uint32_t fsigblock;
> >
> > static void *
> > def_lock_create()
> > @@ -111,18 +112,26 @@ def_rlock_acquire(void *lock)
> > }
> >
> > static void
> > +sig_fastunblock(void)
> > +{
> > + uint32_t oldval;
> > +
> > + oldval = atomic_fetchadd_32(&fsigblock, -FAST_SIGBLOCK_INC);
> > + if (oldval == (FAST_SIGBLOCK_PEND | FAST_SIGBLOCK_INC))
> > + __sys_fast_sigblock(FAST_SIGBLOCK_UNBLOCK, NULL);
> > +}
> > +
> > +static void
> > def_wlock_acquire(void *lock)
> > {
> > Lock *l = (Lock *)lock;
> > -sigset_t tmp_oldsigmask;
> >
> > for ( ; ; ) {
> > - sigprocmask(SIG_BLOCK, &fullsigmask, &tmp_oldsigmask);
> > + atomic_add_rel_32(&fsigblock, FAST_SIGBLOCK_INC);
> > if (atomic_cmpset_acq_int(&l->lock, 0, WAFLAG))
> > break;
> > - sigprocmask(SIG_SETMASK, &tmp_oldsigmask, NULL);
> > + sig_fastunblock();
> > }
> > -oldsigmask = tmp_oldsigmask;
> > }
> >
> > static void
> > @@ -134,7 +143,7 @@ def_lock_release(void *lock)
> > atomic_add_rel_int(&l->lock, -RC_INCR);
> > else {
> > atomic_add_rel_int(&l->lock, -WAFLAG);
> > - sigpro
Re: Fast sigblock (AKA rtld speedup)
On Tue, Jan 08, 2013 at 01:21:48AM -0500, Alfred Perlstein wrote: > On 1/7/13 7:08 PM, Konstantin Belousov wrote: > > On Mon, Jan 07, 2013 at 03:42:01PM -0500, Alfred Perlstein wrote: > >> On 1/7/13 1:22 PM, Konstantin Belousov wrote: > >>> Below is the forward of the patch for which I failed to obtain a private > >>> review. Might be, the list generates more responses. > >> Very cool. > >> > >> Sorry for being rusty here, but is it safe to call fuword in the middle > >> of issignal()? > >> > >> The reason I ask is because it looks like proc lock is required for this > >> block, and I forget it fuword(9) can fault. > >> > >> If fuword(9) can fault then I don't think you can do this. > >> > >> You'll need to use something like fuswintr(9) or wire the page down > >> using vslock(9) or something. > >> > >> I am probably missing something? > > No, you are completely right. This is a serious bug. Thank you. > > > > I was careful to code the td_sigblock_val prefetch etc, but failed > > to do the update properly. There is no reason to execute the update > > in the issignal() at all, the setting of the pending bit can be postponed > > to the moment after the postsig() loop was executed. > > > > I uploaded the updated but not yet tested patch at > > http://people.freebsd.org/~kib/misc/rtld-sigblock.1.patch > > This new patch looks like it may have issues with PSTOP. > > there is a lot of code in issignal() that is missed when this code is in > place, I have not audited the effect of this, are you sure this is what > will work for the majority of cases? What exactly you are concerned with ? Note that fastblock_mask manipulations right before setting of TDP_FAST_SIGPENDING is to disallow the postponing of the delivery of (potentially) synchronous signals caused by traps, as well as SIGKILL/SIGSTOP. > > Honestly, if I were coding it for correctness I would vslock the pages > (or otherwise wire them in) and let issignal() behave normally and not > early exit from it. vslock() is racy, causes user address space fragmentation and was simply impossibly to use correctly until very recent. pgpnmAWeG6ebj.pgp Description: PGP signature
Re: Fast sigblock (AKA rtld speedup)
On Tue, Jan 08, 2013 at 10:02:22AM -0500, Alfred Perlstein wrote: > I think it would be great if it works, I just haven't had time to fully > understand your more recent patch. If you feel it's safe and maybe do a > few test programs to see what happens, that would be best. I revived the test programs I used during the development of the patch, see http://people.freebsd.org/~kib/misc/fast_sigblock.c http://people.freebsd.org/~kib/misc/fast_sigblock2.c Also, I adapted the code to the specific case you asked about, in http://people.freebsd.org/~kib/misc/fast_sigblock3.c When run from the shell, it reports 'Killed'. pgp4Dcuv38C5e.pgp Description: PGP signature
Re: Fast sigblock (AKA rtld speedup)
On Mon, Jan 07, 2013 at 08:22:35PM +0200, Konstantin Belousov wrote: > Below is the forward of the patch for which I failed to obtain a private > review. Might be, the list generates more responses. > > Our rtld has a performance bootleneck, typically exposed by the images > with the lot of the run-time relocation processing, and by the C++ > exception handling. We block the signals delivery during the rtld > performing the relocations, as well as for the dl_iterate_phdr(3) (the > later is used for finding the dwarf unwinding tables). > > The signal blocking is needed to allow the rtld to resolve the symbols > for the signal handlers in the safe way, but also causes 2 syscalls > overhead per each rtld entry. > > The proposed approach allows to shave off those two syscalls, doubling > the FreeBSD performance for the (silly) throw/catch C++ microbenchmark. > > Date: Mon, 13 Aug 2012 15:26:00 +0300 > From: Konstantin Belousov > > ... > > The basic idea is to implement sigprocmask() as single write into usermode > address. If kernel needs to calculate the signal mask for current thread, > it takes into the consideration non-zero value of the word at the agreed > address. Also, usermode is informed about signals which were put on hold > due to fast sigblock active. > > As I said, on my measurements in microbenchmark that did throw/catch in > a loop, I see equal user and system time spent for unpatched system, and > same user time with zero system time on patched system. > > Patch can be improved further, e.g. it would be nice to allow rtld to fall > back to sigprocmask(2) if kernel does not support fast sigblock, to prevent > flag day. Also, the mask enforced by fast sigblock can be made configurable. > > Note that libthr already blocks signals by catching them, and not using rtld > service in the first line handler. I tried to make the change in the spirit > of libthr interceptors, but handoff to libthr appears too complicated to > work. In fact, libthr can be changed to start using fast sigblock instead > of wrapping sigaction, but this is out of scope of the proposal right now. > > Please comment. So there were no overly negative comments, and thanks to Alfred and David for useful notes. The patch at http://people.freebsd.org/~kib/misc/rtld-sigblock.2.patch is the commit candidate. Now kernel informs rtld about supprt for fast_sigblock with new auxv flag. Rtld can operate on old (and possibly future) kernels without fast_sigblock, rtld checks the auxv for presence of the ELF_BSDF_FASTSIGBLK flag before use. pgpOaOnYDPcNK.pgp Description: PGP signature
Re: Fast sigblock (AKA rtld speedup)
On Fri, Jan 11, 2013 at 06:23:44PM +0800, David Xu wrote: > On 2013/01/11 17:54, Konstantin Belousov wrote: > > On Mon, Jan 07, 2013 at 08:22:35PM +0200, Konstantin Belousov wrote: > >> Below is the forward of the patch for which I failed to obtain a private > >> review. Might be, the list generates more responses. > >> > >> Our rtld has a performance bootleneck, typically exposed by the images > >> with the lot of the run-time relocation processing, and by the C++ > >> exception handling. We block the signals delivery during the rtld > >> performing the relocations, as well as for the dl_iterate_phdr(3) (the > >> later is used for finding the dwarf unwinding tables). > >> > >> The signal blocking is needed to allow the rtld to resolve the symbols > >> for the signal handlers in the safe way, but also causes 2 syscalls > >> overhead per each rtld entry. > >> > >> The proposed approach allows to shave off those two syscalls, doubling > >> the FreeBSD performance for the (silly) throw/catch C++ microbenchmark. > >> > >> Date: Mon, 13 Aug 2012 15:26:00 +0300 > >> From: Konstantin Belousov > >> > >> ... > >> > >> The basic idea is to implement sigprocmask() as single write into usermode > >> address. If kernel needs to calculate the signal mask for current thread, > >> it takes into the consideration non-zero value of the word at the agreed > >> address. Also, usermode is informed about signals which were put on hold > >> due to fast sigblock active. > >> > >> As I said, on my measurements in microbenchmark that did throw/catch in > >> a loop, I see equal user and system time spent for unpatched system, and > >> same user time with zero system time on patched system. > >> > >> Patch can be improved further, e.g. it would be nice to allow rtld to fall > >> back to sigprocmask(2) if kernel does not support fast sigblock, to prevent > >> flag day. Also, the mask enforced by fast sigblock can be made > >> configurable. > >> > >> Note that libthr already blocks signals by catching them, and not using > >> rtld > >> service in the first line handler. I tried to make the change in the spirit > >> of libthr interceptors, but handoff to libthr appears too complicated to > >> work. In fact, libthr can be changed to start using fast sigblock instead > >> of wrapping sigaction, but this is out of scope of the proposal right now. > >> > >> Please comment. > > > > So there were no overly negative comments, and thanks to Alfred and David > > for useful notes. > > > > The patch at > > http://people.freebsd.org/~kib/misc/rtld-sigblock.2.patch > > is the commit candidate. Now kernel informs rtld about supprt for > > fast_sigblock with new auxv flag. Rtld can operate on old (and possibly > > future) kernels without fast_sigblock, rtld checks the auxv for > > presence of the ELF_BSDF_FASTSIGBLK flag before use. > > > > The patch looks fine to me. Sorry, I still have a question: > if I found a process does not responsing to a signal, now should we > check both its signal mask and fast block value ? but is there such > a tool available ? for example, in gdb. Fair enough. I added the facility to export the address of the fast sigblock word to the tools, and implemented the procstat extension to print the address with -j. You can indeed use gdb to look up the content of the word when debugging, after getting it address from procstat. http://people.freebsd.org/~kib/misc/rtld-sigblock.3.patch pgpxHIzE6CFYK.pgp Description: PGP signature
Re: Fast sigblock (AKA rtld speedup)
On Sat, Jan 12, 2013 at 12:51:56AM +0400, Lev Serebryakov wrote: > Hello, Konstantin. > You wrote 11 января 2013 г., 13:54:59: > > > KB> http://people.freebsd.org/~kib/misc/rtld-sigblock.2.patch > KB> is the commit candidate. Now kernel informs rtld about supprt for > Is it first patch in this state, which implements idea of shared > memory between kernel and userland to spped up some syscalls? There > were discussions about this approach for speedup of > getpid()/getppid()/gettimeofday(), but without code ready for commit, > am I right? No, you are wrong. FreeBSD have shared page for very long time, and it is used for syscall-less gettimeofday for quite some time. Non-shared page which could carry per-thread or per-process information, neccessary for looking good in the trivial benchmarks, like getpid(), is different case. The patch does not implement the idea of speeding up some syscalls at all. It provides the special purpose one-off implementation of the alternate mechanism, for the internal consumption of rtld. pgpVNx9xwptSj.pgp Description: PGP signature
Re: Fast sigblock (AKA rtld speedup)
On Sat, Jan 12, 2013 at 12:29:06AM +0100, Jilles Tjoelker wrote: > On Fri, Jan 11, 2013 at 10:49:38PM +0200, Konstantin Belousov wrote: > > http://people.freebsd.org/~kib/misc/rtld-sigblock.3.patch > > The new fields td_sigblock_ptr and td_sigblock_val are in the part that > is zeroed for new threads, while the code in rtld appears to expect them > to be copied (on fork, vfork and pthread_create). The fields are > correctly zeroed on exec. Thank you for noting this. Should be fixed in http://people.freebsd.org/~kib/misc/rtld-sigblock.4.patch > > Sharing the magic variable between threads means that one thread holding > an rtld lock will block signals for all other threads as well. This is > different from how the normal signal mask works but I don't know whether > it may break things. However, the patch depends on it in some way > because sigtd() does not know about fast sigblock and will therefore > happily select a thread that is fast-sigblocking to handle a signal > directed at the process. Hm, I do not quite follow, at least not the first part of the paragraph. The fast sigblock pointer is per-thread, so it is not shared in the kernel. Regardless of the kernel side, rtld is only supposed to use the fast block in the default implementation of the rtld locks, which are overriden by the libthr implementation on libthr initialization. There is also an explicit hand-off from the default locks to the external (libthr), and rtld cares to turn off fast sigblock mechanism when the handoff is performed. The selection of the thread for the delivery of signal in the mt process should not matter then, due to the mechanism not supposed to be used in the mt process. > > Although libthr's postpone approach is somewhat ugly, it does not depend > on any non-standard kernel features and does not delay the default > action. Would it be possible to move that code to libc to make things > easier for rtld? It looks like this requires teaching libc about various > threading concepts, though. The concern there is that rtld would need to have a tight coupling with libc, and possibly with libthr too. > > Something feels ugly about not allowing applications to use this, > although I don't know how it would work. On the other hand, the strange > signal state created by this and implicit assumptions that the duration > will be short may be a reason to restrict its use to a relatively small > piece of code. Application use of the interface is equivalent to the application willingly corrupt its own state. pgpCtT7qS2N0Q.pgp Description: PGP signature
Re: Fast sigblock (AKA rtld speedup)
On Sat, Jan 12, 2013 at 08:27:06AM +0800, David Xu wrote: > On 2013/01/12 07:29, Jilles Tjoelker wrote: > > On Fri, Jan 11, 2013 at 10:49:38PM +0200, Konstantin Belousov wrote: > >> http://people.freebsd.org/~kib/misc/rtld-sigblock.3.patch > > The new fields td_sigblock_ptr and td_sigblock_val are in the part that > > is zeroed for new threads, while the code in rtld appears to expect them > > to be copied (on fork, vfork and pthread_create). The fields are > > correctly zeroed on exec. > > > > Sharing the magic variable between threads means that one thread holding > > an rtld lock will block signals for all other threads as well. This is > > different from how the normal signal mask works but I don't know whether > > it may break things. However, the patch depends on it in some way > > because sigtd() does not know about fast sigblock and will therefore > > happily select a thread that is fast-sigblocking to handle a signal > > directed at the process. > > > > Although libthr's postpone approach is somewhat ugly, it does not depend > > on any non-standard kernel features and does not delay the default > > action. Would it be possible to move that code to libc to make things > > easier for rtld? It looks like this requires teaching libc about various > > threading concepts, though. > Long time ago, if i remembered correctly, kib said that he wanted to > merge libthr code into libc, I don't know its state. Yes, this is correct, and I still want this. > > > Something feels ugly about not allowing applications to use this, > > although I don't know how it would work. On the other hand, the strange > > signal state created by this and implicit assumptions that the duration > > will be short may be a reason to restrict its use to a relatively small > > piece of code. > > > True, it seems it is for short duration. pgp2iKMPHHcG5.pgp Description: PGP signature
Re: Fast sigblock (AKA rtld speedup)
On Sat, Jan 12, 2013 at 05:25:47PM +0100, Jilles Tjoelker wrote: > On Sat, Jan 12, 2013 at 07:31:48AM +0200, Konstantin Belousov wrote: > > On Sat, Jan 12, 2013 at 12:29:06AM +0100, Jilles Tjoelker wrote: > > > On Fri, Jan 11, 2013 at 10:49:38PM +0200, Konstantin Belousov wrote: > > > > http://people.freebsd.org/~kib/misc/rtld-sigblock.3.patch > > > > The new fields td_sigblock_ptr and td_sigblock_val are in the part that > > > is zeroed for new threads, while the code in rtld appears to expect them > > > to be copied (on fork, vfork and pthread_create). The fields are > > > correctly zeroed on exec. > > Thank you for noting this. Should be fixed in > > http://people.freebsd.org/~kib/misc/rtld-sigblock.4.patch > > > > Sharing the magic variable between threads means that one thread holding > > > an rtld lock will block signals for all other threads as well. This is > > > different from how the normal signal mask works but I don't know whether > > > it may break things. However, the patch depends on it in some way > > > because sigtd() does not know about fast sigblock and will therefore > > > happily select a thread that is fast-sigblocking to handle a signal > > > directed at the process. > > > Hm, I do not quite follow, at least not the first part of the paragraph. > > > The fast sigblock pointer is per-thread, so it is not shared in the kernel. > > Regardless of the kernel side, rtld is only supposed to use the fast > > block in the default implementation of the rtld locks, which are overriden > > by the libthr implementation on libthr initialization. There is also an > > explicit hand-off from the default locks to the external (libthr), and > > rtld cares to turn off fast sigblock mechanism when the handoff is > > performed. > > > The selection of the thread for the delivery of signal in the mt process > > should not matter then, due to the mechanism not supposed to be used > > in the mt process. > > OK, you are right. If multiple threads exist, the patched code disables > fast sigblock and the current unpatched code already postpones signal > handlers without sigprocmask() syscalls. > > This suggests a different rather simpler change. Libthr can always use > its rtld lock implementation instead of only when multiple threads > exist. This avoids the sigprocmask() syscalls and should not be much > slower than the default implementation in rtld because that also > contains atomic operations (both the unpatched and the patched version). > People that care about performance of exceptions can then link in libthr > (in many cases, it is already linked in to allow for (the possibility > of) threading). > > I have tested this and exceptions were indeed more than twice as fast in > my test program if I created an extra thread doing nothing than in the > fully single-threaded version (with or without libthr). > > With that, I think fast sigblock is too much code and complication for a > niche case. Ok, thank you for the review. I am reverting the branch. I think a solution would need to wait while the work to merge libc and libthr is done. > > Most of the extra atomics in multi-threaded applications are conditional > on __isthreaded (or can be made so); therefore, performance loss from > linking in libthr should be negligible in most cases. > > > > Although libthr's postpone approach is somewhat ugly, it does not depend > > > on any non-standard kernel features and does not delay the default > > > action. Would it be possible to move that code to libc to make things > > > easier for rtld? It looks like this requires teaching libc about various > > > threading concepts, though. > > > The concern there is that rtld would need to have a tight coupling > > with libc, and possibly with libthr too. > > Libc could call _rtld_thread_init() during initialization, somewhat like > libthr does now. The problems here are in libc and libthr, not in rtld. > Before libc is ready, the current code can be used; if libc is always > initialized first, the sigprocmask() calls can be removed because there > is no way a signal handler can be installed that early. > > -- > Jilles Tjoelker pgpJWz_jkPqv2.pgp Description: PGP signature
Re: LLVM Image Activator
On Sun, Jan 13, 2013 at 12:41:09PM +0100, Ed Schouten wrote: > Hi Kostik, > > 2013/1/7 Konstantin Belousov : > > I still do remember the buzz about the binary format 0xCAFEBABE, which > > AFAIR gained image activator support on several OSes, to be garbage > > collected. > > Maybe it would then be a good idea then to add some kind of general > purpose remapping imgact? Example: > > /etc/imgacttab: > > cafebabe /usr/local/bin/java > cffaedfe /usr/local/bin/osx_emulator > 4243c0de /usr/bin/lli > > That way we still give people the freedom to play around with mapping > their own executable formats, but don't need to maintain a bunch of > imgacts. A generic module that could be somewhat customized at runtime to map offset+signature into the shebang path could be a possibility indeed. I strongly prefer to have it as module and not enabled by default. Asking Nathan for writing the thing is too much, IMHO, esp. in the response to the 50-lines hack. pgpczGgp_E_rw.pgp Description: PGP signature
Re: Fast sigblock (AKA rtld speedup)
On Sun, Jan 13, 2013 at 02:32:00PM +0100, Jilles Tjoelker wrote:
> On Sat, Jan 12, 2013 at 05:25:47PM +0100, Jilles Tjoelker wrote:
> > This suggests a different rather simpler change. Libthr can always use
> > its rtld lock implementation instead of only when multiple threads
> > exist. This avoids the sigprocmask() syscalls and should not be much
> > slower than the default implementation in rtld because that also
> > contains atomic operations (both the unpatched and the patched version).
> > People that care about performance of exceptions can then link in libthr
> > (in many cases, it is already linked in to allow for (the possibility
> > of) threading).
>
> > I have tested this and exceptions were indeed more than twice as fast in
> > my test program if I created an extra thread doing nothing than in the
> > fully single-threaded version (with or without libthr).
>
> Here is a patch. It is lightly tested.
>
> The function _thr_rtld_fini() can be removed afterwards because it is no
> longer used.
>
> Index: lib/libthr/thread/thr_init.c
> ===
> --- lib/libthr/thread/thr_init.c (revision 244639)
> +++ lib/libthr/thread/thr_init.c (working copy)
> @@ -363,6 +363,7 @@
> _thr_signal_init();
> if (_thread_event_mask & TD_CREATE)
> _thr_report_creation(curthread, curthread);
> + _thr_rtld_init();
Please add a comment there, describing the reason for initializing
threaded rtld locks.
> }
> }
>
> Index: lib/libthr/thread/thr_kern.c
> ===
> --- lib/libthr/thread/thr_kern.c (revision 244639)
> +++ lib/libthr/thread/thr_kern.c (working copy)
> @@ -57,11 +57,6 @@
> return (0);
>
> __isthreaded = threaded;
> - if (threaded != 0) {
> - _thr_rtld_init();
> - } else {
> - _thr_rtld_fini();
> - }
> return (0);
> }
I do not object.
pgpgZdGDX6FzP.pgp
Description: PGP signature
Re: Fast sigblock (AKA rtld speedup)
On Sun, Jan 13, 2013 at 07:50:14AM -0700, Ian Lepore wrote: > On Sat, 2013-01-12 at 15:04 -0800, John-Mark Gurney wrote: > > David Xu wrote this message on Tue, Jan 08, 2013 at 13:09 +0800: > > > and can not be freed until process is exited, the page is doubly > > > mapped into in kernel and userland, accessing the shared data > > > in kernel has zero overhead though. > > > > Don't forget that there are arches out there w/ VIVT caches which will > > probably eliminate most of the performance benifits if we have the same > > page mapped writable in two different virtual addresses.. > > > > Even worse than eliminate the benefits, since multiple mappings with one > writable disables caching on the whole page, there can be a big penalty > depending on what other data is nearby that suddenly becomes > uncacheable. I was initially very interested in the work to read system > clocks without a syscall until I realized it was going to suffer from > the same problem. Since only kernel writes to the shared page, it should be not a problem. At least there is a specific point where you can insert the neccessary cache flush commands. Also, I suspect that even for arms, the writes are executed from the same core, which could simplify things even more. pgpRq3ZqxpUAh.pgp Description: PGP signature
Re: LLVM Image Activator
On Sun, Jan 13, 2013 at 08:21:37AM -0800, Nathan Whitehorn wrote: > On 01/13/13 05:20, Konstantin Belousov wrote: > > On Sun, Jan 13, 2013 at 12:41:09PM +0100, Ed Schouten wrote: > >> Hi Kostik, > >> > >> 2013/1/7 Konstantin Belousov : > >>> I still do remember the buzz about the binary format 0xCAFEBABE, which > >>> AFAIR gained image activator support on several OSes, to be garbage > >>> collected. > >> > >> Maybe it would then be a good idea then to add some kind of general > >> purpose remapping imgact? Example: > >> > >> /etc/imgacttab: > >> > >> cafebabe /usr/local/bin/java > >> cffaedfe /usr/local/bin/osx_emulator > >> 4243c0de /usr/bin/lli > >> > >> That way we still give people the freedom to play around with mapping > >> their own executable formats, but don't need to maintain a bunch of > >> imgacts. > > > > A generic module that could be somewhat customized at runtime to map > > offset+signature into the shebang path could be a possibility indeed. > > I strongly prefer to have it as module and not enabled by default. > > > > Asking Nathan for writing the thing is too much, IMHO, esp. in > > the response to the 50-lines hack. > > > > I think this is a good idea, since it both prevents a profusion of > similar activators and works nicely in jails and similar environments. I > probably won't write it quickly, but it should not take more than about > 50 lines, so I can't imagine it will be that bad. There are some > complications with this kind of design from the things in the XXX > comment in imgact_llvm.c about handling argv[0] that I need to think > some more about. Great. I do not believe in the 50 lines, but I am happy that you want to work this out. > > Why are you opposed to having it there by default? I think it's actually > quite important that it be there by default. Having it not "standard" > would be fine, but it should at least be in GENERIC. There are minimal > security risks since it just munges begin_argv and doesn't even load the > executable and it's little enough code that there should not be any > kernel bloat to speak of. If things like this aren't enabled by default, > no one can depend on them being there, no one will use it, and the point > is entirely lost. All image activators demonstrated a constant stream of security holes. Even our ELF activator, and I was guilty there too. I definitely do not fight over the inclusion of the proposed activator into GENERIC, but do insist on the config option + module. pgp3Uq5ClzLFK.pgp Description: PGP signature
Re: Fast sigblock (AKA rtld speedup)
On Sun, Jan 13, 2013 at 10:46:27AM -0700, Ian Lepore wrote: > On Sun, 2013-01-13 at 17:06 +0200, Konstantin Belousov wrote: > > On Sun, Jan 13, 2013 at 07:50:14AM -0700, Ian Lepore wrote: > > > On Sat, 2013-01-12 at 15:04 -0800, John-Mark Gurney wrote: > > > > David Xu wrote this message on Tue, Jan 08, 2013 at 13:09 +0800: > > > > > and can not be freed until process is exited, the page is doubly > > > > > mapped into in kernel and userland, accessing the shared data > > > > > in kernel has zero overhead though. > > > > > > > > Don't forget that there are arches out there w/ VIVT caches which will > > > > probably eliminate most of the performance benifits if we have the same > > > > page mapped writable in two different virtual addresses.. > > > > > > > > > > Even worse than eliminate the benefits, since multiple mappings with one > > > writable disables caching on the whole page, there can be a big penalty > > > depending on what other data is nearby that suddenly becomes > > > uncacheable. I was initially very interested in the work to read system > > > clocks without a syscall until I realized it was going to suffer from > > > the same problem. > > > > Since only kernel writes to the shared page, it should be not a problem. > > At least there is a specific point where you can insert the neccessary > > cache flush commands. > > With VIVT cache, all mappings of a page must be uncacheable no matter > which one is writable. It applies even if all the mappings belong to > the kernel (this is why the wired writable pages in the buffer cache > kill executable performance with VIVT caches). > > There's no way to keep a VIVT cache coherent when there are multiple > mappings, because of the virtual indexing: you write to a page, then do > a flush based on that page's virtual address, and there's no way find > cache lines which alias the same physical memory using different virtual > addresses to flush them too. No, you do know all the mapping addresses in this special case. The shared page is mapped at the fixed user address, and at some kernel address. You obviously need to flush both cache lines. > > > Also, I suspect that even for arms, the writes are executed from the > > same core, which could simplify things even more. > > The arm chips that are affected by this don't have multiple cores (at > least, I don't think there are any VIVT cache multi-core arms, it's > probably not even possible). > > -- Ian > pgpPbJpbi4JJQ.pgp Description: PGP signature
Re: patch to add aes and pclmulqdq instructions to gcc
On Wed, Jan 16, 2013 at 11:05:16PM -0800, John-Mark Gurney wrote: > Mike Belopuhov pointed me to the patch in OpenBSD: > http://freshbsd.org/commit/openbsd/0babc91a00b1f1953637bb39c8ec97aef704629e/diff.txt > > While OpenBSD's binutils is quite different than FreeBSD's, I was able > to use his patch to teach binutils how to assemble and disassemble the > aes and pclmulqdq instructions. > > I have done basic tests, such as verified that it can assemble the aesni > module and get the same results, and assemble a sample file for > pclmulqdq.. For each of these tests, I have verified that it's output > matches (as close as possible, as gcc/clang compile callq's differently) > clang on amd64.. Did you removed the manually assembled bytes from aes*.S and replaced them with the commented-out instructions for the test ? There is also newer VEX encoding for the same AESNI set, but adding the support for them might be hard because the source base is old. pgppCIjL2Z9ek.pgp Description: PGP signature
Re: patch to add aes and pclmulqdq instructions to gcc
On Thu, Jan 17, 2013 at 09:31:40AM -0800, John-Mark Gurney wrote: > Konstantin Belousov wrote this message on Thu, Jan 17, 2013 at 13:12 +0200: > > On Wed, Jan 16, 2013 at 11:05:16PM -0800, John-Mark Gurney wrote: > > > Mike Belopuhov pointed me to the patch in OpenBSD: > > > http://freshbsd.org/commit/openbsd/0babc91a00b1f1953637bb39c8ec97aef704629e/diff.txt > > > > > > While OpenBSD's binutils is quite different than FreeBSD's, I was able > > > to use his patch to teach binutils how to assemble and disassemble the > > > aes and pclmulqdq instructions. > > > > > > I have done basic tests, such as verified that it can assemble the aesni > > > module and get the same results, and assemble a sample file for > > > pclmulqdq.. For each of these tests, I have verified that it's output > > > matches (as close as possible, as gcc/clang compile callq's differently) > > > clang on amd64.. > > Did you removed the manually assembled bytes from aes*.S and replaced them > > with the commented-out instructions for the test ? > > Yep.. Thank you, I expect that you will commit the crypto/aes change which removes the .bytes, after the gas patch. > > > There is also newer VEX encoding for the same AESNI set, but adding the > > support for them might be hard because the source base is old. > > I don't see a use for them right now. And considering how anoying it > was to add just these instructions.. :) I am not asking you to do more work. It just my understanding that Intel wants everybody migrate to VEX encoding, where available. pgpTCIoalxfkw.pgp Description: PGP signature
Re: standards/175453: Catching C++ std::bad_cast doesn't work in FreeBSD 9.1
On Sun, Jan 20, 2013 at 04:52:00PM +, Hongli Lai wrote:
>
> >Number: 175453
> >Category: standards
> >Synopsis: Catching C++ std::bad_cast doesn't work in FreeBSD 9.1
> >Confidential: no
> >Severity: non-critical
> >Priority: low
> >Responsible:freebsd-standards
> >State: open
> >Quarter:
> >Keywords:
> >Date-Required:
> >Class: sw-bug
> >Submitter-Id: current-users
> >Arrival-Date: Sun Jan 20 17:00:00 UTC 2013
> >Closed-Date:
> >Last-Modified:
> >Originator: Hongli Lai
> >Release:9.1-RELEASE
> >Organization:
> Phusion
> >Environment:
> FreeBSD freebsd9 9.1-RELEASE FreeBSD 9.1-RELEASE #0 r243825: Tue Dec 4
> 09:23:10 UTC 2012
> r...@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64
> >Description:
> C++ code is not able to catch std::bad_cast exceptions, even though it
> should. If a dynamic_cast is within a try-catch block, then that block fails
> to catch std::bad_cast, and the program crashes with an uncaught exception as
> a result.
>
> I've attached a reproducible test case. You can also find it at
> http://forums.freebsd.org/showthread.php?p=205804#post205804 and
> http://stackoverflow.com/questions/14413703/why-does-catching-stdbad-cast-not-work-on-freebsd-9.
> The code is compiled with the following GCC version:
>
> $ gcc -v
> Using built-in specs.
> Target: amd64-undermydesk-freebsd
> Configured with: FreeBSD/amd64 system compiler
> Thread model: posix
> gcc version 4.2.1 20070831 patched [FreeBSD]
>
> FreeBSD 9.1 seems to be the only platform on which this bug appears. The code
> works as expected on Linux and OS X. According to a commenter, FreeBSD 9.0
> works as expected too. According to another commenter the code fails on
> FreeBSD 9.1 with Clang too.
> >How-To-Repeat:
> See attached C++ program.
> >Fix:
>
>
> Patch attached with submission follows:
>
> #include
> #include
> #include
>
> class foo {
> public:
> virtual ~foo() {}
> };
>
> class bar: public foo {
> public:
> int val;
> bar(): val(123) {}
> };
>
> static void
> cast_test(const foo &f) {
> try {
> const bar &b = dynamic_cast(f);
> printf("%d\n", b.val);
> } catch (const std::bad_cast &) {
> printf("bad cast\n");
> }
> }
>
> int main() {
> foo f;
> cast_test(f);
> return 0;
> }
>
Confirmed, and it seems that the culprit is libstdc++. At least replacing
the system libstdc++.so.6 with the library from the stock build of gcc
4.7.2 (without touching libgcc_s.so.1) makes the catch operator working.
pgp6p3P9N1dFv.pgp
Description: PGP signature
Re: standards/175453: Catching C++ std::bad_cast doesn't work in FreeBSD 9.1
On Sun, Jan 20, 2013 at 08:08:21PM -0800, Pedro Giffuni wrote:
>
>
>
>
> - Messaggio originale -
> > Da: Konstantin Belousov
> >
> > On Sun, Jan 20, 2013 at 04:52:00PM +, Hongli Lai wrote:
> >>
> >> >Number: 175453
> >> >Category: standards
> >> >Synopsis: Catching C++ std::bad_cast doesn't work in FreeBSD
> > 9.1
> >> >Confidential: no
> >> >Severity: non-critical
> >> >Priority: low
> >> >Responsible: freebsd-standards
> >> >State: open
> >> >Quarter:
> >> >Keywords:
> >> >Date-Required:
> >> >Class: sw-bug
> >> >Submitter-Id: current-users
> >> >Arrival-Date: Sun Jan 20 17:00:00 UTC 2013
> >> >Closed-Date:
> >> >Last-Modified:
> >> >Originator: Hongli Lai
> >> >Release: 9.1-RELEASE
> >> >Organization:
> >> Phusion
> >> >Environment:
> >> FreeBSD freebsd9 9.1-RELEASE FreeBSD 9.1-RELEASE #0 r243825: Tue Dec 4
> > 09:23:10 UTC 2012
> >> r...@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64
> >> >Description:
> >> C++ code is not able to catch std::bad_cast exceptions, even though it
> > should. If a dynamic_cast is within a try-catch block, then that block
> > fails to
> > catch std::bad_cast, and the program crashes with an uncaught exception as
> > a
> > result.
> >>
> >> I've attached a reproducible test case. You can also find it at
> > http://forums.freebsd.org/showthread.php?p=205804#post205804 and
> > http://stackoverflow.com/questions/14413703/why-does-catching-stdbad-cast-not-work-on-freebsd-9.
> >
> > The code is compiled with the following GCC version:
> >>
> >> $ gcc -v
> >> Using built-in specs.
> >> Target: amd64-undermydesk-freebsd
> >> Configured with: FreeBSD/amd64 system compiler
> >> Thread model: posix
> >> gcc version 4.2.1 20070831 patched [FreeBSD]
> >>
> >> FreeBSD 9.1 seems to be the only platform on which this bug appears. The
> > code works as expected on Linux and OS X. According to a commenter, FreeBSD
> > 9.0
> > works as expected too. According to another commenter the code fails on
> > FreeBSD
> > 9.1 with Clang too.
> >> >How-To-Repeat:
> >> See attached C++ program.
> >> >Fix:
> >>
> >>
> >> Patch attached with submission follows:
> >>
> >> #include
> >> #include
> >> #include
> >>
> >> class foo {
> >> public:
> >> virtual ~foo() {}
> >> };
> >>
> >> class bar: public foo {
> >> public:
> >> int val;
> >> bar(): val(123) {}
> >> };
> >>
> >> static void
> >> cast_test(const foo &f) {
> >> try {
> >> const bar &b = dynamic_cast(f);
> >> printf("%d\n", b.val);
> >> } catch (const std::bad_cast &) {
> >> printf("bad cast\n");
> >> }
> >> }
> >>
> >> int main() {
> >> foo f;
> >> cast_test(f);
> >> return 0;
> >> }
> >>
> > Confirmed, and it seems that the culprit is libstdc++. At least replacing
> > the system libstdc++.so.6 with the library from the stock build of gcc
> > 4.7.2 (without touching libgcc_s.so.1) makes the catch operator working.
> >
>
> Confirmed,
>
> The problem is actually not in any of the code updates to libstdc++ but
> in the way libstdc++ is configured/build since 9.0.
>
> Reverting the changes in stable/9/gnu/lib/libstdc up to r229037 things work
> just fine.
>
> I suspect the culprit is r233749 and subsequent changes to build libstdc++
> as a filter library for libsupc++.
>
> This also seems similar ot PR kern/171610.
Yes, quite possible. AFAIR, the 'catch' code compares the exception classes
by the shared object ownership. It might get confused due to filter providing
some symbols.
But I did not investigated the cause for real.
pgpGQEJFpdGFS.pgp
Description: PGP signature
Re: standards/175453: Catching C++ std::bad_cast doesn't work in FreeBSD 9.1
On Mon, Jan 21, 2013 at 04:12:00PM +, David Chisnall wrote: > On 21 Jan 2013, at 04:49, Konstantin Belousov wrote: > > > Yes, quite possible. AFAIR, the 'catch' code compares the exception classes > > by the shared object ownership. It might get confused due to filter > > providing > > some symbols. > > > > But I did not investigated the cause for real. > > The issue appears to be that the libstdc++ exports a few functions[1] that > libsupc++ exports, but with different symbol versions. Unfortunately, these > are things that set handlers that are then called from libsupc++ / libcxxrt > when, for example, an exception specification violation is encountered. > > I'm not sure what the solution is here. Is there some version-script-foo > that we can do to say 'filter this symbol with this version as if it were > this one with this version'? We ideally want to keep them with the current > version in libcxxrt / libsupc++, but not introduce linker errors. > > David > > [1] std::set_new_handler(), std::set_terminate(), std::set_unexpected() Can you prepare the minimal isolated test case which demostrates the behaviour ? A plan would be to ask somebody to run the test on the linux. I think we must be bug-to-bug compatible with glibc in regard to the filters quirks. Gnu filters work only on the whole object scope. Solaris linkers do have per-symbol filtering, but Solaris does not implement per-symbol versioning, on the other hand. pgpGQGG6ShR1E.pgp Description: PGP signature
Re: standards/175453: Catching C++ std::bad_cast doesn't work in FreeBSD 9.1
On Mon, Jan 21, 2013 at 05:09:25PM +, David Chisnall wrote: > On 21 Jan 2013, at 16:54, Konstantin Belousov wrote: > > > On Mon, Jan 21, 2013 at 04:12:00PM +, David Chisnall wrote: > >> On 21 Jan 2013, at 04:49, Konstantin Belousov wrote: > >> > >>> Yes, quite possible. AFAIR, the 'catch' code compares the exception > >>> classes > >>> by the shared object ownership. It might get confused due to filter > >>> providing > >>> some symbols. > >>> > >>> But I did not investigated the cause for real. > >> > >> The issue appears to be that the libstdc++ exports a few functions[1] that > >> libsupc++ exports, but with different symbol versions. Unfortunately, > >> these are things that set handlers that are then called from libsupc++ / > >> libcxxrt when, for example, an exception specification violation is > >> encountered. > >> > >> I'm not sure what the solution is here. Is there some version-script-foo > >> that we can do to say 'filter this symbol with this version as if it were > >> this one with this version'? We ideally want to keep them with the > >> current version in libcxxrt / libsupc++, but not introduce linker errors. > >> > >> David > >> > >> [1] std::set_new_handler(), std::set_terminate(), std::set_unexpected() > > > > Can you prepare the minimal isolated test case which demostrates the > > behaviour ? A plan would be to ask somebody to run the test on the linux. > > I think we must be bug-to-bug compatible with glibc in regard to the > > filters quirks. > > I'm not sure what you want the test case to demonstrate. I have a reduced > form of the exception test case, but it only fails for us because of the > symbol versioning issue. Moving the relevant symbols into the same version > in our libsupc++ and libcxxrt as in libstdc++ fixes it (and is probably the > right thing to do. On closer inspection both already expose new and delete > in this symbol version namespace, so it's fine to add a few more). > > > Gnu filters work only on the whole object scope. Solaris linkers do > > have per-symbol filtering, but Solaris does not implement per-symbol > > versioning, on the other hand. > > If you want a simple test case for the filter behaviour, the recipe is: > > Symbol X in filter with version A > Symbol X in filtee with version B > Symbol Y in program uses symbol X > > Program sees version A, filtee sees version B. So can you provide self-contained test.tgz with Makefile and neccessary .c files which demonstrate exactly the behaviour you see ? pgp3EjFIMKvpE.pgp Description: PGP signature
Re: standards/175453: Catching C++ std::bad_cast doesn't work in FreeBSD 9.1
On Mon, Jan 21, 2013 at 05:26:29PM +, David Chisnall wrote: > On 21 Jan 2013, at 17:24, Konstantin Belousov wrote: > > > So can you provide self-contained test.tgz with Makefile and neccessary > > .c files which demonstrate exactly the behaviour you see ? > > I don't have one, this is the behaviour that I see from C++ programs linked > to libstdc++. I understand that you do not have one. Can you write it ? [This is my last mail on the subject, it seems] pgpJUEz8GEDcQ.pgp Description: PGP signature
Re: [patch] DEBUG_FLAGS cleanup
On Wed, Jan 23, 2013 at 09:23:41AM +0100, Erik Cederstrand wrote:
> Hello folks,
>
> Attached is a patch to clean up unconditional use of "-g" in Makefiles,
> instead respecting the global DEBUG_FLAGS setting.
>
> I need this as part of my quest to support deterministic builds. Currently,
> debug information contains stuff like timestamps, absolute paths etc. that
> make binaries non-deterministic, and Clang lacks the necessary flags to
> rectify this. So I'd like DEBUG_FLAGS="" to actually work everywhere.
>
> I'd be thankful for feedback and help committing the changes if it's OK.
>
> Thanks,
> Erik
>
> Index: head/usr.bin/tar/Makefile
> ===
> --- head/usr.bin/tar/Makefile (revision 242791)
> +++ head/usr.bin/tar/Makefile (working copy)
> @@ -38,7 +38,6 @@
> CFLAGS+= -I${LIBARCHIVEDIR}/libarchive_fe
> SYMLINKS=bsdtar ${BINDIR}/tar
> MLINKS= bsdtar.1 tar.1
> -DEBUG_FLAGS=-g
>
> .PHONY: check test clean-test
> check test: $(PROG) bsdtar.1.gz
> Index: head/gnu/usr.bin/cc/cc_tools/Makefile
> ===
> --- head/gnu/usr.bin/cc/cc_tools/Makefile (revision 242791)
> +++ head/gnu/usr.bin/cc/cc_tools/Makefile (working copy)
> @@ -6,7 +6,7 @@
>
> .include "../Makefile.inc"
>
> -CFLAGS+= -g
> +CFLAGS+= $(DEBUG_FLAGS)
> CFLAGS+= -DGENERATOR_FILE -DHAVE_CONFIG_H
>
> # Override LIBIBERTY set by Makefile.inc, We use our own for
This fragment catched my eye, and it is typical for the patch.
Why do you need to add DEBUG_FLAGS to the CFLAGS manually ?
Wouldn't share/mk magic do this automatically ?
If not, I think that it should be considered to fix the remaining
cases where DEBUG_FLAGS are not added in the common mk code, instead
of sprinkling the additions all over the place ?
I think that the cleanup you performed is needed.
pgpTB8XBbjmD1.pgp
Description: PGP signature
Re: Removing default build of gcc
On Fri, Jan 25, 2013 at 08:41:11AM +, David Chisnall wrote: > Hi All, > > In 10.0, the plan is not to ship any GPL'd code, so I'd like to start > disconnecting things from the default build, starting with gcc. I've been > running a gcc-free system for a while, and I think all of the ports that > don't build with clang are now explicitly depending on gcc. Does anyone have > strong opinions on when would be a good time for head on x86 and x86-64 to > default to not building gcc? To clarify: there is no plans to not ship any GPLed code for 10.x. Instead, there are still plans to ship working 10.x. Please do not consider the personal opinion as the statement of the project policy. pgpdIEHYRparg.pgp Description: PGP signature
Re: Removing default build of gcc
On Fri, Jan 25, 2013 at 12:31:39PM -0700, Warner Losh wrote: > > On Jan 25, 2013, at 4:31 AM, Konstantin Belousov wrote: > > > On Fri, Jan 25, 2013 at 08:41:11AM +, David Chisnall wrote: > >> Hi All, > >> > >> In 10.0, the plan is not to ship any GPL'd code, so I'd like to start > >> disconnecting things from the default build, starting with gcc. I've been > >> running a gcc-free system for a while, and I think all of the ports that > >> don't build with clang are now explicitly depending on gcc. Does anyone > >> have strong opinions on when would be a good time for head on x86 and > >> x86-64 to default to not building gcc? > > > > To clarify: there is no plans to not ship any GPLed code for 10.x. > > Instead, there are still plans to ship working 10.x. > > > > Please do not consider the personal opinion as the statement of the project > > policy. > > The goal is to try not to ship GPL'd code in 10. The goal is not to ship 10 > without GPL'd code if that results in a broken system. The goal also as > articulated at different forum, was for Tier 1 systems. Tier 2 and 3 systems > may use GPL code as a fallback if the non-gpl'd code doesn't work on those > platforms. > > That is to say, it is a goal, not an absolute requirement. All you said is reasonable and quite coincides with what I thought. Unfortunately, it has very tangential relations to what is proposed to do and to the political agenda declared in the message started the thread. I am really tired of the constant struggle against the consumation of the FreeBSD as the test-bed for the pre-alpha quality software. E.g., are we fine with broken C++ runtime in 9 ? pgp_agmpE4euE.pgp Description: PGP signature
Re: Removing default build of gcc
On Fri, Jan 25, 2013 at 03:36:15PM -0500, Pedro Giffuni wrote: > On 01/25/2013 14:59, Konstantin Belousov wrote: > > On Fri, Jan 25, 2013 at 12:31:39PM -0700, Warner Losh wrote: > >> On Jan 25, 2013, at 4:31 AM, Konstantin Belousov wrote: > >> > >>> On Fri, Jan 25, 2013 at 08:41:11AM +, David Chisnall wrote: > >>>> Hi All, > >>>> > >>>> In 10.0, the plan is not to ship any GPL'd code, so I'd like to start > >>>> disconnecting things from the default build, starting with gcc. I've > >>>> been running a gcc-free system for a while, and I think all of the ports > >>>> that don't build with clang are now explicitly depending on gcc. Does > >>>> anyone have strong opinions on when would be a good time for head on x86 > >>>> and x86-64 to default to not building gcc? > >>> To clarify: there is no plans to not ship any GPLed code for 10.x. > >>> Instead, there are still plans to ship working 10.x. > >>> > >>> Please do not consider the personal opinion as the statement of the > >>> project > >>> policy. > >> The goal is to try not to ship GPL'd code in 10. The goal is not to ship > >> 10 without GPL'd code if that results in a broken system. The goal also as > >> articulated at different forum, was for Tier 1 systems. Tier 2 and 3 > >> systems may use GPL code as a fallback if the non-gpl'd code doesn't work > >> on those platforms. > >> > >> That is to say, it is a goal, not an absolute requirement. > > All you said is reasonable and quite coincides with what I thought. > > > > Unfortunately, it has very tangential relations to what is proposed to > > do and to the political agenda declared in the message started the thread. > > I don't care much about gcc in current. It doesn't seem like the right time > to kill it but it is a dead end and we should be using the pre pkg'ed > version > instead (I know, easier said than done, but the Debian guys did it). > > Either way, there is no hurry but it is a desirable goal. > > > I am really tired of the constant struggle against the consumation of > > the FreeBSD as the test-bed for the pre-alpha quality software. E.g., > > are we fine with broken C++ runtime in 9 ? > > The libstdc++ issue is really REALLY worrying. > I would prefer if the hack to attempt using libstdc++ as a filter > library were reverted altogether in 9.x. > > I had a lots of stress with that issue as some people pointed at > my libstdc++ updates as possible root cause. I felt some natural > relief when the real cause was found but I certainly wonder why > the change was made in 9.x though since it's clear that codebase > will not be migrated to libc++. You were finger-pointed due to the rule 'blame the last committer from the VCS log'. Even less so, you were asked about it because you probably knew most about possible cause. I am not worried about the bug itself, which needs a proper identification and fixing. I am indeed wery disappointed regarding the attitude of the person who introduced the bug. Reverting the split may be the best action in my opinion. Both in head and stable. pgp_TsCxdi6uk.pgp Description: PGP signature
Re: Removing default build of gcc
On Sat, Jan 26, 2013 at 10:23:58AM +, David Chisnall wrote: > On 25 Jan 2013, at 19:59, Konstantin Belousov wrote: > > > I am really tired of the constant struggle against the consumation of > > the FreeBSD as the test-bed for the pre-alpha quality software. E.g., > > are we fine with broken C++ runtime in 9 ? > > Please can you stop the FUD here? It really isn't helpful. We have a > working C++ runtime in 9.1: libcxxrt. In fact, we have a working C++11 > stack in 9.1, with the combination of libcxxrt, libc++, and clang++. > Unfortunately, the filter library configuration for libsupc++ left > some symbols in the wrong symbol version (the ABI library version > instead of the STL library version) and so there are some issues in > corner cases[1], which I am working on fixing. I have a fix for the > most common corner cases, but I want to make sure that I have caught > all of them before I commit. This will happen tomorrow. > > The filter library is very important to have working for 10.0, because > we will need the ports and compat versions of libstdc++ to use it if > we want to be able to mix code that uses libstdc++ (i.e. any ports > that don't work correctly with libc++) and libc++ (i.e. any ports that > use C++11, which is going to be an increasing number over the next few > years). > > David > > [1] In particular, terminate handlers are not correctly set, and > exceptions thrown from the runtime library are not of the expected > type. This means that C++ code that calls std::set_terminate() will > not work and neither will code that calls __cxa_bad_alloc() and > friends and then tries to catch the resulting exception. The only code > I have seen that actually does this is a test case that I wrote for > libcxxrt. In general, code encountering either of these problems is > already in a very broken state and the only difference you will see is > a less helpful error message before it crashes. I do not see how the code demonstrated as failing in standards/175453 could be classified as 'broken' or 'so special that nobody does it'. It is perfectly valid, and my reaction for such breakage is that C++ runtime is completely broken. How pointing to this fact can be FUD ? Your initial assesment of the problem as a misbehaviour of the combination of filtering and versioning made no sense to me, I asked you to provide the isolated test case, which you did not. Our implementation of filters indeed only allows for the filtered symbol to have the same version as the filtee symbol. I re-read the SUN documentation about filters (who had designed them), and looked at what Linux does there. It seems to me that Sun document spirit forces us to behave in a way which our rtld currently does. Linux behaviour is useless there, IMHO, since their rtld just inserts filtee before filter in the global lookup order (I may be wrong there). I do suggest (again) that the way to fix the issue is to provide the isolated test-case and decide if the behaviour of rtld is right. If we conclude that the problem is not in rtld but in the versioning mismatch between libraries (libstdc++ and libsupc++), I again suggest to provide the patch for review first. The ABI seems to become too unwieldly, and making ad-hoc changes there could paint us into the corner. pgpu2cXQjxZaw.pgp Description: PGP signature
Re: Removing default build of gcc
On Sat, Jan 26, 2013 at 02:53:16AM -0600, Mark Linimon wrote: > On Fri, Jan 25, 2013 at 03:36:15PM -0500, Pedro Giffuni wrote: > > I don't care much about gcc in current. > > clang, even on -9, can't build the following: > > - most of kde > - graphics/GraphicsMagick > - editors/emacs21 > - www/libxul19 > > any one of which I would consider showstoppers for making a gcc-less > system. IMO this is not the approach we should take in regard to gcc and ports. Ports should use port-provided compiler, and be untangled from the base toolchain. I believe that forcing ports committers to port 20K+ packages to clang is a waste of the FreeBSD resources and is is destined to fail despite the efforts. pgpTHTXzllHoH.pgp Description: PGP signature
Re: C++ runtime version patch for testing
On Sun, Jan 27, 2013 at 01:28:44PM +, David Chisnall wrote:
> Hi All,
>
> Here is a patch that, I believe, should fix the symbol version mismatches
> between the runtime and the STL implementation. I have run the exception
> tests from libcxxrt with this patch applied and:
>
> - libsupc++ & libstdc++
> - libcxxrt & libstdc++
> - libcxxrt & libc++
>
> All tests pass for me now. Please let me know if there are any problems with
> this, otherwise I'll aim to commit it today or tomorrow with a 1-week MFC.
>
> David
>
> Index: gnu/lib/libsupc++/Version.map
> ===
> --- gnu/lib/libsupc++/Version.map (revision 245840)
> +++ gnu/lib/libsupc++/Version.map (working copy)
> @@ -142,6 +142,28 @@
> _ZdaPvRKSt9nothrow_t;
> _ZdlPv;
> _ZdlPvRKSt9nothrow_t;
> +extern "C++" {
> + std::set_new_handler*;
What are the symbols you assigning the version there ? I cannot find
anything in the libstdc++.so export list which would match the line.
> + std::set_terminate*;
> + std::set_unexpected*;
> + std::bad_alloc*;
> +
> + std::bad_alloc*;
std::bad_alloc seems to be duplicated.
Besides that, pristine libstdc++.so exports 'std::bad_alloc::what() const'
at the GLIBCXX_3.4.9 namespace. You did this for the *::what()' from
libcxxrt but not for the libsupc++.
> + std::bad_cast*;
> + std::exception*;
> +
> + "typeinfo for std::bad_alloc";
> + "typeinfo for std::bad_cast";
> + "typeinfo for std::exception";
> +
> + "typeinfo name for std::bad_alloc";
> + "typeinfo name for std::bad_cast";
> + "typeinfo name for std::exception";
> +
> + "vtable for std::bad_alloc";
> + "vtable for std::bad_cast";
> + "vtable for std::exception";
> +};
> };
>
> CXXABI_1.3.1 {
> Index: lib/libcxxrt/Version.map
> ===
> --- lib/libcxxrt/Version.map (revision 245840)
> +++ lib/libcxxrt/Version.map (working copy)
> @@ -209,18 +209,7 @@
>
> "std::type_info::type_info(std::type_info const&)";
> "std::type_info::type_info(std::type_info const&)";
> -"std::type_info::~type_info()";
> -"std::type_info::~type_info()";
> -"std::type_info::~type_info()";
> "std::type_info::operator=(std::type_info const&)";
[omitted]
Do applications record the dependency on the libcxxrt directly,
using the DT_NEEDED tag ?
pgpll5DaeQpk0.pgp
Description: PGP signature
Re: C++ runtime version patch for testing
On Sun, Jan 27, 2013 at 03:17:51PM +, David Chisnall wrote:
> On 27 Jan 2013, at 15:03, Konstantin Belousov wrote:
>
> > On Sun, Jan 27, 2013 at 01:28:44PM +, David Chisnall wrote:
> >> + std::set_new_handler*;
> > What are the symbols you assigning the version there ? I cannot find
> > anything in the libstdc++.so export list which would match the line.
>
> std::set_new_handler(void (*)())
>
> # objdump -T /usr/lib/libsupc++.so | c++filt | grep new_h
> 9010 __float128DF .text 000e GLIBCXX_3.4
> std::set_new_handler(void (*)())
Apparently c++filt from 2.23.1 binutils has bug, c++filt is
not able to demangle the set_new_handler.
>
> >> + std::set_terminate*;
> >> + std::set_unexpected*;
> >> + std::bad_alloc*;
> >> +
> >> + std::bad_alloc*;
> > std::bad_alloc seems to be duplicated.
>
> Thanks, removed.
>
> > Besides that, pristine libstdc++.so exports 'std::bad_alloc::what() const'
> > at the GLIBCXX_3.4.9 namespace. You did this for the *::what()' from
> > libcxxrt but not for the libsupc++.
>
> Ooops. I wrote a script that checked for version mismatches, but for some
> reason I missed this one. Running it again, it shows two mismatches, both
> fixed in the new version of the diff.
>
You need to add 'std::bad_typeid::what() const' to 3.4.9 as well, it
seems.
>
> >> + std::bad_cast*;
> >> + std::exception*;
> >> +
> >> + "typeinfo for std::bad_alloc";
> >> + "typeinfo for std::bad_cast";
> >> + "typeinfo for std::exception";
> >> +
> >> + "typeinfo name for std::bad_alloc";
> >> + "typeinfo name for std::bad_cast";
> >> + "typeinfo name for std::exception";
> >> +
> >> + "vtable for std::bad_alloc";
> >> + "vtable for std::bad_cast";
> >> + "vtable for std::exception";
> >> +};
> >> };
> >>
> >> CXXABI_1.3.1 {
> >> Index: lib/libcxxrt/Version.map
> >> ===
> >> --- lib/libcxxrt/Version.map (revision 245840)
> >> +++ lib/libcxxrt/Version.map (working copy)
> >> @@ -209,18 +209,7 @@
> >>
> >> "std::type_info::type_info(std::type_info const&)";
> >> "std::type_info::type_info(std::type_info const&)";
> >> -"std::type_info::~type_info()";
> >> -"std::type_info::~type_info()";
> >> -"std::type_info::~type_info()";
> >> "std::type_info::operator=(std::type_info const&)";
> > [omitted]
> >
> > Do applications record the dependency on the libcxxrt directly,
> > using the DT_NEEDED tag ?
>
> I don't believe so, they get it indirectly via libc++ or libstdc++. How can
> I check?
>
Do readelf -d and look for the needed tags. If libcxxrt
is not passed on the linker command line and not recorded as needed
in the libc++/libstdc++, it should be fine.
Your changes to libcxxrt obviously break the ABI, removing the symbols
from the version namespace, but my hope is that namespace for libcxxrt
is actually not part of the _system_ ABI. Thus the question.
pgp98rtdFNHEi.pgp
Description: PGP signature
Re: C++ runtime version patch for testing
On Sun, Jan 27, 2013 at 04:01:48PM +, David Chisnall wrote: > On 27 Jan 2013, at 15:52, Konstantin Belousov wrote: > > > On Sun, Jan 27, 2013 at 03:17:51PM +, David Chisnall wrote: > > > > Apparently c++filt from 2.23.1 binutils has bug, c++filt is > > not able to demangle the set_new_handler. > > I'm using the one from head, which may be the elftoolchain project one? It > seems to be missing a man page... > In head, the c++ demangler is coming from either binutils or gcc copy of libiberty. I did not looked which one is selected. > > You need to add 'std::bad_typeid::what() const' to 3.4.9 as well, it > > seems. > > Added > > > Do readelf -d and look for the needed tags. If libcxxrt > > is not passed on the linker command line and not recorded as needed > > in the libc++/libstdc++, it should be fine. > > 0x0001 (NEEDED) Shared library: [libcxxrt.so.1] > > This is with something compiled with -stdlib=libc++. It seems the > (NEEDED) is there even if I compile with clang -lc++, instead of > clang++, so the linker is adding it via the indirect dependency? Or > does it show up because libc++ has that line too? > Until recently, the default behaviour of the ELF linker was to record all second order needed libraries (i.e. libraries needed by a library specified on the linker command line) in the final link result. This is controllable with --copy-dt-needed-entries linker flag and recently the default were flipped to --no-copy. > > Your changes to libcxxrt obviously break the ABI, removing the symbols > > from the version namespace, but my hope is that namespace for libcxxrt > > is actually not part of the _system_ ABI. Thus the question. > > I'm okay with breaking the libcxxrt ABI at this point. libc++ is > not part of the standard install, and is there for testing in 9.1. > libcxxrt isn't linked against anything unless you use libc++ (or > libstdc++ and a line in libmap.conf) so nothing non-experimental > should use it. For 9.2, I'd like to have an ABI that we can support > long term. > You might be okay with ABI breakage, but the project is not. Having ABI breakage on the stable branch is very unfortunate both from the technical and PR points of view. You should consider using .gnu.warning. section or some other mechanism to explicitely warn about using libcxxrt begin unsupported, or better, remove it from the stable branch. pgpnLbnIYumwN.pgp Description: PGP signature
Re: C++ runtime version patch for testing
On Sun, Jan 27, 2013 at 06:15:51PM +0200, Konstantin Belousov wrote: > On Sun, Jan 27, 2013 at 04:01:48PM +, David Chisnall wrote: > > On 27 Jan 2013, at 15:52, Konstantin Belousov wrote: > > > > > On Sun, Jan 27, 2013 at 03:17:51PM +, David Chisnall wrote: > > > > > > Apparently c++filt from 2.23.1 binutils has bug, c++filt is > > > not able to demangle the set_new_handler. > > > > I'm using the one from head, which may be the elftoolchain project one? It > > seems to be missing a man page... > > > In head, the c++ demangler is coming from either binutils or gcc copy > of libiberty. I did not looked which one is selected. > > > > You need to add 'std::bad_typeid::what() const' to 3.4.9 as well, it > > > seems. > > > > Added > > > > > Do readelf -d and look for the needed tags. If libcxxrt > > > is not passed on the linker command line and not recorded as needed > > > in the libc++/libstdc++, it should be fine. > > > > 0x0001 (NEEDED) Shared library: [libcxxrt.so.1] > > > > This is with something compiled with -stdlib=libc++. It seems the > > (NEEDED) is there even if I compile with clang -lc++, instead of > > clang++, so the linker is adding it via the indirect dependency? Or > > does it show up because libc++ has that line too? > > > Until recently, the default behaviour of the ELF linker was to record > all second order needed libraries (i.e. libraries needed by a library > specified on the linker command line) in the final link result. This is > controllable with --copy-dt-needed-entries linker flag and recently > the default were flipped to --no-copy. > > > > > Your changes to libcxxrt obviously break the ABI, removing the symbols > > > from the version namespace, but my hope is that namespace for libcxxrt > > > is actually not part of the _system_ ABI. Thus the question. > > > > I'm okay with breaking the libcxxrt ABI at this point. libc++ is > > not part of the standard install, and is there for testing in 9.1. > > libcxxrt isn't linked against anything unless you use libc++ (or > > libstdc++ and a line in libmap.conf) so nothing non-experimental > > should use it. For 9.2, I'd like to have an ABI that we can support > > long term. > > > You might be okay with ABI breakage, but the project is not. > Having ABI breakage on the stable branch is very unfortunate both > from the technical and PR points of view. You should consider using > .gnu.warning. section or some other mechanism to explicitely warn > about using libcxxrt begin unsupported, or better, remove it from > the stable branch. I noted one more thing to verify. Shouldn't the GLIBCXX_3.4.9 version inherit from the GLIBCXX_3.4, both for libsupc++ and libcxxrt ? I cannot verify the stock libstdc++ right now, but I suspect that namespace is inherited, and this must be fixed before the commit, if true. pgpKQhR7iEfFy.pgp Description: PGP signature
Re: clang++ 3.3 issue (excessively slow compile vs. gcc 4.6 in just one file of a port)
On Mon, Nov 18, 2013 at 11:54:00PM +0100, Matthias Andree wrote: > Glib shares the fate, because it defers to std:: containers where possible. > > A nice way would require additional work and get the linkers to complain > that symbols resolve with a different, incompatible ABI. That would, > however, require second-guessing other ABIs and namespaces, and would > hardly be portable -- or add ABI versions into the object files and .a > and .so libraries, unless we already have ABI tags somewhere down deep > in the ELF stuff. I never checked. The ABI there is an ABI of library, so linkers has nothing to do with such conflict resolution. Linker indeed could be tricked into stopping the linking if libstdc++ is linked into libc++-using binary. This can be hacked together with linker script for libc++ and use of ASSERT() and DEFINED() functions. But the right solution for the port troubles is to just switch to consistently use (newest) gcc for ports. I assume that Uses/compiler.mk finally allows to do this from the make.conf ? Anybody could share a working incantation to put there ? pgppOQSzEBOQo.pgp Description: PGP signature
Re: Intercepting calls in PIC mode
On Fri, Jul 04, 2014 at 04:12:51PM +0400, Ivan A. Kosarev wrote:
> Hello,
>
> Consider the following:
>
> ---
> #include
> #include
>
> extern "C" void* memset(void *block, int c, size_t size)
> __attribute__((weak, alias("__int_memset"), visibility("default")));
>
> extern "C" __attribute__((visibility("default")))
> void* __int_memset(void *block, int c, size_t size) {
> puts("Hello");
> return NULL;
> }
>
> int main()
> {
> void *(*F)(void *b, int c, size_t len) = memset;
> char a[5];
> memset(a, 0, sizeof(a));
> F(a, 0, sizeof(a));
> return 0;
> }
> ---
>
> It intercepts the memset() calls without issue on both x86-64 FreeBSD
> 9.2 and Linux. However, with the -fPIC option specified in the cc's
> command line, only the first (direct) call work on FreeBSD, but not the
> second (indirect) one. Note is that on Linux both the calls are
> intercepted--no matter whether the -fPIC option is specified or not.
Your example is rather convoluted, I will try to explain below why.
First, I am sure that C99 does not allow to override the semantic of the
standard-defined functions. That said, a call to memset(3) can be
inlined by a compiler, so there could be nothing to intercept.
Second, FreeBSD implementation of the weak ELF symbols is non-compliant.
The dynamic linker prioritizes non-weak symbols over the weak. This at
least explains why your code snippet does not segfaults: the memset(3)
from libc is not interposed by your memset() implementation, so libc can
at least initialize itself. If you remove weak attribute from the memset(),
debug version of libc fails with assertions in jemalloc, while normal
build just segfaults.
That said, there are also differences in the static linker behaviour.
Clang generates the following code to obtain the address of the memset(3)
function:
movqmemset@GOTPCREL(%rip), %rsi
The in-tree ld from binutils 2.17.redhat generates the
R_X86_64_GLOB_DAT relocation to fill the GOT entry for the memset
symbol. Processor of the GLOB_DAT in the rtld-elf always starts
the lookup of the requested symbol in the object next from main.
For your code, this means libc is searched for memset to fill the
slot, and you get a libc symbol.
The ld from the stock build of binutils 2.24, on the other hand, does not
generate a relocation at all, it resolves memset internally from the same
object file and fills the offset directly into instruction. I.e., when
the program is linked with new ld, it works as you intend. This is
probably the reason why it worked for you on Linux.
I am not sure what conclusion could be made from the story I just told you.
Might be, 'do not try to interpose std C functions' and 'put interposers
into the LD_PRELOADed objects' ?
pgpoVwcWj8Xbk.pgp
Description: PGP signature
llvm build error on ppc
FYI, I did make tinderbox today for HEAD at r274464, and got the following both for ppc and ppc64 worlds. Other arches build successfully. ===> lib/clang/libllvmtablegen (all) /scratch/tmp/kib/src/lib/clang/libllvmselectiondag/../../../contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp: In member function 'void llvm::SDDbgInfo::erase(const llvm::SDNode*)': /scratch/tmp/kib/src/lib/clang/libllvmselectiondag/../../../contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:632: error: a function-definition is not allowed here before ':' token /scratch/tmp/kib/src/lib/clang/libllvmselectiondag/../../../contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:634: error: could not convert '((llvm::DenseMapBase, llvm::DenseMapInfo >, const llvm::SDNode*, llvm::SmallVector, llvm::DenseMapInfo >*)(&((llvm::SDDbgInfo*)this)->llvm::SDDbgInfo::DbgValMap))->llvm::DenseMapBase::erase [with DerivedT = llvm::DenseMap, llvm::DenseMapInfo >, KeyT = const llvm::SDNode*, ValueT = llvm::SmallVector, KeyInfoT = llvm::DenseMapInfo](llvm::DenseMapIterator, llvm::DenseMapInfo, false>(((const llvm::DenseMapIterator, llvm::DenseMapInfo, false>&)((const llvm::DenseMapIterator, llvm::DenseMapInfo, false>*)(& I)' to 'bool' /scratch/tmp/kib/src/lib/clang/libllvmselectiondag/../../../contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:635: error: expected primary-expression before '}' token /scratch/tmp/kib/src/lib/clang/libllvmselectiondag/../../../contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:635: error: expected `)' before '}' token /scratch/tmp/kib/src/lib/clang/libllvmselectiondag/../../../contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:635: error: expected primary-expression before '}' token /scratch/tmp/kib/src/lib/clang/libllvmselectiondag/../../../contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:635: error: expected `;' before '}' token --- SelectionDAG.o --- *** [SelectionDAG.o] Error code 1 make[8]: stopped in /scratch/tmp/kib/src/lib/clang/libllvmselectiondag 1 error ___ freebsd-toolchain@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: llvm build error on ppc
On Thu, Nov 13, 2014 at 07:11:45PM +0100, Dimitry Andric wrote: > On 13 Nov 2014, at 18:13, Konstantin Belousov wrote:FYI, > > > > I did make tinderbox today for HEAD at r274464, and got the following > > both for ppc and ppc64 worlds. Other arches build successfully. > > > > ===> lib/clang/libllvmtablegen (all) > > /scratch/tmp/kib/src/lib/clang/libllvmselectiondag/../../../contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp: > > In member function 'void llvm::SDDbgInfo::erase(const llvm::SDNode*)': > > /scratch/tmp/kib/src/lib/clang/libllvmselectiondag/../../../contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:632: > > error: a function-definition is not allowed here before ':' token > > /scratch/tmp/kib/src/lib/clang/libllvmselectiondag/../../../contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:634: > > error: could not convert '((llvm::DenseMapBase > llvm::SDNode*, llvm::SmallVector, > > llvm::DenseMapInfo >, const llvm::SDNode*, > > llvm::SmallVector, llvm::DenseMapInfo > llvm::SDNode*> > > >*)(&((llvm::SDDbgInfo*)this)->llvm::SDDbgInfo::DbgValMap))->llvm::DenseMapBase > KeyT, ValueT, KeyInfoT>::erase [with DerivedT = llvm::DenseMap > llvm::SDNode*, llvm::SmallVector, > > llvm::DenseMapInfo >, KeyT = const llvm::SDNode*, > > ValueT = llvm::SmallVector, KeyInfoT = > > llvm::DenseMapInfo](llvm::DenseMapIterator > llvm::SDNode*, llvm::SmallVector, > > llvm::DenseMapInfo, false>(((const > > llvm::DenseMapIterator > lue*, 2u>, llvm::DenseMapInfo, false>&)((const > > llvm::DenseMapIterator > llvm::SmallVector, llvm::DenseMapInfo > llvm::SDNode*>, false>*)(& I)' to 'bool' > > /scratch/tmp/kib/src/lib/clang/libllvmselectiondag/../../../contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:635: > > error: expected primary-expression before '}' token > > /scratch/tmp/kib/src/lib/clang/libllvmselectiondag/../../../contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:635: > > error: expected `)' before '}' token > > /scratch/tmp/kib/src/lib/clang/libllvmselectiondag/../../../contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:635: > > error: expected primary-expression before '}' token > > /scratch/tmp/kib/src/lib/clang/libllvmselectiondag/../../../contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:635: > > error: expected `;' before '}' token > > --- SelectionDAG.o --- > > *** [SelectionDAG.o] Error code 1 > > Hi Kostik, > > Sorry about this. I imported an upstream fix for llvm, but since they > switched to C++11, it introduces some constructs that g++ 4.2.1 does not > support. Clang compiles these even if -std=c++11 is not specified on > the command line, which is why I didn't notice the problem. Is ppc the only architecture where we do build clang, but with gcc ? Since the failure only occured on ppc/pcc64, and not on e.g. arm. ___ freebsd-toolchain@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: clang confuses kgdb on static symbols
On Tue, Oct 20, 2015 at 03:44:13PM -0400, Ed Maste wrote:
> On 20 October 2015 at 13:27, John Baldwin wrote:
> >
> > If '-gdwarf-4' works then you can just set that in the DEBUG makeoptions as
> > a
> > test, otherwise try hacking kern.mk to disable this bit:
> >
> > #
> > # Add -gdwarf-2 when compiling -g. The default starting in clang v3.4
> > # and gcc 4.8 is to generate DWARF version 4. However, our tools don't
> > # cope well with DWARF 4, so force it to genereate DWARF2, which they
> > # understand. Do this unconditionally as it is harmless when not needed,
> > # but critical for these newer versions.
> > #
> > .if ${CFLAGS:M-g} != "" && ${CFLAGS:M-gdwarf*} == ""
> > CFLAGS+=-gdwarf-2
> > .endif
>
> Note that Clang defaults to DWARF 2 on FreeBSD, so removing the
> override in kern.mk isn't sufficient.
>
> >From contrib/llvm/tools/clang/lib/Driver/Tools.cpp:
> else if (!A->getOption().matches(options::OPT_g0) &&
> !A->getOption().matches(options::OPT_ggdb0)) {
> // Default is dwarf-2 for Darwin, OpenBSD, FreeBSD and Solaris.
> const llvm::Triple &Triple = getToolChain().getTriple();
> if (Triple.isOSDarwin() || Triple.getOS() == llvm::Triple::OpenBSD ||
> Triple.getOS() == llvm::Triple::FreeBSD ||
> Triple.getOS() == llvm::Triple::Solaris)
> CmdArgs.push_back("-gdwarf-2");
> else
> CmdArgs.push_back("-g");
> }
>
> It may be time for us to remove this default from Clang.
I doubt that names of the local symbols have anything to do with the dwarf
version.
We cannot stop generating dwarf-2 until in tree gdb and kgdb are substituted
by a working replacement.
___
freebsd-toolchain@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: clang confuses kgdb on static symbols
On Tue, Oct 20, 2015 at 04:07:35PM -0400, Ed Maste wrote: > On 20 October 2015 at 15:55, Konstantin Belousov wrote: > > > > We cannot stop generating dwarf-2 until in tree gdb and kgdb are substituted > > by a working replacement. > > Note that I'm only suggesting removing the baked-in default from > Clang, not the setting in kern.mk. Wouldn't this cause another outburst of 'works by default' discussion from some other place ? ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: powerpc64 11.0-CURRENT's clang binds -m32 -mcpu=powerpc a.out to /libexec/ld-elf.so.1
On Mon, Dec 14, 2015 at 11:06:51PM -0800, Mark Millard wrote:
> # more main.c
> int main()
> {
> return 0;
> }
>
>
>
> # ls -l `which cc`
> -r-xr-xr-x 7 root wheel 54137976 Dec 14 00:06 /usr/bin/cc
>
> # cc --version
> FreeBSD clang version 3.7.0 (tags/RELEASE_370/final 246257) 20150906
> Target: powerpc64-unknown-freebsd11.0
> Thread model: posix
>
> # cc -m32 -mcpu=powerpc main.c
> # file a.out
> a.out: ELF 32-bit MSB executable, PowerPC or cisco 4500, version 1 (FreeBSD),
> dynamically linked, interpreter /libexec/ld-elf.so.1, FreeBSD-style, for
> FreeBSD 11.0 (1100091), not stripped
>
>
>
> By contrast powerpc64-gcc binds the a.out produced to /libexec/ld-elf32.so.1
> instead:
>
> # ls -l `which gcc`
> lrwxr-xr-x 1 root wheel 48 Dec 5 05:38 /usr/bin/gcc ->
> /usr/local/bin/powerpc64-portbld-freebsd11.0-gcc
>
> # gcc --version
> gcc (FreeBSD Ports Collection for powerpc64) 5.2.0
> Copyright (C) 2015 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions. There is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
>
> # gcc -m32 -mcpu=powerpc main.c
> # file a.out
> a.out: ELF 32-bit MSB executable, PowerPC or cisco 4500, version 1 (FreeBSD),
> dynamically linked, interpreter /libexec/ld-elf32.so.1, FreeBSD-style, for
> FreeBSD 11.0 (1100091), not stripped
>
This is a bug in gcc, most likely in the spec file. All FreeBSD
ABIs use either /libexec/ld-elf.so.1 or (for older versions)
/usr/libexec/ld-elf.so.1.
>
> Context details:
>
> # freebsd-version -ku; uname -aKU
> 11.0-CURRENT
> 11.0-CURRENT
> FreeBSD FBSDG5C0 11.0-CURRENT FreeBSD 11.0-CURRENT #1 r291891M: Wed Dec 9
> 09:15:33 PST 2015
> root@FBSDG5C0:/usr/obj/xtoolchain/powerpc.powerpc64/usr/src/sys/GENERIC64vtsc-NODEBUG
> powerpc 1100091 1100091
>
> # pkg info powerpc64-gcc
> powerpc64-gcc-5.2.0_1
> Name : powerpc64-gcc
> Version: 5.2.0_1
> Installed on : Wed Dec 9 02:18:14 2015 PST
> Origin : devel/powerpc64-gcc
> Architecture : freebsd:11:powerpc:64
> Prefix : /usr/local
> Categories : devel
> . . .
>
> ===
> Mark Millard
> markmi at dsl-only.net
>
> ___
> freebsd-toolchain@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
> To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
___
freebsd-toolchain@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: 11.0-CURRENT (r292413) on a rpi2b: arm-gnueabi-freebsd/bin/ar, _fseeko, and memset vs memory alignment (SCTRL bit[1]=1?): Explains the Bus error?
On Mon, Dec 28, 2015 at 02:07:29PM +, Andrew Turner wrote: > There was some concern about reading non-naturally aligned data in the > kernel not being atomic, this could be checked by, in debug > configurations, enabling alignment checks on entry to the kernel and > disabling them on exit. We could also have a flag for clang to tell it > we are in the kernel, there is already something similar for iOS. What do you mean ? Non-aligned reads are non-atomic on all known arches, including x86. In the Core2 time this non-atomicity became easily observable due to the microacrhitecture changes, which happens simultaneously with a mis-aligned bug in DPCPU code. I do not think we ever depend on the atomicity of the unaligned reads in MI and in x86 MD code since then, why would arm differ there ? ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: 3 quick questions about stack alignment for powerpc (32-bit) signal handlers [the change that caused misaligned]
On Tue, Feb 02, 2016 at 10:05:16AM -0600, Justin Hibbits wrote: > Good catch! I'll commit the change tonight. I looked once at the powerpc sigsend(), and I think that it has an issue. The usfp is calculated by taking the stack pointer at the time of signal delivery and substracting the sigframe size. This means that a transient misalignment during some code (e.g. leaf function) is transferred to the signal handler execution. Other arches explicitely realign stack pointer for the signal frame before the frame is formed. I am not sure if the problem reported in the thread is caused by this or not, but forced realignment in sendsig() is required for ABI compliance. > > - Justin > > On Tue, Feb 2, 2016 at 3:48 AM, Mark Millard wrote: > > I tried the change to -32 and 32 (from -20 and 20) on/for the powerpc > > (32-bit) PowerMac that I use and the results were: > > > > A) "info frame" in gdb shows signal handlers are now started with 16-byte > > aligned stack frames. (Applies to gcc 4.2.1 based contexts too, not just to > > the clang 3.8.0 ones with the __vfprintf-tied segmentation faults during > > signals.) > > > > and. . . > > > > B) The "clang 3.8.0 compiled __vfprintf" segmentation faults in libc/stdio > > library code during signal handlers that use such code no longer happen > > because the alignment matches the code requirements. > > > > I've added this information to Bug 206810. > > > > > > (Note: There are a couple of segmentation fault contexts that I've never > > tied down to any specific property: no discovered evidence of signal > > handler involvement or of __vfprintf involvement, for example. These are > > still a problem. But where I had tied the faults to signal handlers using > > __vfprintf now instead work fine in my experimental clang 3.8.0 based > > builds.) > > > > > > === > > Mark Millard > > markmi at dsl-only.net > > > > On 2016-Feb-1, at 12:11 AM, Mark Millard wrote: > > > > The -16/16 code below produced correct alignment but too little space. > > > > The -20/20 code below produces enough space but misalignment. > > > > To maintain 16-byte alignment while increasing the space would have > > required going from -16/16 to -32/32. At least that is how I understand > > this code. > > > > > >> Index: sys/powerpc/powerpc/sigcode32.S > >> === > >> --- sys/powerpc/powerpc/sigcode32.S > >> (.../head/sys/powerpc/powerpc/sigcode32.S) (revision 209975) > >> +++ sys/powerpc/powerpc/sigcode32.S > >> (.../projects/clang380-import/sys/powerpc/powerpc/sigcode32.S) (working > >> copy) > >> @@ -45,9 +45,9 @@ > >> */ > >>.globl CNAME(sigcode32),CNAME(szsigcode32) > >> CNAME(sigcode32): > >> - addi1,1,-16 /* reserved space for callee */ > >> + addi1,1,-20 /* reserved space for callee */ > >>blrl > >> - addi3,1,16+SF_UC/* restore sp, and get > >> &frame->sf_uc */ > >> + addi3,1,20+SF_UC/* restore sp, and get > >> &frame->sf_uc */ > >>li 0,SYS_sigreturn > >>sc /* sigreturn(scp) */ > >>li 0,SYS_exit > > > > > > > > The "working copy" is -r266778 from 2014-May-27. > > > > -r209975 is from 2010-Jul-13. > > > > > > === > > Mark Millard > > markmi at dsl-only.net > > > > On 2016-Jan-31, at 10:58 PM, Mark Millard wrote: > > > > Just a correction to a sentence that I wrote. I had written: > > > >> Frame at:0x...90 vs. 0x...1c > >> call by frame: 0x...b0 vs. 0x...1c > >> Arglist at: 0x...70 vs. 0x...dc > >> Locals at: 0x...70 vs. 0x...dc > >> Previous frame's sp: 0x...90 vs. 0x...1c > >> > >> It looks like 4 additional pad bytes on the user/process stack are needed > >> to get back to alignment. > > > > Of course the figures on the right need to get smaller, not larger: The > > stack grows towards smaller addresses. So to get to 0x...0 on the right I > > should have said: > > > > It looks like 12 additional pad bytes on the user/process stack are needed > > to get back to alignment. > > > > That would produce: > > > > Frame at:0x...90 vs. 0x...10 > > call by frame: 0x...b0 vs. 0x...10 > > Arglist at: 0x...70 vs. 0x...d0 > > Locals at: 0x...70 vs. 0x...d0 > > Previous frame's sp: 0x...90 vs. 0x...10 > > > > === > > Mark Millard > > markmi at dsl-only.net > > > > On 2016-Jan-31, at 10:47 PM, Mark Millard wrote: > > > > More evidence: By adding "break raise" and then using "info frame" to show > > the alignment at that point I can show that the later signal delivery > > changes the alignment on the user process stack compared to when raise was > > called. (Later I show the same for thr_kill.) > > > >> Breakpoint 2, __raise (s=29) at /usr/src/lib/libc/gen/raise.c:50 > >> warning: Source file is more recent than executable. > >> 50if (__sys_thr_self(
Re: clang -pg, libm and the _end symbol
On Wed, Feb 24, 2016 at 12:27:03PM +0100, Raphael Kubo da Costa wrote:
> I'm reviewing an update to the textproc/miller port in bug 207194, and
> noticed it does some ugly things in post-configure to seemingly
> work around the following problem (on 11-HEAD at least):
>
> % echo 'int main(void) { return 0; }' > foo.c
> % clang -pg foo.c -lm
> /usr/bin/ld: undefined reference to symbol `_end' (try adding -lc)
> //lib/libc.so.7: could not read symbols: Bad value
> cc: error: linker command failed with exit code 1 (use -v to see
> invocation)
>
> (FWIW, using another library such as -lz instead of -lm retuls in the
> same problem)
>
> Adding LDFLAGS+=-lc to the port's Makefile would've been enough, but I'm
> not sure if it'd be just working around an actual bug, plus libtool
> automatically strips -lc from the linker invocation:
>
> 7534 *-*-openbsd* | *-*-freebsd* | *-*-dragonfly* | *-*-bitrig*)
> 7535 # Do not include libc due to us having libc/libc_r.
> 7536 test X-lc = "X$arg" && continue
>
> The port builds and links fine on 9.3 without any workarounds, and if I
> explicitly use ld.gold to link the above file it also works on HEAD.
>
> Is clang working as expected or is this a bug?
This is probably not a clang bug, but could be. More likely it is ld problem.
I do not want to dig into the issue, but I can provide you some points to
look at further.
The _end symbol is magic, it is defined as the address of the last byte
+ 1 of zero-initialized data section (.bss). But the symbol is not
provided by any object file participating in the linkage of the binary,
instead it is created by the linker after all sections are combined in
the segments and segments are laid out.
The symbol is creation requested by the linker script, look at the
/usr/libdata/ldscripts for them, first line of the file contains
comment explaining which final binary format is served by the each
script.
We are aware that binutils 2.25.1 ld for aarch64 has bug where _end is
not exported from executable, I was not able to track the bug.
To diagnose your issue, look up which linker script is used for -pg
linking, look for the _end symbol there. If it is properly requested,
then the bug is in base linker.
___
freebsd-toolchain@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: clang -pg, libm and the _end symbol
On Wed, Feb 24, 2016 at 01:54:25PM +0100, Dimitry Andric wrote:
> On 24 Feb 2016, at 12:27, Raphael Kubo da Costa wrote:
> >
> > I'm reviewing an update to the textproc/miller port in bug 207194, and
> > noticed it does some ugly things in post-configure to seemingly
> > work around the following problem (on 11-HEAD at least):
> >
> > % echo 'int main(void) { return 0; }' > foo.c
> > % clang -pg foo.c -lm
> > /usr/bin/ld: undefined reference to symbol `_end' (try adding -lc)
> > //lib/libc.so.7: could not read symbols: Bad value
> > cc: error: linker command failed with exit code 1 (use -v to see
> > invocation)
>
> Try using: clang -pg foo.c -lm_p
>
> That works for me (also with gcc).
It probably not quite works, in the sense that it resolves _end to
something not correctly provided by libm_p.a. In other words, sbrk(),
if used for anything, would be broken.
___
freebsd-toolchain@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: warning: DWARF2 only supports one section per compilation unit
On Tue, Mar 29, 2016 at 07:49:14PM +0200, Dimitry Andric wrote: > These are harmless, and have been showing up for at least a year now. > Clang is just notifying you that due to DWARF2 limitations, it will not > emit debug info for more than one code section. > > All such warnings in our tree are about artificially added sections, > such as .note.GNU-stack, .reloc and __xen_guest, and it is not likely > that anybody cares that those sections will not have debug info. > > Lastly, I never found an option to shut up the warnings. Well, apart > from using -gdwarf-4. :-) .note.GNU-stack should not have any content at all. Issuing the warning for empty section is pointless, and it would hide in the noise a real bug if anything migrates under the section. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: Problems with out libgcc_s.so in base
On Fri, Aug 19, 2016 at 01:14:32AM +0200, Tijl Coosemans wrote: > On Thu, 18 Aug 2016 14:48:28 +0200 Dimitry Andric wrote: > > On 18 Aug 2016, at 11:15, Tijl Coosemans wrote: > >> On Wed, 17 Aug 2016 14:17:10 -0700 Steve Kargl > >> wrote: > >>> % gfortran6 -o z foo.f90 && ./z > >>> /lib/libgcc_s.so.1: version GCC_4.6.0 required by \ > >>> /usr/local/lib/gcc6/libgfortran.so.3 not found > >>> % ldconfig -r | grep libgcc > >>>6:-lgcc_s.1 => /lib/libgcc_s.so.1 > >>>735:-lgcc_s.1 => /usr/local/lib/gcc6/libgcc_s.so.1 > >>> > >>> Clearly, ldd is looking for 735 but finds 6. If the lang/gcc6 could > >>> be convinced to build, install, and use libgcc_s6.so.1, then the > >>> problem is solved without a wrapper. > >> > >> In this case the real cause of the problem is that compilers and linkers > >> search /lib and /usr/lib last and ldconfig searches them first. Renaming > >> the library is just a hack around that. > > > > Well, even if you would adjust the compilers and linkers to look in > > /usr/local/lib first, > > No, I wanted to change /etc/rc.d/ldconfig to put /lib and /usr/lib last. > That would match base ld(1) so anything that links successfully at > compile-time will also link successfully at run-time (if there are no > other search order mismatches leading to conflicts). > > But, this means that in case of a name conflict between base and ports, > the ports provided library is assumed to be the right one. I'm not 100% > sure this is smart. Usually the ports version of a library is more > recent and if the name is the same it should be backward compatible, but > if that's not the case (older or not compatible) base utilities may fail > to run (like ./z in the example above) and that's maybe worse than ports > or locally built programs failing. > > > how would you solve the problem of having > > multiple, possibly incompatible versions of the same library in > > different directories? > > > > For example, on one of my systems, I now have these: > > > > /usr/local/lib/gcc47/libgcc_s.so.1 > > /usr/local/lib/gcc48/libgcc_s.so.1 > > /usr/local/lib/gcc49/libgcc_s.so.1 > > /usr/local/lib/gcc5/libgcc_s.so.1 > > /usr/local/lib/gcc6/libgcc_s.so.1 > > /usr/local/lib/gcc7/libgcc_s.so.1 > > > > So which one are you going to put at the front of the path? The gcc7 > > version? If you are lucky that one is backwards compatible with all the > > previous ones, but still I would like it much better if a program > > compiled by, say, gcc5 was linked *explicitly* against the gcc5 version > > of libgcc_s.so. > > > > Steve's proposed scheme solves that quite nicely, in my opinion. The > > problem is only in the details, as usual. There will be many configure > > scripts and libtool-like utilities out there, that assume libgcc must be > > linked using -lgcc_s, not -lgcc_s$VERSION. > > This is a separate problem that has been discussed many times before. > The ports tree adds -Wl,-rpath to *FLAGS in several places to choose > a library. I now noticed there is a FAQ about this at > https://gcc.gnu.org/faq.html#rpath. It gives some suggestions including > creating wrapper scripts, but they wouldn't work when code is compiled > with gfortran but linked with Clang cc/c++. The only thing that works > in this case is -Wl,-rpath. Another option would be to create a port > that installs a recent version of libgcc in /usr/local/lib and let the > gcc ports use that instead of their own copy. The option which would fix all this mess is: 1. add rpath for gcc lib/ directory into spec file and 2. make ports collection use its own compiler instead of fighting with the base. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: Problems with out libgcc_s.so in base
On Fri, Aug 19, 2016 at 09:50:28PM +0200, Tijl Coosemans wrote:
> On Fri, 19 Aug 2016 10:28:14 +0300 Konstantin Belousov
> wrote:
> > The option which would fix all this mess is:
> > 1. add rpath for gcc lib/ directory into spec file
> > and
> > 2. make ports collection use its own compiler instead of fighting with
> >the base.
>
> That still doesn't cover all cases, e.g.:
>
> port exec -> base lib -> libgcc_s.so.1
> -> port lib -> recent libgcc_s.so.1
>
> An example is:
>
> % echo 'int main(void){}' > test.c
> % clang37 -o test test.c -lexecinfo -lgfortran -L/usr/local/lib/gcc5
> -Wl,-rpath,/usr/local/lib/gcc5
> % ./test
> /lib/libgcc_s.so.1: version GCC_4.6.0 required by
> /usr/local/lib/gcc5/libgfortran.so.3 not found
>
> The base library (libexecinfo) doesn't have DT_RPATH or DT_RUNPATH so the
> only way rtld can find the right libgcc_s.so is if the executable (test)
> has DT_RPATH and no DT_RUNPATH. Clang runs ld with --enable-new-dtags
> so the executable has DT_RUNPATH. DT_RPATH is deprecated in the Linux
> world so there are probably more ports that use --enable-new-dtags.
> Another example seems to be Rust.
Indeed, and I rechecked the find_library() code against the latest gABI
once more.
So base libraries linked against libgcc_s are
/lib/libcxxrt.so.1
/usr/lib/libc++.so.1
/usr/lib/libexecinfo.so.1
/usr/lib/libprivatedevdctl.so.0
and only libexecinfo would be somewhat problematic. libcxxrt and libc++
come from C++ runtime, which must be provided by the port's compiler,
the private library should not be linked to by behaving applications.
I suppose that libexecinfo pull unwinder from libgcc_s. It might be
possible just dlopen() libgcc_s for _Unwind_Backtrace, it seems.
___
freebsd-toolchain@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: Time to enable partial relro
On Thu, Aug 25, 2016 at 05:50:31PM -0500, Pedro Giffuni wrote: > Hello; > > GNU RELRO support was committed in r230784 (2012-01-30) but we never > enabled it by default. > > There was some discussion about it on > https://reviews.freebsd.org/D3001 > > By now, all Linux distributions, NetBSD and DragonFly support it and > it is the default for most systems in binutils 2.27. > > This doesn't affect performance, I ran it through an exp-run last > year, no other OS has had issues etc ... seems safe and can be > disabled if needed when linking. Exp-run does not test anything interesting about relro. If all testing that was done is basically just an exp-run, then there was no useful runtime testing done. > > I think it's time to enable it be default in our base binutils. If > there are no objections, I will just commit the attached patch over > the weekend. There are objections, the change must be runtime tested on large and representative set of real-world applications before turning the knob. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: Time to enable partial relro
On Fri, Aug 26, 2016 at 10:00:58AM -0500, Pedro Giffuni wrote: > > > On 08/26/16 05:56, Konstantin Belousov wrote: > > On Thu, Aug 25, 2016 at 05:50:31PM -0500, Pedro Giffuni wrote: > >> Hello; > >> > >> GNU RELRO support was committed in r230784 (2012-01-30) but we never > >> enabled it by default. > >> > >> There was some discussion about it on > >> https://reviews.freebsd.org/D3001 > >> > >> By now, all Linux distributions, NetBSD and DragonFly support it and > >> it is the default for most systems in binutils 2.27. > >> > >> This doesn't affect performance, I ran it through an exp-run last > >> year, no other OS has had issues etc ... seems safe and can be > >> disabled if needed when linking. > > Exp-run does not test anything interesting about relro. If all testing > > that was done is basically just an exp-run, then there was no useful > > runtime testing done. > > > > The exp-run does cover Java and other VM-type thingies that bootstrap. > For upstream binutils this is now the default (at least for linux, > they never ask us if we want to follow). So the change has been tested > extensively but perhaps not on cases that are relevant to us. > > Note that the "fix" for any port is ultimately trivial: > LDFLAGS+= "-z norelro" > > >> > >> I think it's time to enable it be default in our base binutils. If > >> there are no objections, I will just commit the attached patch over > >> the weekend. > > > > There are objections, the change must be runtime tested on large and > > representative set of real-world applications before turning the knob. > > > > You are not giving any hint on what would be a "representative set of > real-world applications". Given that you committed the initial support > your objection stands very high and is a blocker. :( Well, I indeed do not think that 'works on my desktop' is good enough testing for this change. I understand that it is impossible to test all ports at runtime, or even a significant percentage of them, not to mention programs which sit on the user's storage. What actually worries me in the fate of such changes is that submitter never starts monitoring user complains and see whether the change causes some delayed random breakage. It just ends up in the 'BSD is broken' basket with one more supportive case. > > As I see it committing it now would give ample time to test this in > current before it hits any release. If you want more extensive testing > merging it in -stable right after the 11-Release is guaranteed to help > weed out any remaining update ports may need. It is useless 'testing' when the cause of breakage is not actively looked for and identified. You commit it now, five months later some unlucky user reports that his application coredumps. What next ? ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: Time to enable partial relro
On Sat, Aug 27, 2016 at 11:06:54AM -0500, Pedro Giffuni wrote:
>
>
> On 08/26/16 20:10, Pedro Giffuni wrote:
> >
> >
> ...>> I think we should move forward, just want to make sure it doesn???t
> >> break some arch completely before moving ahead. While lld is a goal,
> >> the goal is also to have a ld.bdf installed for 12, iirc, as a fallback.
> >
> > And very right you are, this has all the chances of breaking MIPS*:
> >
> > "A configure option --enable-relro={yes|no} to decide
> > whether -z relro should be the default behaviour for
> > the linker in ELF based targets. If this configure
> > option is not specified then relro will be enabled
> > automatically for all Linux based targets except FRV,
> > HPPA, IA64 and MIPS."
> >
> > _
> >
> > I will update the patch to exclude MIPS (and MIPS64 JIC).
> >
> > Pedro.
> >
> > *https://gcc.gnu.org/ml/gcc/2016-08/msg00134.html
> >
>
> Looking more into this, and the arm report from Mark Millard (thanks!),
> binutils has tests for RELRO in their testsuite that would be an
> important indicator before enabling the option.
>
> It surprises me that we don't have an easy way to run those checks from
> the port, so I borrowed the regression-test mode from GCC and I am
> attaching it.
>
> The tests may depend on some gnu-isms but we don't appear to do too
> well on the tests:
>
> === ld Summary ===
>
> # of expected passes511
> # of unexpected failures78
> # of expected failures 4
> # of unresolved testcases 35
> # of untested testcases 1
> # of unsupported tests 9
> /usr/ports/devel/binutils/work/binutils-2.27/ld/ld-new 2.27
And ? In which way this data is useful or indicative of anything ?
Why this tests are relevant to the proposed change ? AFAIK, binutils
tests typically compare ld output against expected binary.
And, number of the unexpected failures in your showcase is quite worrying.
___
freebsd-toolchain@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: How to turn off SSP stack-protector on 11.0S
On Tue, Nov 29, 2016 at 12:32:28PM +1100, Dewayne Geraghty wrote: > Is WITHOUT_SSP actually honoured and is building a world and/or ports > without SSP possible? Advise/suggestions appreciated. > > Amongst the 9 different server configurations that we build/support, we've > been asked to build a machine dedicated to PROLOG use. (yes really). > > As such we're trying to turn off everything that isn't needed for this > particular server. For those concerned with security, it is an air-gap > machine receiving data via usb. > > We've built/installed 11.0S from source. Now we're building the custom > server. However, even with WITHOUT_SSP= in both /etc/make.conf and > /etc/src.conf, we come up against little issues like: > "can not find /usr/lib/libssp_nonshared.a" So, does your host have /usr/lib/libssp_nonshared.a ? How did you installed 11.0, and what does designator 11.0S above mean ? Easy way out is to claim that r307146 should help you, but I suspect that there is something more broken in your configuration or build/install method. > > An example: > Stage 2.3: build tools > ===> bin/csh (obj,build-tools) > grep 'ERR_' /usr/src/bin/csh/../../contrib/tcsh/sh.err.c | grep '^#define' > >> sh.err.h > cc -E -O2 -pipe -g0 -ggdb0 -DSTRIP_FBSDID -UDEBUGGING -UDEBUG > -DUSB_HAVE_DISABLE_ENUM -I. -I/usr/src/bin/csh > -I/usr/src/bin/csh/../../contrib/tcsh -D_PATH_TCSHELL='"/bin/csh"' -g > -std=gnu99 -Qunused-arguments > -I/usr/obj/prod/110001/D/K8/usr/src/tmp/legacy/usr/include > /usr/src/bin/csh/../../contrib/tcsh/tc.const.c > /usr/src/bin/csh/../../contrib/tcsh/sh.char.h /usr/src/bin/csh/config.h > /usr/src/bin/csh/../../contrib/tcsh/config_f.h > /usr/src/bin/csh/../../contrib/tcsh/sh.types.h sh.err.h -D_h_tc_const | > grep 'Char STR' | sed -e 's/Char \([a-zA-Z0-9_]*\)\(.*\)/extern Char > \1[];/' | sort >> tc.const.h > cc -o gethost -L/usr/obj/prod/110001/D/K8/usr/src/tmp/legacy/usr/lib -O2 > -pipe -g0 -ggdb0 -DSTRIP_FBSDID -UDEBUGGING -UDEBUG > -DUSB_HAVE_DISABLE_ENUM -I. -I/usr/src/bin/csh > -I/usr/src/bin/csh/../../contrib/tcsh -D_PATH_TCSHELL='"/bin/csh"' -g > -std=gnu99 -Qunused-arguments > -I/usr/obj/prod/110001/D/K8/usr/src/tmp/legacy/usr/include > /usr/src/bin/csh/../../contrib/tcsh/gethost.c > /usr/bin/ld: cannot find /usr/lib/libssp_nonshared.a > cc: error: linker command failed with exit code 1 (use -v to see invocation) > *** [gethost] Error code 1 > > Note the > /usr/bin/ld: cannot find /usr/lib/libssp_nonshared.a > > It seems that the linker is trying to use the above library during the > build of all static images/executables. P.S. Toolchain@ is the place where you more likely to get a useful feedback. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: Function attribute for optimization level
I think that toolchain@ is more suitable list for the discussion. On Sun, Jun 04, 2017 at 05:44:31PM -0500, Eric van Gyzen wrote: > _thr_rtld_init() calls memcpy() for the sole purpose of resolving its > PLT entry. With clang 4.0 and the current code, compiler optimization > defeats this attempt by completely eliding the call. Other compilers > or code might emit inline instructions instead of the library call, > also defeating the purpose. After looking more closely at the whole situation, I have a question that we probably must answer first. Is clang -ffreestanding mode broken ? memcpy(3) is not included into the set of the environment features required for a C11 freestanding implementation, and clang pretending that it knows the semantic of the call sounds broken. > > I propose adding "__no_optimization" to sys/cdefs.h. The patch is > below. Empirical testing shows that clang 3.7 and later support > "optnone", and gcc 4.6 and later support "optimize()". Clang 3.4 does > not support either, so it takes the define-to-empty case. I did not > test clang 3.5 or 3.6. Where this attribute should be applied ? To the _thr_rtld_init() function ? > > Side note: GCC 4.6 with optimize(0) on amd64 emits two movq > instructions for memset(x,0,16), but GCC 5 emits a call to memset(). > > I have done no research to see if other popular codebases have such a > definition. If you know of one, please tell me; I would gladly adopt > an already common name for this proposal, for the sake of portability. > > Thanks in advance for your feedback. > > Eric > > > diff --git a/sys/sys/cdefs.h b/sys/sys/cdefs.h index > 9cdc03c861cb..e370f6d6459e 100644 --- a/sys/sys/cdefs.h +++ > b/sys/sys/cdefs.h @@ -396,6 +396,14 @@ #define __unreachable() > ((void)0) #endif > > +#if __has_attribute(optnone) +#define __no_optimization > __attribute__((optnone)) +#elif __has_attribute(optimize) +#define > __no_optimization __attribute__((optimize(0))) +#else +#define > __no_optimization +#endif + /* XXX: should use `#if __STDC_VERSION__ < > 199901'. */ #if !__GNUC_PREREQ__(2, 7) && !defined(__INTEL_COMPILER) > #define __func__ NULL ___ > freebsd-hack...@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscr...@freebsd.org" ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: Function attribute for optimization level
On Mon, Jun 05, 2017 at 01:03:24PM +0300, Konstantin Belousov wrote: > I think that toolchain@ is more suitable list for the discussion. > > On Sun, Jun 04, 2017 at 05:44:31PM -0500, Eric van Gyzen wrote: > > _thr_rtld_init() calls memcpy() for the sole purpose of resolving its > > PLT entry. With clang 4.0 and the current code, compiler optimization > > defeats this attempt by completely eliding the call. Other compilers > > or code might emit inline instructions instead of the library call, > > also defeating the purpose. > After looking more closely at the whole situation, I have a question > that we probably must answer first. Is clang -ffreestanding mode > broken ? memcpy(3) is not included into the set of the environment > features required for a C11 freestanding implementation, and clang > pretending that it knows the semantic of the call sounds broken. Ok, I realized that I only added -ffreestanding to the rtld Makefile. So clang is optimizing correctly there. Should we compile both libc and libthr in the freestanding environment as well ? I am sure that there are a lot of similar assumptions that libc and libthr code calls into itself and not into the arbitrary re-implementation of the same code as generated by modern compilers. Then hopefully the __no_optimization hack is not needed. > > > > > I propose adding "__no_optimization" to sys/cdefs.h. The patch is > > below. Empirical testing shows that clang 3.7 and later support > > "optnone", and gcc 4.6 and later support "optimize()". Clang 3.4 does > > not support either, so it takes the define-to-empty case. I did not > > test clang 3.5 or 3.6. > Where this attribute should be applied ? To the _thr_rtld_init() function ? > > > > > Side note: GCC 4.6 with optimize(0) on amd64 emits two movq > > instructions for memset(x,0,16), but GCC 5 emits a call to memset(). > > > > I have done no research to see if other popular codebases have such a > > definition. If you know of one, please tell me; I would gladly adopt > > an already common name for this proposal, for the sake of portability. > > > > Thanks in advance for your feedback. > > > > Eric > > > > > > diff --git a/sys/sys/cdefs.h b/sys/sys/cdefs.h index > > 9cdc03c861cb..e370f6d6459e 100644 --- a/sys/sys/cdefs.h +++ > > b/sys/sys/cdefs.h @@ -396,6 +396,14 @@ #define __unreachable() > > ((void)0) #endif > > > > +#if __has_attribute(optnone) +#define __no_optimization > > __attribute__((optnone)) +#elif __has_attribute(optimize) +#define > > __no_optimization __attribute__((optimize(0))) +#else +#define > > __no_optimization +#endif + /* XXX: should use `#if __STDC_VERSION__ < > > 199901'. */ #if !__GNUC_PREREQ__(2, 7) && !defined(__INTEL_COMPILER) > > #define __func__ NULL ___ > > freebsd-hack...@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to > > "freebsd-hackers-unsubscr...@freebsd.org" > ___ > freebsd-toolchain@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain > To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org" ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: head -r320458 (e.g.) amd64 -> powerpc (32-bit) cross-buildworld fails for time libc++ static_asserts and overflow/underflow of long long (system clang/clang++ 4 based build)
On Thu, Jun 29, 2017 at 12:47:10PM +0200, Dimitry Andric wrote: > One nasty problem with this is that it is not possible to figure out at > compile time what the size of time_t is. You always need some sort of > configure-time test, and an external define. It is arguably possible, with constexpr. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: lang/gcc6-aux for head beyond __nonnull related issues: vm_ooffset_t and vm_pindex_t related changes (and more)
On Wed, Jul 19, 2017 at 10:06:13PM -0400, Alexander Kabaev wrote: > On Wed, 19 Jul 2017 22:50:04 +0200 (CEST) > Gerald Pfeifer wrote: > > > On Fri, 14 Apr 2017, Alexander Kabaev wrote: > > > it was suggested multiple times that the whole fixinc step is > > > ultimately harmful and serves no useful purpose and probably should > > > be disabled in built packages outright. Is there a reason not to do > > > it? Even Redhat appears to do the slimming in their rpms: > > > > For the more current lang/gcc* ports (not the gcc5-aux and gcc6-aux > > ports which I do not maintain) I have now removed packaging the > > headers processed by fixincludes, so any problems in that direction > > should be gone. > > > > Gerald > > Thank you, Gerald! This is very good news, thank you. Gcc should be much more usable now. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: FCP-100: armv7 plan
On Sat, Sep 09, 2017 at 02:34:10PM -0600, Ian Lepore wrote: > Adding the AT_HWCAP stuff would be relatively easy. I'm not as sure > what to do about getauxval(). To be maximally linux-compatible (which > would be good for ports) we'd put getauxval() in libc and make it work > just like the linux one. That's a bit at odds with the support we have > now, which is procstat_getauxv() in libprocstat. It's not very > compatible with how linux getauxval() works, so using just that might > lead to a lot of patches in ports. libprocstat is for accessing other processes information and address space. Our libc already has _elf_aux_info, but it is not exported. If you have a clear description of the desired API, I can add it (I do not want to read glibc code). ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: splitting libc -> libc + libsys and static linking
On Tue, Apr 03, 2018 at 10:39:44PM +0200, Dimitry Andric wrote: > On 3 Apr 2018, at 22:32, Brooks Davis wrote: > > > > We (mostly Ali) are working on a patch to to split the actual syscalls > > (__sys_) out of libc and into a libsys. For dynamic linking, > > this is fairly straightforward (link libc against libsys, maybe as a > > filter). For static linking, I'm looking for feedback on the right > > approach. Do we link libsys.a into libc.a? Do we try to teach all the > > compilers to add -lsys? I'm pretty sure we don't modify all the ports > > that statically link programs. Is there some easy approach I'm missing? > > The approach chosen for e.g. libc++ (and before that, libstdc++), was to > put all needed objects from the dependent libs in the same .a file. This would make libsys.a half as useless. One of the indended use of libsys.a in the base system is to get rid of libc in rtld dependencies. Try to experiment with the linker script for libc.a. > > See: > https://svnweb.freebsd.org/base/head/lib/libc%2B%2B/Makefile?revision=321369&view=markup#l61 > > and: > https://svnweb.freebsd.org/base/head/gnu/lib/libstdc%2B%2B/Makefile?revision=315175&view=markup#l57 > > For dynamic linking, you could use your approach of putting libsys in > the NEEDED section, or add it to the libc.so linker script, like what is > done for libc++.so. For dynamic library, this is wrong. libsys must not appear in the dependency list of the final binaries, otherwise you break ABI and old binaries which were not linked against libsys. This library only contains private-namespace symbols, and should be not exposed to applications. As was already mentioned, a filtering for the libsys.so symbols in libc is good enough approach. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: A head buildworld race visible in the ci.freebsd.org build history
On Mon, Jun 18, 2018 at 12:42:46PM -0700, Bryan Drewery wrote: > On 6/15/2018 10:55 PM, Mark Millard wrote: > > In watching ci.freebsd.org builds I've seen a notable > > number of one time failures, such as (example from > > powerpc64): > > > > --- all_subdir_lib/libufs --- > > ranlib -D libufs.a > > ranlib: fatal: Failed to open 'libufs.a' > > *** [libufs.a] Error code 70 > > > > where the next build works despite the change being > > irrelevant to whatever ranlib complained about. > > > > Other builds failed similarly: > > > > --- all_subdir_lib/libbsm --- > > ranlib -D libbsm_p.a > > ranlib: fatal: Failed to open 'libbsm_p.a' > > *** [libbsm_p.a] Error code 70 > > > > and: > > > > --- kerberos5/lib__L --- > > ranlib -D libgssapi_spnego_p.a > > --- libgssapi_spnego.a --- > > ranlib -D libgssapi_spnego.a > > --- libgssapi_spnego_p.a --- > > ranlib: fatal: Failed to open 'libgssapi_spnego_p.a' > > *** [libgssapi_spnego_p.a] Error code 70 > > > > and so on. > > > > > > It is not limited to powerpc64. For example, for aarch64 > > there are: > > > > --- libpam_exec.a --- > > building static pam_exec library > > ar -crD libpam_exec.a `NM='nm' NMFLAGS='' lorder pam_exec.o | tsort -q` > > ranlib -D libpam_exec.a > > ranlib: fatal: Failed to open 'libpam_exec.a' > > *** [libpam_exec.a] Error code 70 > > > > and: > > > > --- all_subdir_lib/libusb --- > > ranlib -D libusb.a > > ranlib: fatal: Failed to open 'libusb.a' > > *** [libusb.a] Error code 70 > > > > and: > > > > --- all_subdir_lib/libbsnmp --- > > ranlib: fatal: Failed to open 'libbsnmp.a' > > --- all_subdir_lib/ncurses --- > > --- all_subdir_lib/ncurses/panelw --- > > --- panel.pico --- > > --- all_subdir_lib/libbsnmp --- > > *** [libbsnmp.a] Error code 70 > > > > > > Even amd64 gets such: > > > > --- libpcap.a --- > > ranlib -D libpcap.a > > ranlib: fatal: Failed to open 'libpcap.a' > > *** [libpcap.a] Error code 70 > > > > and: > > > > > > --- libkafs5.a --- > > ranlib: fatal: Failed to open 'libkafs5.a' > > --- libkafs5_p.a --- > > ranlib: fatal: Failed to open 'libkafs5_p.a' > > --- cddl/lib__L --- > > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/lua/lbaselib.c:60:26: > > note: include the header or explicitly provide a declaration for > > 'toupper' > > --- kerberos5/lib__L --- > > *** [libkafs5_p.a] Error code 70 > > > > make[5]: stopped in /usr/src/kerberos5/lib/libkafs5 > > --- libkafs5.a --- > > *** [libkafs5.a] Error code 70 > > > > and: > > > > > > --- lib__L --- > > ranlib -D libclang_rt.asan_cxx-i386.a > > ranlib: fatal: Failed to open 'libclang_rt.asan_cxx-i386.a' > > *** [libclang_rt.asan_cxx-i386.a] Error code 70 > > > > > > (Notice the variability in what .a the ranlib's fail for.) > > > > > > > > > > > > > I looked at this a few days ago and don't believe it's actually a build > race. I think there is something wrong with the ar/ranlib on that system > or something else. I've found no evidence of concurrent building of the > .a files in question. FWIW, I got the similar failure when I did last checks for the OFED commit. For me, it was libgcc.a. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: svn commit: r339898 - head/lib/libc/amd64/sys
On Sat, Nov 03, 2018 at 08:52:16AM -0400, Charlie Li wrote: > On 01/11/2018 15:43, Charlie Li wrote: > > On 01/11/2018 12:04, Brooks Davis wrote: > >> Is this failure with devel/llvm70? It's currently missing the patch > >> required to make this work. https://reviews.freebsd.org/D17709 contains > >> this patch among others. I'll see about getting it applied. > >> > > Yes, devel/llvm70. Will build with your port commit at my next opportunity. > > > After building world and kernel r340097, kernel runs fine, but every > userspace program in world crashes with Illegal instruction. They all > crash in exactly the same way. Example backtrace from bmake, running > from objdir (first discovered after updating a poudriere jail and > attempting to even start it): > > Reading symbols from > /usr/obj/usr/src/amd64.amd64/usr.bin/bmake/make...Reading symbols from > /usr/obj/usr/src/amd64.amd64/usr.bin/bmake/make.debug...done. > done. > [New LWP 100097] > Core was generated by `/usr/obj/usr/src/amd64.amd64/usr.bin/bmake/make > --help'. > Program terminated with signal SIGILL, Illegal instruction. > #0 0x002f5664 in _init () > (gdb) bt > #0 0x002f5664 in _init () > #1 0x002290fe in _start (ap=, cleanup= out>) at /usr/src/lib/csu/amd64/crt1.c:66 > > Given the line number referenced in crt1.c, I'm guessing this condition > may have existed since at least r339351. Some minimal amount of facts instead of guesses would be much more useful. What is the instruction which faulted ? Disassemble the text at 0x2f5664. Regardless of what is the instruction, show either the output from 'x86info -f' on the machine, or cpu identification lines from the _verbose_ boot dmesg. make is statically linked, do dynamically linked program fault ? ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: svn commit: r339898 - head/lib/libc/amd64/sys
On Sat, Nov 03, 2018 at 06:59:02PM -0400, Charlie Li wrote: > On 03/11/2018 11:29, Konstantin Belousov wrote: > > Some minimal amount of facts instead of guesses would be much more useful. > > > Yeah, being sleep deprived and hurried (on my end) certainly doesn't help. > > What is the instruction which faulted ? Disassemble the text at 0x2f5664. > > Regardless of what is the instruction, show either the output from > > 'x86info -f' on the machine, or cpu identification lines from the > > _verbose_ boot dmesg. > > > It appears that 0x2f5664 does not exist: Or rather, it is a middle of the valid instruction. Next frame looks like it is process_irelocs(), if trusting the line numbers. So most likely it is something related to calling wrong relocator function, if anything. Perhaps you could try to trace the things manually, doing single-stepping of the startup code in debugger. There should be very modest amount of the irelocs, perhaps only one, and see where things go off the way. Might be try to vary the clang version, we know that this work with 6.0.1, and according to your report, breaks with 7.0. Try clang trunk ? > > Disassembly of section .init: > > 002f565c <_init>: > 2f565c: 48 83 ec 08 sub$0x8,%rsp > 2f5660: e8 fb 3c f3 ff callq 229360 > 2f5665: e8 b6 ff ff ff callq 2f5620 > <__do_global_ctors_aux> > 2f566a: 48 83 c4 08 add$0x8,%rsp > 2f566e: c3 retq > > CPU ident: > > CPU: Intel(R) Core(TM) i7-5500U CPU @ 2.40GHz (2394.52-MHz K8-class CPU) > Origin="GenuineIntel" Id=0x306d4 Family=0x6 Model=0x3d Stepping=4 > > Features=0xbfebfbff > > Features2=0x7ffafbbf > AMD Features=0x2c100800 > AMD Features2=0x121 > Structured Extended > Features=0x21c27ab > Structured Extended Features3=0x9c00 > XSAVE Features=0x1 > VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID > TSC: P-state invariant, performance statistics > > make is statically linked, do dynamically linked program fault ? > > > After some more checks, only the statically linked programs crash. > > -- > Charlie Li > Can't think of a witty .sigline today… > > (This email address is for mailing list use only; replace local-part > with vishwin for off-list communication) > ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: svn commit: r339898 - head/lib/libc/amd64/sys
On Sun, Nov 04, 2018 at 12:43:34AM -0700, Julian Elischer wrote: > what's an ifunc? > A special kind of relocation, controlled by the user code. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: svn commit: r339898 - head/lib/libc/amd64/sys
On Mon, Nov 05, 2018 at 09:10:13PM -0500, Charlie Li wrote:
> On 03/11/2018 19:45, Konstantin Belousov wrote:
> > Or rather, it is a middle of the valid instruction.
> > Next frame looks like it is process_irelocs(), if trusting the line
> > numbers. So most likely it is something related to calling wrong
> > relocator function, if anything.
> >
> > Perhaps you could try to trace the things manually, doing
> > single-stepping of the startup code in debugger. There should be very
> > modest amount of the irelocs, perhaps only one, and see where things go
> > off the way.
> >
> After a few more complete buildworlds, including one with all
> bootstrapping enabled, this doesn't look compiler-specific. Static
> binaries built with the in-tree base LLVM (6.0.1) also crash.
For you, but not for me.
>
> I stepped through bmake with base lldb, comparing the working copy in my
> system from circa r339990 with r340173 built with bootstrapped
> toolchain. Only the differing parts are shown for conciseness.
>
> Circa r339990:
> Process 82271 stopped
> * thread #1, name = 'make', stop reason = step over
> frame #0: 0x0024ab06 make`_init_tls at tls.c:471
>468 }
>469 tls = _rtld_allocate_tls(NULL, TLS_TCB_SIZE, TLS_TCB_ALIGN);
>470
> -> 471 _set_tp(tls);
>472 #endif
>473 }
> (lldb) n
> Process 82271 stopped
> * thread #1, name = 'make', stop reason = step over
> frame #0: 0x00255e60 make`_set_tp(tp=0x0008002f7830) at
> _set_tp.c:38
>35
>36 void
>37 _set_tp(void *tp)
> -> 38 {
>39
>40 amd64_set_fsbase(tp);
>41 }
> (lldb) s
> Process 82271 stopped
> * thread #1, name = 'make', stop reason = step in
> frame #0: 0x00255e64 make`_set_tp(tp=0x0008002f7830) at
> _set_tp.c:40
>37 _set_tp(void *tp)
>38 {
>39
> -> 40 amd64_set_fsbase(tp);
>41 }
> (lldb) s
> Process 82271 stopped
> * thread #1, name = 'make', stop reason = step in
> frame #0: 0x00256580
> make`amd64_set_fsbase(addr=0x0008002f7830) at amd64_set_fsbase.c:43
>40 #include
>41 #include
>42 #include
> -> 43 #include "libc_private.h"
>44
>45 static int
>46 amd64_set_fsbase_cpu(void *addr)
> (lldb)
>
> r340173:
> Process 22663 stopped
> * thread #1, name = 'make', stop reason = step over
> frame #0: 0x00247c96 make`_init_tls at tls.c:471
>468 }
>469 tls = _rtld_allocate_tls(NULL, TLS_TCB_SIZE, TLS_TCB_ALIGN);
>470
> -> 471 _set_tp(tls);
>472 #endif
>473 }
> (lldb) n
> Process 22663 stopped
> * thread #1, name = 'make', stop reason = step over
> frame #0: 0x00252eb0 make`_set_tp(tp=0x0008002ed830) at
> _set_tp.c:38
>35
>36 void
>37 _set_tp(void *tp)
> -> 38 {
>39
>40 amd64_set_fsbase(tp);
>41 }
> (lldb) s
> Process 22663 stopped
> * thread #1, name = 'make', stop reason = step in
> frame #0: 0x00252eb4 make`_set_tp(tp=0x0008002ed830) at
> _set_tp.c:40
>37 _set_tp(void *tp)
>38 {
>39
> -> 40 amd64_set_fsbase(tp);
>41 }
> (lldb) s
> Process 22663 stopped
> * thread #1, name = 'make', stop reason = step in
> frame #0: 0x00252eb5 make`_set_tp(tp=0x0008002ed830) at
> _set_tp.c:40
>37 _set_tp(void *tp)
>38 {
>39
> -> 40 amd64_set_fsbase(tp);
>41 }
> (lldb) s
> Process 22663 stopped
> * thread #1, name = 'make', stop reason = step in
> frame #0: 0x002ebdb0 make
> -> 0x2ebdb0: movq 0x3ce9(%rip), %r11
> 0x2ebdb7: callq 0x2ebda0 ; _fini
> 0x2ebdbc: jmp0x2ebd92 ; _init + 6
> 0x2ebdc1: pushq $0x0
> (lldb) n
> Process 22663 stopped
> * thread #1, name = 'make', stop reason = instruction step over
> frame #0: 0x002ebdb7 make
> -> 0x2ebdb7: callq 0x2ebda0 ; _fini
I guess this is where things go off for you, but I may be wrong.
After ifuncification, 'amd64_set_fsbase()' line should be represented
by the asm as either
call
and the place in plt is just
jmp *(rip-based offset to GOT)(%rip)
In fact the call to amd64_set_fsbase() in the tail-call position, so
the first call is jmp.
For me, everything works. If for you it does not you should look at the
instructions and see which values went off. You completely omitted that
details from your trace, so I cannot even guess which part was corrupted.
Again, for me it works with the in-tree toolchain, so I am quite sure that
you have trouble with the toolchain.
___
freebsd-toolchain@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: Problem with mutex.lock()
On Mon, Feb 11, 2019 at 11:09:29AM +, Banta Plan wrote:
> I compiled this program and it crashes if executed.
>
> Compiled with: FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540)
> (based on LLVM 6.0.1)
>
> ## CMakeLists.txt
> cmake_minimum_required(VERSION 3.13)
> project(mainP)
>
> set(CMAKE_CXX_STANDARD 17)
>
> add_executable(mainP main.cpp)
>
> find_package(Threads)
> target_link_libraries(mainP ${CMAKE_THREAD_LIBS_INIT})
>
> ## main.cpp
>
> #include
> #include
> #include
> #include
> #include
>
> int main() {
> using namespace std::chrono_literals;
> std::mutex m;
> m.lock();
> std::thread([&] {
> std::cout << "Before sleepFor" << std::endl;
> std::this_thread::sleep_for(2s);
>
> std::cout << "Before unlock" << std::endl;
> m.unlock();
This unlock is performed on the mutex that the calling thread does not
hold the lock on. The behaviour for such case is undefined, from my reading
of C++17 std.
FreeBSD pthread mutexes check this condition.
> std::cout << "After unlock" << std::endl;
>
> }).detach();
> std::cout << "Before lock2" << std::endl;
> m.lock();
> std::cout << "After lock2" << std::endl;
>
> return 0;
> }
> ##
>
> This program compiles and runs under MacOS (Clang Version: Apple LLVM version
> 10.0.0 (clang-1000.11.45.5)) and
> manjaro arm64 with gcc (GCC) 8.2.1 20180831
>
>
>
> ___
> freebsd-toolchain@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
> To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
___
freebsd-toolchain@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: Problem with mutex.lock()
On Mon, Feb 11, 2019 at 02:04:38PM +, Banta Plan wrote:
> I think you can reduce the problem to:
> ##
> int main() {
> std::mutex m;
> m.lock();
> m.lock();
>
> return 0;
> }
> ##
>
> This should deadlock.
Where is it specified that the program should deadlock ?
The behaviour for this case is undefined.
> But this will crash the program.
Which is not disallowed by standard.
___
freebsd-toolchain@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: Optimization bug with floating-point?
On Fri, Mar 15, 2019 at 05:50:37AM +1100, Peter Jeremy wrote: > On 2019-Mar-13 23:30:07 -0700, Steve Kargl > wrote: > >AFAICT, all libm float routines need to be modified to conditional > >include ieeefp.h and call fpsetprec(FP_PD). This will work around > >issues is FP and libm. FreeBSD needs to issue an erratum about > >the numerical issues with clang. > > I vaguely recall looking into the x87 initialisation a long time ago > and STR that the startup code (either crtX or in the kernel) does > a fninit() to set the precision. I don't recall exactly where. At boot, a clean initial FPU state is stored in fpu_initialstate. Then on first FPU access from userspace (first for the given process context), this saved state is copied into hardware registers. The quirk is that for i386 binaries on amd64, we adjust fpu control word to what is expected by i386 binaries. > > IMO, calling fpsetprec() in every libm float function is overkill. It > should be enough to fpsetprec() before main() and add a note in the > man pages that libm is built to use the default FPU configuration and > changing the configuration (precision or rounding) may result in larger > errors. Changing default precision in crt1 would break the ABI. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: Optimization bug with floating-point?
On Thu, Mar 14, 2019 at 12:59:14PM -0700, John Baldwin wrote: > On 3/14/19 12:20 PM, Konstantin Belousov wrote: > > On Fri, Mar 15, 2019 at 05:50:37AM +1100, Peter Jeremy wrote: > >> On 2019-Mar-13 23:30:07 -0700, Steve Kargl > >> wrote: > >>> AFAICT, all libm float routines need to be modified to conditional > >>> include ieeefp.h and call fpsetprec(FP_PD). This will work around > >>> issues is FP and libm. FreeBSD needs to issue an erratum about > >>> the numerical issues with clang. > >> > >> I vaguely recall looking into the x87 initialisation a long time ago > >> and STR that the startup code (either crtX or in the kernel) does > >> a fninit() to set the precision. I don't recall exactly where. > > At boot, a clean initial FPU state is stored in fpu_initialstate. > > Then on first FPU access from userspace (first for the given process > > context), this saved state is copied into hardware registers. The > > quirk is that for i386 binaries on amd64, we adjust fpu control word > > to what is expected by i386 binaries. > > > >> > >> IMO, calling fpsetprec() in every libm float function is overkill. It > >> should be enough to fpsetprec() before main() and add a note in the > >> man pages that libm is built to use the default FPU configuration and > >> changing the configuration (precision or rounding) may result in larger > >> errors. > > Changing default precision in crt1 would break the ABI. > > So what I don't understand then is what is gcc doing different than clang > in this case. I assume neither GCC _nor_ clang are adjusting the FPU in > compiler-generated code, and in fact as Steve's earlier tests shows, the > precision is set to PD by default when a clang-built binary is run. Precision control only affect elementary floating-point instructions. Could this be the cause ? SDM vol 1 8.1.5.2 Precision Control Field The precision-control bits only affect the results of the following floating-point instructions: FADD, FADDP, FIADD, FSUB, FSUBP, FISUB, FSUBR, FSUBRP, FISUBR, FMUL, FMULP, FIMUL, FDIV, FDIVP, FIDIV, FDIVR, FDIVRP, FIDIVR, and FSQRT. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: New external GCC toolchain ports/packages
On Fri, Dec 20, 2019 at 09:24:00AM -0800, John Baldwin wrote: > On 12/19/19 12:06 PM, Ryan Libby wrote: > > On Wed, Dec 18, 2019 at 1:49 PM John Baldwin wrote: > >> > >> In the interest of supporting newer versions of GCC for a base system > >> toolchain, I've renamed the external GCC packages from -gcc > >> to -gcc6. These are built as flavors of a new devel/freebsd-gcc6 > >> port. The xtoolchain package is not used for these new packages, instead > >> one does 'pkg install mips-gcc6' to get the GCC 6.x MIPS compiler and > >> uses 'CROSS_TOOLCHAIN=mips-gcc6'. I've also gone ahead and updated this > >> compiler to 6.5.0. > >> > >> I will leave the old ports/packages around for now to permit an easy > >> transition, but going forward, the -gcc6 packages should be preferred > >> to -xtoolchain-gcc for all but riscv (riscv64-gcc and > >> riscv64-xtoolchain-gcc > >> are separate from the powerpc64-gcc set of packages). > >> > >> In addition, I've also just added a devel/freebsd-gcc9 package which > >> builds -gcc9 packages. It adds powerpc and riscv flavors relative > >> to freebsd-gcc6 and uses GCC 9.2.0. To date in my testing I've yet to > >> be able to finish a buildworld on any of the platforms I've tried > >> (amd64, mips, sparc64), but the packages should permit other developers > >> to get the tree building with GCC 9. To use these packages one would do > >> something like: > >> > >> # pkg install amd64-gcc9 > >> # make buildworld CROSS_TOOLCHAIN=amd64-gcc9 > >> > >> You can install both the gcc6 and gcc9 versions of a package at the same > >> time, e.g. amd64-gcc6 and amd64-gcc9. Having different packages for major > >> versions is similar to llvm and will also let us keep a known-good > >> toolchain package for older releases while using newer major versions on > >> newer FreeBSD releases (e.g gcc9 for 13.0 and gcc6 for 12.x). > >> > >> I do plan to switch the default toolchains for make universe/tinderbox > >> for targets using -xtoolchain-gcc based on GCC 6 over to the > >> freebsd-gcc6 variants in the next week or so. > >> > >> -- > >> John Baldwin > > > > Awesome, thanks! I was able to get amd64 buildworld and buildkernel to > > succeed with just a few changes, and none to the port. I'll work on > > getting the changes in. > > I have been able to get it building as well, mostly by muting a few > warnings, adding libcompiler_rt to rtld's link for i386, disabling I am curious about the rtld issue. Can you show me the pristine error with gcc, please ? > googletest (needs an upstream patch to stop using signed wchar_t), > and a hack to jemalloc. I was able to build riscv as well with those > same changes and am working through builds of other platforms. > > I'm happy to compare notes. The jemalloc one is a bit weird. > > -- > John Baldwin > ___ > freebsd-toolchain@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain > To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org" ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: New external GCC toolchain ports/packages
On Fri, Dec 20, 2019 at 09:51:15AM -0800, Ryan Libby wrote: > On Fri, Dec 20, 2019 at 9:31 AM Konstantin Belousov > wrote: > > > > On Fri, Dec 20, 2019 at 09:24:00AM -0800, John Baldwin wrote: > > > On 12/19/19 12:06 PM, Ryan Libby wrote: > > > > On Wed, Dec 18, 2019 at 1:49 PM John Baldwin wrote: > > > >> > > > >> In the interest of supporting newer versions of GCC for a base system > > > >> toolchain, I've renamed the external GCC packages from -gcc > > > >> to -gcc6. These are built as flavors of a new devel/freebsd-gcc6 > > > >> port. The xtoolchain package is not used for these new packages, > > > >> instead > > > >> one does 'pkg install mips-gcc6' to get the GCC 6.x MIPS compiler and > > > >> uses 'CROSS_TOOLCHAIN=mips-gcc6'. I've also gone ahead and updated > > > >> this > > > >> compiler to 6.5.0. > > > >> > > > >> I will leave the old ports/packages around for now to permit an easy > > > >> transition, but going forward, the -gcc6 packages should be > > > >> preferred > > > >> to -xtoolchain-gcc for all but riscv (riscv64-gcc and > > > >> riscv64-xtoolchain-gcc > > > >> are separate from the powerpc64-gcc set of packages). > > > >> > > > >> In addition, I've also just added a devel/freebsd-gcc9 package which > > > >> builds -gcc9 packages. It adds powerpc and riscv flavors > > > >> relative > > > >> to freebsd-gcc6 and uses GCC 9.2.0. To date in my testing I've yet to > > > >> be able to finish a buildworld on any of the platforms I've tried > > > >> (amd64, mips, sparc64), but the packages should permit other developers > > > >> to get the tree building with GCC 9. To use these packages one would > > > >> do > > > >> something like: > > > >> > > > >> # pkg install amd64-gcc9 > > > >> # make buildworld CROSS_TOOLCHAIN=amd64-gcc9 > > > >> > > > >> You can install both the gcc6 and gcc9 versions of a package at the > > > >> same > > > >> time, e.g. amd64-gcc6 and amd64-gcc9. Having different packages for > > > >> major > > > >> versions is similar to llvm and will also let us keep a known-good > > > >> toolchain package for older releases while using newer major versions > > > >> on > > > >> newer FreeBSD releases (e.g gcc9 for 13.0 and gcc6 for 12.x). > > > >> > > > >> I do plan to switch the default toolchains for make universe/tinderbox > > > >> for targets using -xtoolchain-gcc based on GCC 6 over to the > > > >> freebsd-gcc6 variants in the next week or so. > > > >> > > > >> -- > > > >> John Baldwin > > > > > > > > Awesome, thanks! I was able to get amd64 buildworld and buildkernel to > > > > succeed with just a few changes, and none to the port. I'll work on > > > > getting the changes in. > > > > > > I have been able to get it building as well, mostly by muting a few > > > warnings, adding libcompiler_rt to rtld's link for i386, disabling > > I am curious about the rtld issue. Can you show me the pristine error with > > gcc, please ? > > For me, it's gcc generates __udivmoddi4 for the -m32 build but we used > -nostdlib, and we get a link error. My hack right now is to explicitly > link libgcc, but I don't think this is what we are intending to do > because we seem to go through effort to reimplement other libgcc > functions. > > Here's the text: > --- ld-elf32.so.1.full --- > /usr/local/bin/x86_64-unknown-freebsd13.0-ld: rtld_printf.o: in > function `ksprintn': > /usr/src/freebsd/libexec/rtld-elf/rtld_printf.c:115: undefined > reference to `__udivmoddi4' > /usr/local/bin/x86_64-unknown-freebsd13.0-ld: > /usr/src/freebsd/libexec/rtld-elf/rtld_printf.c:117: undefined > reference to `__udivmoddi4' > collect2: error: ld returned 1 exit status > *** [ld-elf32.so.1.full] Error code 1 > > You can see my hack at the link below. I'm not sure what the > right solution is. Does changing the base type from int to u_int help ? > > > > > > googletest (needs an upstream patch to stop using signed wchar_t), > > > and a hack to jemalloc. I was able to build riscv as well with those > > > same changes and am working through builds of other platforms. > > > > > Actually, both those have upstream fixes. (For the jemalloc one, gcc is > technically right, but because of a literal signed char argument to a > macro.) > > > > I'm happy to compare notes. The jemalloc one is a bit weird. > > I pushed non-polished changes here. I'm working to get them reviewed. > Please feel free to comment on any/all. > https://github.com/rlibby/freebsd/commits/gcc9-fixes > > > > > > > -- > > > John Baldwin > > > ___ > > > freebsd-toolchain@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain > > > To unsubscribe, send any mail to > > > "freebsd-toolchain-unsubscr...@freebsd.org" ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: x86 process memory map
On Mon, Mar 30, 2020 at 08:18:08AM +0200, Paul Floyd wrote: > When I run procstat on a small 32bit app that just calls sleep (on FreeBSD > 12.1 amd64) then I see at the end of the map > > 22353 0xfbffe000 0xfffde000 ---00 0 0 - -- > 22353 0xfffde000 0xe000 rw-33 1 0 ---D- df > 22353 0xe000 0xf000 r-x11 94 0 - ph > > I think the last block is for signal handlers and the last but one block is > the user stack. > But what is between 0xfbffe000 and 0xfffde000? It's a bit less than 64Mbytes. > This has no protection flags, no resident pages or references and no type. My > guess is that this is some sort of guard page for the user stack. > It is the stack grow area and the guard, combined. Read the mmap(2), in particular explanation of MAP_STACK and MAP_GUARD. > Can anyone confirm this? Also, does anyone know if this has been present in > FreeBSD for a long time? My copy of “The Design and Implementation of > FreeBSD" 2e doesn't show anything between stack and shared libraries. > > A+ > Paul > > > ___ > freebsd-toolchain@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain > To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org" ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: x86 process memory map
On Tue, Mar 31, 2020 at 03:04:18PM +0200, Paul FLOYD wrote: > > It is the stack grow area and the guard, combined. Read the mmap(2), in> > > particular explanation of MAP_STACK and MAP_GUARD. > I see from the mmap man page that this appeared in FreeBSD 11.1. > > Do you know where the size of the stack guard gets defined? I tried searching > on google for stuff like 'stack MAP_GUARD size' but didn't find anything. Guard fills the rest of the stack region, so that [guard; mapped stack] fill the whole area as requested by mmap. Start of the guard is at the starting address requested from mmap (or auto-selected), end of the mapped area is at guard start + total len. I believe it is explained in the man page. > > (I need this to fix setting up the client stack on Valgrind x86, which is > currently broken) > > A+ > Paul > > ___ > freebsd-toolchain@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain > To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org" ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: Getting started with clang debuginfo
On Mon, Jul 27, 2020 at 10:37:53AM -0400, Mark Johnston wrote: > On Mon, Jul 27, 2020 at 11:27:40AM +0200, Paul FLOYD wrote: > > Hi > > > > I'm investigating some of the remaining issues with Valgrind on FreeBSD. > > One of the two remaining major issues that I'm aware of is with Valgrind > > reading dwarf debuginfo from clang compiled binaries. The problem isn't > > too bad with clang 8 on FreeBSD 12.1. On 13-CURRENT with clang 10 things > > are noticeably worse. For GCC built binaries I'm not aware of any issues. > > > > I'm not familiar (yet) with the debuginfo code in Valgrind. > > > > To get me going, does anyone have any pointers to > > - documentation on clang debuginfo Clang generates DWARF which is documented by the DWARF standard(s), available at http://dwarfstd.org/ > > - any info on differences wrt GCC (I have seen that GCC does have some > > debuginfo extensions) Gcc also generates DWARF. It is up to the compiler to interpret the standard and provide compliant metadata according to it. But I would expect that the practical difference or troubles in parsing the compiler' output is due to different versions of the used standard. Both gcc and clang allow to specify which level of standard should be used, see the description of the -gdwarf- switch. Different versions of the same compiler might default to different version of DWARF as well. > > - any tools that would be useful like ascii dumps > > Are you asking about tools to dump DWARF info? I've used dwarfdump > (from ports) quite a bit in the past. readelf -w is similar and > available in the base system. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: Undefined reference to __atomic_store_8
On Fri, Aug 07, 2020 at 08:42:12PM +0400, Gleb Popov wrote: > Hello toolchain@ > > I'm testing a new release of GHC (Haskell compiler) and it fails to link to > i386 architectures with > > /wrkdirs/usr/ports/lang/ghc/work/ghc-8.10.1/libraries/ghc-prim/dist-install/build/ > libHSghc-prim-0.6.1-ghc8.10.1.so: undefined reference to `__atomic_store_8' > /wrkdirs/usr/ports/lang/ghc/work/ghc-8.10.1/libraries/ghc-prim/dist-install/build/ > libHSghc-prim-0.6.1-ghc8.10.1.so: undefined reference to `__atomic_load_8' > > Any suggestions on how to fix this? Either link to libatomic from recent gcc, or switch target cpu to something that is pentium or newer. I doubt that it is reasonable to run GHC on 486. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: Undefined reference to __atomic_store_8
On Sat, Aug 08, 2020 at 12:37:42PM +0400, Gleb Popov wrote: > On Sat, Aug 8, 2020 at 1:29 AM Konstantin Belousov > wrote: > > > On Fri, Aug 07, 2020 at 08:42:12PM +0400, Gleb Popov wrote: > > > Hello toolchain@ > > > > > > I'm testing a new release of GHC (Haskell compiler) and it fails to link > > to > > > i386 architectures with > > > > > > > > /wrkdirs/usr/ports/lang/ghc/work/ghc-8.10.1/libraries/ghc-prim/dist-install/build/ > > > libHSghc-prim-0.6.1-ghc8.10.1.so: undefined reference to > > `__atomic_store_8' > > > > > /wrkdirs/usr/ports/lang/ghc/work/ghc-8.10.1/libraries/ghc-prim/dist-install/build/ > > > libHSghc-prim-0.6.1-ghc8.10.1.so: undefined reference to > > `__atomic_load_8' > > > > > > Any suggestions on how to fix this? > > > > Either link to libatomic from recent gcc, or switch target cpu to something > > that is pentium or newer. I doubt that it is reasonable to run GHC on 486. > > > > By switching target CPU you mean passing -march=pentium? For code generated by gcc or clang, yes. If the reference to the symbol was generated by ghc, then I do not know. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: Undefined reference to __atomic_store_8
On Sun, Aug 09, 2020 at 02:37:42PM +0200, Tijl Coosemans wrote: > On Sun, 9 Aug 2020 15:36:51 +0400 Gleb Popov wrote: > > On Sat, Aug 8, 2020 at 5:30 PM Konstantin Belousov > > wrote: > >> For code generated by gcc or clang, yes. > >> If the reference to the symbol was generated by ghc, then I do not know. > > > > This doesn't seem to work. > > > > The code referencing __atomic_load_n() is C and GHC buildsystem already > > passes -march=i686. Still, the problem persists. > > Interestingly, 12.1-RELEASE-p2 doesn't have this problem, but > > 12.1-RELEASE-p7 does. > > > > What library provides these symbols when clang is used? And I'm a bit > > obscured how -march flag can affect these symbols' visibility at all? > > There is no such library. i586 supports 64 bit atomic operations so > the compiler should emit instructions for that instead of a function > call. Check that *.o files that match "__atomic_load" are in fact built > with -march. > > 12.1-RELEASE-p7 probably has a newer clang that fixed an issue where the > compiler would always emit instructions, even for i386 and i486. I do not believe there were any change in the toolchain between p2 and p7, this is more likely indicates some fluctuation in the build. The only change that could be even remotely declared as possibly related is EN-20:10.build r360473. Indeed you need to look at the .o files that reference _8 symbol. I would closely look at the compilation command used for them, for start. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: Undefined reference to __atomic_store_8
On Tue, Aug 11, 2020 at 03:56:35PM +0400, Gleb Popov wrote: > On Sun, Aug 9, 2020 at 7:43 PM Konstantin Belousov > wrote: > > > I do not believe there were any change in the toolchain between p2 and p7, > > this is more likely indicates some fluctuation in the build. The only > > change that could be even remotely declared as possibly related is > > EN-20:10.build r360473. > > > > Right, I was using a wrong set of port's OPTIONS that hide the problem. > > Indeed you need to look at the .o files that reference _8 symbol. I would > > closely look at the compilation command used for them, for start. > > > > After digging it a bit I found that the following command > > cc -x c > /wrkdirs/usr/ports/lang/ghc/work/ghc-8.10.1/libraries/ghc-prim/cbits/atomic.c > -o /tmp/ghc_1.s -fno-PIC -Wimplicit -S \ > -include > /wrkdirs/usr/ports/lang/ghc/work/ghc-8.10.1/includes/dist-install/build/ghcversion.h > -I/usr/local/include \ > -I/wrkdirs/usr/ports/lang/ghc/work/ghc-8.10.1/libraries/base/include \ > -I/wrkdirs/usr/ports/lang/ghc/work/ghc-8.10.1/libraries/base/dist-install/build/include > \ > -I/wrkdirs/usr/ports/lang/ghc/work/ghc-8.10.1/libraries/base/dist-install/build/dist-install/build/include > \ > -I/wrkdirs/usr/ports/lang/ghc/work/ghc-8.10.1/libraries/integer-gmp/include > \ > -I/wrkdirs/usr/ports/lang/ghc/work/ghc-8.10.1/libraries/integer-gmp/dist-install/build/include > \ > -I/wrkdirs/usr/ports/lang/ghc/work/ghc-8.10.1/libraries/integer-gmp/dist-install/build/dist-install/build/include > \ > -I/wrkdirs/usr/ports/lang/ghc/work/ghc-8.10.1/rts/dist/build > -I/wrkdirs/usr/ports/lang/ghc/work/ghc-8.10.1/includes \ > -I/wrkdirs/usr/ports/lang/ghc/work/ghc-8.10.1/includes/dist-derivedconstants/header > \ > -I/wrkdirs/usr/ports/lang/ghc/work/ghc-8.10.1/includes/dist-install/build \ > -march=i686 -U__i686 > > produce an assembly file with > > calll __atomic_load_8 > > instruction. > > The value of -march flag seems to be ignored. > > Interestingly, previous version of GHC calls C compiler in the following > way: > > cc -U__i686 '-march=i686' -fno-stack-protector -DTABLES_NEXT_TO_CODE > '-march=i686' -x c > /wrkdirs/usr/ports/lang/ghc/work/ghc-8.10.1/libraries/ghc-prim/cbits/atomic.c > -o /tmp/ghc_1.s -Wimplicit -S \ > -include > /wrkdirs/usr/ports/lang/ghc/work/ghc-8.6.5-boot/lib/ghc-8.6.5/include/ghcversion.h > \ > -I/usr/local/include \ > -I/wrkdirs/usr/ports/lang/ghc/work/ghc-8.6.5-boot/lib/ghc-8.6.5/base-4.12.0.0/include > \ > -I/wrkdirs/usr/ports/lang/ghc/work/ghc-8.6.5-boot/lib/ghc-8.6.5/include > > And this command produces an assembly without calls to __atomic_load_8 > > Any ideas what makes it appear? But was the compiler version identical in both cases ? Also, it might be that -O/-O2 could make a difference between inlining cmpxchg8b vs. calling libatomic, even with -march=i686. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: Undefined reference to __atomic_store_8
On Wed, Aug 12, 2020 at 01:41:58PM +0200, Tijl Coosemans wrote:
> On Wed, 12 Aug 2020 09:44:25 +0400 Gleb Popov wrote:
> > On Wed, Aug 12, 2020 at 9:21 AM Gleb Popov wrote:
> >> Indeed, this looks like a culprit! When compiling using first command line
> >> (the long one) I get following warnings:
> >>
> >> /wrkdirs/usr/ports/lang/ghc/work/ghc-8.10.1/libraries/ghc-prim/cbits/atomic.c:369:10:
> >> warning: misaligned atomic operation may incur significant performance
> >> penalty [-Watomic-alignment]
> >> return __atomic_load_n((StgWord64 *) x, __ATOMIC_SEQ_CST);
> >> ^
> >> /wrkdirs/usr/ports/lang/ghc/work/ghc-8.10.1/libraries/ghc-prim/cbits/atomic.c:417:3:
> >> warning: misaligned atomic operation may incur significant performance
> >> penalty [-Watomic-alignment]
> >> __atomic_store_n((StgWord64 *) x, (StgWord64) val, __ATOMIC_SEQ_CST);
> >> ^
> >> 2 warnings generated.
> >>
> >> I guess this basically means "I'm emitting a call there". So, what's the
> >> correct fix in this case?
> >
> > I just noticed that Clang emits these warnings (and the call instruction)
> > only for functions handling StgWord64 type. For the same code with
> > StgWord32, like
> >
> > StgWord
> > hs_atomicread32(StgWord x)
> > {
> > #if HAVE_C11_ATOMICS
> > return __atomic_load_n((StgWord32 *) x, __ATOMIC_SEQ_CST);
> > #else
> > return __sync_add_and_fetch((StgWord32 *) x, 0);
> > #endif
> > }
> >
> > no warning is emitted as well as no call.
> >
> > How does clang infer alignment in these cases? What's so special about
> > StgWord64?
>
> StgWord64 is uint64_t which is unsigned long long which is 4 byte
> aligned on i386. Clang wants 8 byte alignment to use the fildll
> instruction.
This all is very strange.
How could code use fildll to load 8 bytes as bit-value ? FILDLL converts
single and double precision fp into long-double fp, so it would change
the bit-value.
Also, both ISA and x86 psABI only require 4-byte alignment for the
double precision fp variables.
If the variable memory spans over two cache lines, then SDM states that
the access can be not atomic, but I believe it cannot happen for any
existing CPU. It might be slow. For some future CPUs, Intel provides
control which would cause such accesses to trap.
>
> You could change the definition of the StgWord64 type to look like:
>
> typedef uint64_t StgWord64 __attribute__((aligned(8)));
>
> But this only works if all calls to hs_atomicread64 pass a StgWord64
> as argument and not some other 64 bit value.
>
>
> Another solution I already mentioned in a previous message: replace
> HAVE_C11_ATOMICS with 0 in hs_atomicread64 so it uses
> __sync_add_and_fetch instead of __atomic_load_n. That uses the
> cmpxchg8b instruction which doesn't care about alignment. It's much
> slower but I guess 64 bit atomic loads are rare enough that this
> doesn't matter much.
___
freebsd-toolchain@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
[Differential] [Accepted] D1428: readelf: Handle note types from different operating systems
kostikbel accepted this revision. kostikbel added a reviewer: kostikbel. This revision is now accepted and ready to land. REVISION DETAIL https://reviews.freebsd.org/D1428 To: emaste, kostikbel Cc: rpaulo, freebsd-toolchain ___ freebsd-toolchain@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
[Differential] [Accepted] D1826: libdwarf: Add symbol value when processing .rela relocations
kostikbel accepted this revision.
kostikbel added a comment.
This revision is now accepted and ready to land.
It looks fine, but definitely is extremely subtle. The dwarf_get_reloc_size()
returns zero for non-abs relocations, so dwarf_write_{m.l}sb() does nothing for
other relocation types.
This at least deserves explanation in a comment.
REVISION DETAIL
https://reviews.freebsd.org/D1826
To: emaste, rpaulo, kostikbel
Cc: freebsd-toolchain
___
freebsd-toolchain@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
[Differential] [Accepted] D1826: libdwarf: Add symbol value when processing .rela relocations
kostikbel accepted this revision. This revision is now accepted and ready to land. REVISION DETAIL https://reviews.freebsd.org/D1826 To: emaste, rpaulo, kostikbel Cc: freebsd-toolchain ___ freebsd-toolchain@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
