Profile-guided optimization is a relatively new GCC feature that
improves the quality of generated code by:
- compiling a copy of the source program with some profiling hooks
- running this copy of the program on some representative input data
- recompiling the program using the profiling data produced by the
previous stage; the profiling data lets GCC's optimizer generate more
efficient code
I think it would be cool to add support for PGO to PostgreSQL's build
system (for 8.1). There are a lot of situations where PostgreSQL is
compiled once, and then used for weeks or months (compilations for
inclusion in a distro being the extreme example). In that kind of
situation, trading some additional compile-time for even a small
improvement in run-time performance is worthwhile, IMHO.
I've attached a proof-of-concept patch that implements this. Caveats:
- you'll need to re-run autoconf
- the libgcov.a stuff is a temporary hack, you may need to adjust it for
where libgcov.a is on your system
- I've only bothered adding support for GCC 3.4 (IIRC profile-guided
optimization was introduced in GCC 3.3, but 3.4 adds a simpler interface
to using it). By the time 8.1 is out I think GCC 3.4+ will be pretty
prevalent anyway.
- the patch should remove the .gcda and .gcno files that are produced by
GCC; I haven't done that yet
The patch adds a new make target ("profile-opt") that does the PGO steps
outlined above -- the "representative input data" is the regression
tests running in serial mode. I haven't run any benchmarks yet (if
someone wants to try that, I'd be very curious to see the results).
Comments?
-Neil
? autom4te.cache
Index: GNUmakefile.in
===================================================================
RCS file: /home/neilc/private-cvsroot/pgsql-server/GNUmakefile.in,v
retrieving revision 1.40
diff -c -r1.40 GNUmakefile.in
*** GNUmakefile.in 30 Jul 2004 12:26:39 -0000 1.40
--- GNUmakefile.in 30 Sep 2004 08:06:20 -0000
***************
*** 49,62 ****
# Garbage from autoconf:
@rm -rf autom4te.cache/
! check: all
! check installcheck:
$(MAKE) -C src/test $@
GNUmakefile: GNUmakefile.in $(top_builddir)/config.status
./config.status $@
##########################################################################
--- 49,79 ----
# Garbage from autoconf:
@rm -rf autom4te.cache/
! singlecheck check: all
! singlecheck check installcheck:
$(MAKE) -C src/test $@
GNUmakefile: GNUmakefile.in $(top_builddir)/config.status
./config.status $@
+ # XXX: do we need to run clean first?
+ profile-opt:
+ ifneq ($(profile_opt_support), yes)
+ @echo "Your C compiler does not contain support for profile-guided optimization."
+ @echo "PostgreSQL currently supports PGO when compiled with GCC 3.4 or later"
+ @false
+ else
+ @echo "Building PostgreSQL with support for profile generation"
+ $(MAKE) clean
+ $(MAKE) all CFLAGS="$(CFLAGS) -fprofile-generate"
+ @echo "Running regression tests to generate profile data"
+ $(MAKE) -C src/test/regress singlecheck
+ @echo "Rebuilding PostgreSQL to use profile data"
+ $(MAKE) clean
+ $(MAKE) all CFLAGS="$(CFLAGS) -fprofile-use"
+ @echo "Removing profile data"
+ endif
##########################################################################
Index: configure.in
===================================================================
RCS file: /home/neilc/private-cvsroot/pgsql-server/configure.in,v
retrieving revision 1.378
diff -c -r1.378 configure.in
*** configure.in 27 Sep 2004 02:17:14 -0000 1.378
--- configure.in 30 Sep 2004 08:06:20 -0000
***************
*** 253,258 ****
--- 253,261 ----
# Need to specify -fno-strict-aliasing too in case it's gcc 3.3 or later.
PGAC_PROG_CC_NO_STRICT_ALIASING
+ # Check whether the compiler supports profile-guided optimization
+ PGAC_PROG_CC_PROFILE_GUIDED_OPT
+
# supply -g if --enable-debug
if test "$enable_debug" = yes && test "$ac_cv_prog_cc_g" = yes; then
CFLAGS="$CFLAGS -g"
Index: config/c-compiler.m4
===================================================================
RCS file: /home/neilc/private-cvsroot/pgsql-server/config/c-compiler.m4,v
retrieving revision 1.12
diff -c -r1.12 c-compiler.m4
*** config/c-compiler.m4 2 Feb 2004 04:07:18 -0000 1.12
--- config/c-compiler.m4 30 Sep 2004 08:06:21 -0000
***************
*** 150,155 ****
--- 150,177 ----
CFLAGS="$CFLAGS $pgac_cv_prog_cc_no_strict_aliasing"
fi])# PGAC_PROG_CC_NO_STRICT_ALIASING
+ # PGAC_PROG_CC_PROFILE_GUIDED_OPT
+ # -------------------------------
+ # Find out if the C compiler supports profile-guided optimization. For
+ # the moment, we only support GCC (and GCC 3.4+ at that).
+ AC_DEFUN([PGAC_PROG_CC_PROFILE_GUIDED_OPT],
+ [AC_CACHE_CHECK([if $CC supports profile-guided optimization],
+ pgac_cv_prog_cc_pgo_support,
+ [pgac_save_CFLAGS=$CFLAGS
+ pgac_cv_prog_cc_pgo_support=no
+ if test "$GCC" = yes; then
+ CFLAGS="$pgac_save_CFLAGS -fprofile-use"
+ _AC_COMPILE_IFELSE([AC_LANG_PROGRAM()],
+ [pgac_cv_prog_cc_pgo_support=yes
+ break])
+ fi
+
+ CFLAGS=$pgac_save_CFLAGS
+ ])
+
+ gcc_pgo_support=$pgac_cv_prog_cc_pgo_support
+ AC_SUBST(gcc_pgo_support)
+ ]) # PGAC_PROG_CC_PROFILE_GUIDED_OPT
# The below backpatches the following Autoconf change:
#
Index: src/Makefile.global.in
===================================================================
RCS file: /home/neilc/private-cvsroot/pgsql-server/src/Makefile.global.in,v
retrieving revision 1.195
diff -c -r1.195 Makefile.global.in
*** src/Makefile.global.in 18 Sep 2004 13:28:54 -0000 1.195
--- src/Makefile.global.in 30 Sep 2004 08:06:21 -0000
***************
*** 185,190 ****
--- 185,193 ----
CFLAGS += -Wall -Wmissing-prototypes -Wmissing-declarations
endif
+ # profile-guided optimization
+ profile_opt_support = @gcc_pgo_support@
+
# Kind-of compilers
YACC = @YACC@
Index: src/backend/Makefile
===================================================================
RCS file: /home/neilc/private-cvsroot/pgsql-server/src/backend/Makefile,v
retrieving revision 1.104
diff -c -r1.104 Makefile
*** src/backend/Makefile 1 Aug 2004 18:07:42 -0000 1.104
--- src/backend/Makefile 30 Sep 2004 08:06:21 -0000
***************
*** 35,41 ****
ifneq ($(PORTNAME), win32)
postgres: $(OBJS)
! $(CC) $(CFLAGS) $(LDFLAGS) $(export_dynamic) $^ $(LIBS) -o $@
endif
endif
--- 35,41 ----
ifneq ($(PORTNAME), win32)
postgres: $(OBJS)
! $(CC) $(CFLAGS) $(LDFLAGS) $(export_dynamic) $^ /usr/lib/gcc/i486-linux/3.4.1/libgcov.a $(LIBS) -o $@
endif
endif
Index: src/bin/initdb/Makefile
===================================================================
RCS file: /home/neilc/private-cvsroot/pgsql-server/src/bin/initdb/Makefile,v
retrieving revision 1.44
diff -c -r1.44 Makefile
*** src/bin/initdb/Makefile 29 Aug 2004 04:13:01 -0000 1.44
--- src/bin/initdb/Makefile 30 Sep 2004 08:06:22 -0000
***************
*** 20,26 ****
all: submake-libpq submake-libpgport initdb
initdb: $(OBJS) $(libpq_builddir)/libpq.a
! $(CC) $(CFLAGS) $(OBJS) $(libpq) $(LDFLAGS) $(LIBS) -o [EMAIL PROTECTED](X)
dirmod.c: % : $(top_srcdir)/src/port/%
rm -f $@ && $(LN_S) $< .
--- 20,26 ----
all: submake-libpq submake-libpgport initdb
initdb: $(OBJS) $(libpq_builddir)/libpq.a
! $(CC) $(CFLAGS) $(OBJS) /usr/lib/gcc/i486-linux/3.4.1/libgcov.a $(libpq) $(LDFLAGS) $(LIBS) -o [EMAIL PROTECTED](X)
dirmod.c: % : $(top_srcdir)/src/port/%
rm -f $@ && $(LN_S) $< .
Index: src/bin/pg_config/Makefile
===================================================================
RCS file: /home/neilc/private-cvsroot/pgsql-server/src/bin/pg_config/Makefile,v
retrieving revision 1.8
diff -c -r1.8 Makefile
*** src/bin/pg_config/Makefile 1 Aug 2004 06:56:38 -0000 1.8
--- src/bin/pg_config/Makefile 30 Sep 2004 08:06:22 -0000
***************
*** 14,20 ****
rm -f $@ && $(LN_S) $< .
pg_config: $(OBJS)
! $(CC) $(CFLAGS) $(OBJS) $(LDFLAGS) $(LIBS) -o [EMAIL PROTECTED](X)
install: all installdirs
$(INSTALL_SCRIPT) pg_config$(X) $(DESTDIR)$(bindir)/pg_config$(X)
--- 14,20 ----
rm -f $@ && $(LN_S) $< .
pg_config: $(OBJS)
! $(CC) $(CFLAGS) $(OBJS) /usr/lib/gcc/i486-linux/3.4.1/libgcov.a $(LDFLAGS) $(LIBS) -o [EMAIL PROTECTED](X)
install: all installdirs
$(INSTALL_SCRIPT) pg_config$(X) $(DESTDIR)$(bindir)/pg_config$(X)
Index: src/bin/pg_ctl/Makefile
===================================================================
RCS file: /home/neilc/private-cvsroot/pgsql-server/src/bin/pg_ctl/Makefile,v
retrieving revision 1.15
diff -c -r1.15 Makefile
*** src/bin/pg_ctl/Makefile 29 Aug 2004 04:13:01 -0000 1.15
--- src/bin/pg_ctl/Makefile 30 Sep 2004 08:06:22 -0000
***************
*** 20,26 ****
all: submake-libpq submake-libpgport pg_ctl
pg_ctl: $(OBJS) $(libpq_builddir)/libpq.a
! $(CC) $(CFLAGS) $(OBJS) $(libpq) $(LDFLAGS) $(LIBS) -o [EMAIL PROTECTED](X)
exec.c: % : $(top_srcdir)/src/port/%
rm -f $@ && $(LN_S) $< .
--- 20,26 ----
all: submake-libpq submake-libpgport pg_ctl
pg_ctl: $(OBJS) $(libpq_builddir)/libpq.a
! $(CC) $(CFLAGS) $(OBJS) /usr/lib/gcc/i486-linux/3.4.1/libgcov.a $(libpq) $(LDFLAGS) $(LIBS) -o [EMAIL PROTECTED](X)
exec.c: % : $(top_srcdir)/src/port/%
rm -f $@ && $(LN_S) $< .
Index: src/bin/psql/Makefile
===================================================================
RCS file: /home/neilc/private-cvsroot/pgsql-server/src/bin/psql/Makefile,v
retrieving revision 1.47
diff -c -r1.47 Makefile
*** src/bin/psql/Makefile 24 May 2004 01:01:37 -0000 1.47
--- src/bin/psql/Makefile 30 Sep 2004 08:06:22 -0000
***************
*** 27,33 ****
all: submake-libpq submake-libpgport psql
psql: $(OBJS) $(libpq_builddir)/libpq.a
! $(CC) $(CFLAGS) $(OBJS) $(libpq) $(LDFLAGS) $(LIBS) -o [EMAIL PROTECTED](X)
exec.c: % : $(top_srcdir)/src/port/%
rm -f $@ && $(LN_S) $< .
--- 27,33 ----
all: submake-libpq submake-libpgport psql
psql: $(OBJS) $(libpq_builddir)/libpq.a
! $(CC) $(CFLAGS) $(OBJS) /usr/lib/gcc/i486-linux/3.4.1/libgcov.a $(libpq) $(LDFLAGS) $(LIBS) -o [EMAIL PROTECTED](X)
exec.c: % : $(top_srcdir)/src/port/%
rm -f $@ && $(LN_S) $< .
Index: src/bin/scripts/Makefile
===================================================================
RCS file: /home/neilc/private-cvsroot/pgsql-server/src/bin/scripts/Makefile,v
retrieving revision 1.28
diff -c -r1.28 Makefile
*** src/bin/scripts/Makefile 26 May 2004 17:24:05 -0000 1.28
--- src/bin/scripts/Makefile 30 Sep 2004 08:06:22 -0000
***************
*** 20,26 ****
all: submake-libpq submake-backend $(PROGRAMS)
%: %.o
! $(CC) $(CFLAGS) $^ $(libpq) $(LDFLAGS) $(LIBS) -o [EMAIL PROTECTED](X)
createdb: createdb.o common.o exec.o dumputils.o $(top_builddir)/src/backend/parser/keywords.o
createlang: createlang.o common.o exec.o print.o mbprint.o
--- 20,26 ----
all: submake-libpq submake-backend $(PROGRAMS)
%: %.o
! $(CC) $(CFLAGS) $^ /usr/lib/gcc/i486-linux/3.4.1/libgcov.a $(libpq) $(LDFLAGS) $(LIBS) -o [EMAIL PROTECTED](X)
createdb: createdb.o common.o exec.o dumputils.o $(top_builddir)/src/backend/parser/keywords.o
createlang: createlang.o common.o exec.o print.o mbprint.o
Index: src/test/regress/GNUmakefile
===================================================================
RCS file: /home/neilc/private-cvsroot/pgsql-server/src/test/regress/GNUmakefile,v
retrieving revision 1.47
diff -c -r1.47 GNUmakefile
*** src/test/regress/GNUmakefile 18 Jun 2004 06:14:25 -0000 1.47
--- src/test/regress/GNUmakefile 30 Sep 2004 08:06:22 -0000
***************
*** 127,142 ****
## Run tests
##
! check: all
-rm -rf ./testtablespace
mkdir ./testtablespace
$(SHELL) ./pg_regress --temp-install --top-builddir=$(top_builddir) --schedule=$(srcdir)/parallel_schedule --multibyte=$(MULTIBYTE) $(MAXCONNOPT)
! installcheck: all
! -rm -rf ./testtablespace
! mkdir ./testtablespace
$(SHELL) ./pg_regress --schedule=$(srcdir)/serial_schedule --multibyte=$(MULTIBYTE)
# old interfaces follow...
--- 127,144 ----
## Run tests
##
! setup-tablespace:
-rm -rf ./testtablespace
mkdir ./testtablespace
+
+ check: all setup-tablespace
$(SHELL) ./pg_regress --temp-install --top-builddir=$(top_builddir) --schedule=$(srcdir)/parallel_schedule --multibyte=$(MULTIBYTE) $(MAXCONNOPT)
! installcheck: all setup-tablespace
$(SHELL) ./pg_regress --schedule=$(srcdir)/serial_schedule --multibyte=$(MULTIBYTE)
+ singlecheck: all setup-tablespace
+ $(SHELL) ./pg_regress --temp-install --top-builddir=$(top_builddir) --schedule=$(srcdir)/serial_schedule --multibyte=$(MULTIBYTE)
# old interfaces follow...
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
