Today the following logic is in-place for loading spec files:
* Discover <startfiles>/specs
** If available, load it
** Else generate builtin specs (equivalent of -dumpspecs)
** `%include`, `%include_noerr`, `%rename` not allowed
* If `-specs=<file>` passed, load them in order
** `%include`, `%include_noerr`, `%rename` not allowed
This asymmetry results in very difficult choices, and experience when
one tries to modify default spec files, and does not want to leak them
via build-systems, or wants to ensure they are always in-effect. It
leads to distributors rebuilding toolchains just to update a trivial
single `--with-specs` configuration time option, as performing
`-dumpspecs`, patching that, and shipping alternative .specs like that
is cumbersome and error-prone.
Also, having more flexible specs that can be loaded by default, would
help producing context-aware build-flags whilst ensuring working
builds in other contexts.
For example consider an ELF Package Metadata notes spec file
```
*link:
+ --package-metadata={\"type\":\"rpm\",\"name\":\"%:getenv(RPM_PACKAGE_NAME
\",\"version\":\"%:getenv(RPM_PACKAGE_VERSION -%:getenv(RPM_PACKAGE_RELEASE
\",\"architecture\":\"%:getenv(RPM_ARCH \",\"osCpe\":\"@OSCPE@\"}))))
```
If specified in CFLAGS to build systems as
`-specs=redhat-package-notes`, it can leak from build environment into
the package-config files and extensions build systems. Then later when
end-users build code outside of RPM build environment would lead to
errors.
See this bug report, that states that a different implementation of
similar flags got enabled as default LDFLAGS, they leaked into another
language compiled extensions ecosystems, resulting in build-failures
or incorrect-context-unaware metadata. See
[RHB#2043092](https://bugzilla.redhat.com/show_bug.cgi?id=2043092)
Wouldn't it be nice if one could instead specify in the <startsfiles>/specs
```
%include_noerr </run/rpm/active-build-package-note.spec>
```
With expectation that default built-in specs are still loaded as
usual, and only then the `specs` file processed, with ability to use
`%include_noerr`, and dynamically inject build context aware specs.
For example Dak/Koji/OBS/Launchpad can then generate highly context
aware build flags, to set -march, build notes, hardening options to
truly take effect above all, despite all the build-systems in their
way. But even simple things like '-g0' to actually not even generate
debug symbols when not needed, or inverse force generating them to
then split them into debug symbols. A simple change like this would
universally speed up archive test rebuilds.
Whilst ELF package notes are used in this example, other useful things
can be injected like these. For example, distribution rebuilds for
various -march levels, OpenSSF hardening guide flags, custom toolchain
plugins, etc.
Anecdotally, many distributions rebuild GCC multiple times just to
change default mtune/march alone or set a single "--with-specs=". Such
rebuilds could be avoided and made cheaper to implement by shipping
multiple packages with `specs.overlay` that does just that.
Further more such behavior would actually match the documentation of
the [Spec files] and all existing guides on how to do SPECS. I.e. "use
%rename, redefine your command, or use + to append to existing one"
and so on.
[Spec files]: https://gcc.gnu.org/onlinedocs/gcc/Spec-Files.html
The current practice of prohibiting '%include_noerr' in
<startfiles>/specs, and requiring it to be a fully flat one (output of
-dumpspecs) is very restrictive to the point of not being used at all.
I would like to propose a couple of design choices to address above
limitations, either with or without breaking existing behaviour, and
also update documentation to specify all of this to prevent pit falls.
Just before loading user specified `-specs=<file>`, attempt to lookup
`<startfiles>/specs.overlay`, and load it with "%include" commands
allowed and pretend it was user specified one, as the first
"-specs=<file>".
Given <specs> file is an existing internal file api, that has been
driven to the ground of being empty, and really to modify (whilst
maintaining all features compatibility with -dumpspecs
equivalent). Modify the current load behavior to this:
1) Always load built-in generated specs
2) Then load <startfiles>/specs if found, relax permissions and allow
it to use "%include" commands, potentially move it closer to when user
specs are loaded.
Impact of behaviour change is small but not zero. By default specs
does not exist, thus default behaviour remains the same.
All existing <startfiles>/specs that are full equivalents of
`-dumpspecs` will continue to result in identical configuration.
Incomplete `-dumpspecs` which either didn't explicitly delete
spec_name by setting it to empty, may potentially have different
behavior if the default '-dumpspecs' sets them to something.
If this proposal is accepted, there is no need to create a brand-new
file api "specs.overlay". But behavior of <startfiles>/specs will be
widely different across GCC releases, which may cause confusion if
this changes makes "specs" files popular again.
Today in the [Spec files] documentation it is not mentioned that when
creating a custom Spec file, and testing it with `-specs` command,
will result in widely different behavior if moved to
<startfiles>/specs location. As right now it is not mentioned that
'%include" commands are blocked, and that the built-in specs are not
loaded, meaning '%rename' and '+' append functionality effectively
cannot be used.
But only want to update documentation for all supported gcc series,
once there is consensus on any of the above proposals.
[Spec files]: https://gcc.gnu.org/onlinedocs/gcc/Spec-Files.html
[Clang config files] passed on commandline, work identical to the
automatically discovered ones. On commandline they are passed with
`--config`.
The configuration files language is more simple one. It is a flat text
file of comments, or literal command-line options appended. There is
one `@file` directive to include another file.
Automatic configuration file discovery uses compiler name and target
arch to discover and load a matching configuration file. For example
`x86_64-pc-linux-gnu-clang-g++`, tries to load
`x86_64-pc-linux-gnu-clang++.cfg`, or
`x86_64-pc-linux-gnu-clang-g++.cfg`, or `clang++.cfg`
[Clang config files]:
https://clang.llvm.org/docs/UsersManual.html#configuration-files
In addition to toolchain internal locations (typically under
`/usr/lib`), system-wide locations are also typically searched by
Clang, i.e. `/etc/clang-18`. An equivalent `/etc/gcc` spec files
discovery location does not exist today, but also not sure if it is
needed. Distributions can use symlinks, and dynamically generated spec
files with relevant includes as needed using their usual means to
manage such things (update-alternatives, dpkg-reconfigure, etc).
Equivalent to `--config` in GCC Spec file terms would be a `.cfg` file
that automatically injects all strings into `*self_spec: +` lines, and
converts `@file` includes into `#include_noerr` commands.
Shockingly enough I did set <startfiles>/specs to
```
*self_spec:
+ %{!O:%{!O1:%{!O2:%{!O3:%{!O0:%{!Os:%{!0fast:%{!0g:%{!0z:-O2}}}}}}}}}
-fhardened %{!Wall:%{!Wno-all:-Wall}} %{!Wformat=0:-Wformat=2}
%{!Wconversion:%{!Wno-conversion:-Wconversion}}
%{!Wimplicit-fallthrough=0:-Wimplicit-fallthrough=3}
-Wl,--as-needed,-O1,--sort-common,-z,noexecstack,-z,relro,-z,now
-fno-delete-null-pointer-checks -fno-strict-overflow -fno-strict-aliasing
-fno-omit-frame-pointer -mno-omit-leaf-frame-pointer
-mDO_NOT_COPY_AND_PASTE_INTO_SPECS
```
and that actually manages to compile a huge number of C/C++ software
with common build systems, despite removing linking command and thus
removing `-lgcc_s` from the link command and actually miss-building
lots of things. (Intentionally added invalid arg to prevent others
making the same mistake with a blind copy & paste).
Dimitri John Ledkov (2):
specs: load specs.overlay with all commands enabled
specs: always generate built-in specs, before reading "specs" file
gcc/gcc.cc | 23 +++++++++++++++++------
1 file changed, 17 insertions(+), 6 deletions(-)
--
2.43.0
