Re: Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 07:33 +, devel-requ...@lists.fedoraproject.org wrote: > -- > > Date: Fri, 5 Jun 2020 09:16:21 +0200 > From: Frantisek Zatloukal > Subject: Re: Fedora 33 System-Wide Change proposal: CompilerPolicy > Change > To: Development discussions related to Fedora > > Message-ID: > > Content-Type: multipart/alternative; > boundary="64befd05a7510a78" > > --64befd05a7510a78 > Content-Type: text/plain; charset="UTF-8" > > On Fri, Jun 5, 2020 at 9:11 AM Igor Raits > wrote: > > > Also we probably should mention that -fstack-clash-protection is not > > available in clang, so in theory binaries can be less secure due to > > that. > > > > This seems to be worked on as per https://reviews.llvm.org/D68720?id=224102 > (on > x86, I am not sure on what arches GCC supports -fstack-clash-protection ). stack-clash-protection is under active development in Clang/LLVM. Serge has done the x86_64 implementation and engineers from IBM are currently working on implementations for Power and Z series. I'm also in contact with ARM engineers to try and get AArch64 covered as well. Realistically I expect Power and Z to land in LLVM 11 and AArch64 in LLVM 12. jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 14:38 +, devel-requ...@lists.fedoraproject.org wrote: > > Date: Fri, 5 Jun 2020 15:23:57 +0100 > From: "Richard W.M. Jones" > Subject: Re: Fedora 33 System-Wide Change proposal: CompilerPolicy > Change > To: Development discussions related to Fedora > > Cc: Vitaly Zaitsev > Message-ID: <20200605142357.gq15...@redhat.com> > Content-Type: text/plain; charset=us-ascii > > On Fri, Jun 05, 2020 at 08:27:13AM -0400, Neal Gompa wrote: > > On Fri, Jun 5, 2020 at 8:26 AM Neal Gompa wrote: > > > On Fri, Jun 5, 2020 at 6:47 AM Vitaly Zaitsev via devel > > > wrote: > > > > On 05.06.2020 09:52, Kevin Kofler wrote: > > > > > I am opposed to this change. Chromium and Firefox build fine with GCC. > > > > > I > > > > > think that a distribution should be built with a consistent toolchain > > > > > wherever possible. > > > > > > > > Clang is much better than GCC nowadays. It has better architecture, > > > > support lots of optimizations and analyzers. > > > > > > > > GCC is a legacy compiler. It should be completely replaced by Clang in > > > > the nearest future. > > > > > > > > > > Having worked in a distribution that uses Clang by default > > > (OpenMandriva), I can say that this is *not true*. Switching from GCC > > > to Clang cost OpenMandriva a lot of performance. It also cost them a lot > > > of > > > security hardening at the compiler level. GCC-built binaries are still > > > better, and remain better as long as people are continually using and > > > developing for it. > > > > > > This change appears to largely be driven by the maintainers of web > > > browser packages that upstream have no GCC validation and it has to be > > > done in Fedora downstream. I know Chromium is a lost cause (Google > > > couldn't possibly care any less than they do now, especially since > > > they don't even care about Python 2 being EOL), but has anyone talked > > > to Mozilla about introducing GCC-based CI for Firefox code? I assume > > > they have a CI infrastructure that's relatively pluggable. > > > > > > Note that having stuff mix compilers is also a bad idea because LTO is > > > compatible across the two compilers. If you want to use LTO, you need > > > to use the same compiler across the chain, or stuff will break. > > > > > > > Yay thinkos... I mean that LTO is *not* compatible across the two compilers. > > Does this change conflict with: > > https://fedoraproject.org/wiki/LTOByDefault Not it does not. The LTO bytecode streams do not survive past any given package build. ie, they are used within the build, then discarded. They're not supposed to show up in any installed libraries. Thus the fact that the two compilers use totally different LTO representations (and always will) is a non-issue here. Jeff ps. THe reason we discard the LTO data is because both compilers consider the LTO bytecodes unstable between major releases. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 12:27 +, devel-requ...@lists.fedoraproject.org wrote: > > On Fri, 2020-06-05 at 10:14 +0100, Jonathan Wakely wrote: > > On 04/06/20 16:30 -0400, Ben Cotton wrote: > > > https://fedoraproject.org/wiki/Changes/CompilerPolicy > > [snip] > > > == Documentation == > > > Several years ago Red Hat's tools team championed for Fedora policy > > > to strongly > > > discourage the use of LLVM/Clang for package building. Exceptions > > > were made for > > > packages that could only be built with Clang/LLVM: > > > > > > https://docs.fedoraproject.org/en-US/packaging-guidelines/#compiler > > > > > > > > > At that point in history Red Hat had no Clang/LLVM engineers or > > > expertise. In > > > fact, the LLVM packages were actually maintained by an engineer on > > > the desktop > > > team (they had a hard requirement for llvm-pipe, so they got to own > > > the > > > Clang/LLVM bits). The policy essentially was a risk management > > > strategy for > > > Fedora. > > > > > > Times have changed and as a result we should revisit that Fedora > > > policy. > > > > > > The Red Hat tools team believes that LLVM/Clang and GCC should be > > > considered equals from > > > a Fedora policy standpoint. Selection of one toolchain over the > > > should should be > > > > s/should should be/other should be/ > > > > > driven by the upstream project's preferences not by Fedora specific > > > policy. > > > > I think this needs to be clear that we're talking about the compiler > > here, not the C++ standard library. Fedora has no libc++ expertise > > I'm > > aware of. > > > > I think we want to be a lot more cautious about using libc++, because > > it would potentially require large parts of the C++ stack to be built > > twice and installed in parallel (think Python 2 and Python 3). For > > example if your package depends on Boost and you want to use libc++ > > then you need Boost rebuilt. Similarly for any C++ library, Qt, > > gtkmm, > > etc. etc. > > > > And I do not intend to offer C++ support for people who decide to use > > Clang + libc++ but don't try to resolve their own build failures :) > > > > > > > What that means in practice is that if the project upstream prefers > > > Clang/LLVM, > > > then Fedora should in turn be using Clang/LLVM to build those > > > packages. As a > > > concrete example, let's consider Chromium. > > > > > > Chromium upstream has been building with Clang/LLVM for several > > > years. > > > Yet policy > > > has forced Fedora package owners to shoulder a significant burden > > > to make it > > > build with GCC. Furthermore, Fedora does not get as much benefit > > > at it could by > > > forcing Chromium to be built with GCC since most other instances > > > are built with > > > Clang/LLVM. > > > > > > By changing policy Fedora package maintainers no longer have to > > > waste > > > time trying > > > to make Chromium build/work with GCC and Fedora gains additional > > > "many eyes" and > > > "many users" benefits by relying on the same tools to build Chrome > > > as the > > > upstream developers and other distributions. > > > > > > Additionally, if an upstream project currently uses GCC, but > > > switches to > > > Clang/LLVM (or vice-versa), then the package in Fedora should > > > switch > > > in a similar > > > manner. The only policy restriction should be that the compiler > > > must > > > exist in Fedora. > > > > If upstream supports both compilers, that's probably not a good > > reason > > to change anything in Fedora. > > > > > In some ways this means there is no "default" compiler for Fedora. > > > The default > > > is whatever the upstream project supports/recommends. However, > > > there are > > > probably many packages with upstreams that are ambivalent about > > > their compiler > > > choice. For those packages I would recommend we keep the status > > > quo at the > > > current time. For a package with a dead upstream, the Fedora > > > packager should be > > > able to select the compiler they want to use for the package. > > > > I would prefer the policy to have a stronger preference for GCC when > > there is no good reason to change. Currently it's worded like keeping > > the status quo is Jeff's personal opinion. I would like it to be > > policy. > > > > i.e. where the current policy is that you *must* use GCC, unless > > granted a specific exception, I would like it to be that you *should* > > use GCC unless there's a good reason not to. > > > > "Upstream only builds+tests with Clang and using GCC requires lots of > > work from the Fedora maintainer to fix problems that upstream don't > > care about" is a good reason to use Clang. > > > > "I've heard the cool kids use Clang, so I changed my package to use > > it, but I don't know what these compiler errors mean" is not a good > > reason to switch if it just makes work for other people. > > > > When there is a real advantage to using a particular compiler, I do > > think it's good that packager
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 12:27 +, devel-requ...@lists.fedoraproject.org wrote: > > Date: Fri, 5 Jun 2020 08:26:04 -0400 > From: Neal Gompa > Subject: Re: Fedora 33 System-Wide Change proposal: CompilerPolicy > Change > To: Development discussions related to Fedora > > Cc: Vitaly Zaitsev > Message-ID: > > Content-Type: text/plain; charset="UTF-8" > > On Fri, Jun 5, 2020 at 6:47 AM Vitaly Zaitsev via devel > wrote: > > On 05.06.2020 09:52, Kevin Kofler wrote: > > > I am opposed to this change. Chromium and Firefox build fine with GCC. I > > > think that a distribution should be built with a consistent toolchain > > > wherever possible. > > > > Clang is much better than GCC nowadays. It has better architecture, > > support lots of optimizations and analyzers. > > > > GCC is a legacy compiler. It should be completely replaced by Clang in > > the nearest future. > > > > Having worked in a distribution that uses Clang by default > (OpenMandriva), I can say that this is *not true*. Switching from GCC > to Clang cost OpenMandriva a lot of performance. It also cost them a lot of > security hardening at the compiler level. GCC-built binaries are still > better, and remain better as long as people are continually using and > developing for it. I'm not suggesting switching the default. I'm suggesting the compiler choice be made by the upstream projects. Some prefer LLVM, others prefer GCC. Fedora should get out of the way and use the same tools that the upstream projects are using. > > This change appears to largely be driven by the maintainers of web > browser packages that upstream have no GCC validation and it has to be > done in Fedora downstream. I know Chromium is a lost cause (Google > couldn't possibly care any less than they do now, especially since > they don't even care about Python 2 being EOL), but has anyone talked > to Mozilla about introducing GCC-based CI for Firefox code? I assume > they have a CI infrastructure that's relatively pluggable. There's been a ton of discussions with the upstream Firefox and Chromium communitities. While we were able to get Firefox to continue to support GCC, it was exceedingly painful and ultimately the Firefox developers are primarily using Clang/LLVM. And note I've called out Chromium and Firefox simply because they're the ones I'm familiar with. There well could be other packages now and in the future that are jumping through hoops because of current Fedora policy. > > Note that having stuff mix compilers is also a bad idea because LTO is > compatible across the two compilers. If you want to use LTO, you need > to use the same compiler across the chain, or stuff will break. Simply not true. See my earlier message. The LTO streams live for the duration of a package build and are then discarded because they are not considered stable release-to-release by the compiler. > > I would rather see us still *strongly prefer* GCC rather than allowing > this to be freely changeable at a whim. I disagree. I strongly believe toolchain decisions need to be made at the upstream project level. I say this as someone who has been a GCC developer for 30 years, who is an active GCC steering committee member and who has led GCC development in the past. It's time to remove the anti Clang/LLVM bias in Fedora. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 15:56 +, devel-requ...@lists.fedoraproject.org wrote: > Send devel mailing list submissions to > devel@lists.fedoraproject.org > > To subscribe or unsubscribe via email, send a message with subject or > body 'help' to > devel-requ...@lists.fedoraproject.org > > You can reach the person managing the list at > devel-ow...@lists.fedoraproject.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of devel digest..." > > Today's Topics: > >1. Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change > (Mark Wielaard) >2. Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change > (Josh Boyer) >3. Re: Fedora 33 System-Wide Change proposal: swap on zram > (Igor Raits) >4. Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change > (Jeff Law) > > > -- > > Date: Fri, 05 Jun 2020 17:47:51 +0200 > From: Mark Wielaard > Subject: Re: Fedora 33 System-Wide Change proposal: CompilerPolicy > Change > To: l...@redhat.com, Development discussions related to Fedora > > Message-ID: > <3c8151c120d43cfa33d1dae11ba1c2b4f6e16be9.ca...@fedoraproject.org> > Content-Type: text/plain; charset="UTF-8" > > Hi, > > On Fri, 2020-06-05 at 09:25 -0600, Jeff Law wrote: > > The LTO bytecode streams do not survive past any given package build. ie, > > they > > are used within the build, then discarded. They're not supposed to show up > > in > > any installed libraries. > > > > Thus the fact that the two compilers use totally different LTO > > representations > > (and always will) is a non-issue here. > > One issue I am concerned about here is debuginfo quality. GCC produced > pretty bad debuginfo with LTO in older versions. The EarlyDebug work > did make it usable. And we needed some work on the DARF consumers to > correctly process GCC 10 LTO produced DWARF (I actually have two small > patches for elfutils pending so it correctly parses the cross CU- > references GCC 10 LTO now produces). > > I don't know if anybody did any analysis on llvm LTO produced > debuginfo. In the past it was observed that llvm doesn't do VTA (Var > Tracking Assignments) that is default in GCC. And VTA is really > important for debugging and performance/tracing tools like systemtap. > So even without LTO I am concerned about the observability of llvm > generated binaries. And these are reasonable concerns (even more so in an LTO world and I expect I'll be spending most of my summer poking at debuginfo stuff for GCC LTO :-). IMHO this is one of the issues an upstream project needs to evaluate for their toolchain choice -- no different than performance, diagnostics, sanitizer support, plug-in support, etc etc. The level of observability needed by any particular upstream project can vary. ie, the kernel may well have different needs than the Ruby interpreter. I have not done any evaluation of LLVM's debug info. I would not be surprised if GCC is generally better, particularly with Alex's work over the last couple years. But again, I think the upstream project is in the best position to determine how to balance this against the other factors in toolchain selection. Jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 10:51 +, devel-requ...@lists.fedoraproject.org wrote: > Send devel mailing list submissions to > > Date: Fri, 05 Jun 2020 12:42:03 +0200 > From: Florian Weimer > Subject: Re: Fedora 33 System-Wide Change proposal: CompilerPolicy > Change > To: Igor Raits > Cc: Development discussions related to Fedora > > Message-ID: <87pnadu744@oldenburg2.str.redhat.com> > Content-Type: text/plain > > * Igor Raits: > > > From what I see, GCC supports it on x86, x86_64, s390x, riscv64, > > ppc64le. So this just does not include ARM / AArch64 from Fedora > > architectures. > > GCC has aarch64 support for stack-clash-protection, but it only works > well with 64K pages (otherwise detection is not reliable). This is due > to a choice by the target maintainers I do not understand. At least it > does not break anything. Correct. > > I don't know the state on armhfp. It used the generic GCC > implementation in the past, which we considered too buggy to enable. I don't think anyone ever stepped up to implement stack-clash protection on 32bit ARM. I couldn't justify spending the time on it when stack-clash hit because 32bit ARM isn't supported by RHEL. And after doing x86, ppc, s390 and aarch64 for GCC I was burnt out as hell. I expect LLVM 11 should have stack-clash protection for x86, ppc and s390. I'm working with ARM to find resources to do an AArch64 implementation with the goal of landing it for LLVM 12. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 10:51 +, devel-requ...@lists.fedoraproject.org wrote: > > Date: Fri, 5 Jun 2020 12:42:30 +0200 > From: Vitaly Zaitsev > Subject: Re: Fedora 33 System-Wide Change proposal: CompilerPolicy > Change > To: devel@lists.fedoraproject.org > Message-ID: <08a3d005-da6e-248f-4f70-611a05414...@easycoding.org> > Content-Type: text/plain; charset=utf-8 > > On 04.06.2020 22:30, Ben Cotton wrote: > > Fedora has historically forced packages to build with GCC unless the > > upstream project for the package only supported Clang/LLVM. This > > change proposal replaces that policy with one where compiler selection > > for Fedora follows the package's upstream preferences. > > +1 for this change. > > Also RHBZ#1559007 need to be fixed as well. I'm very tired of patching > Fedora build flags in order to remove unsupported by Clang options. I don't think Tom (as the LLVM lead within Red Hat) nor I would object to having a way to do this more cleanly. While Clang/LLVM and GCC agree on many options, I can certainly envision the need to have toolchain specific flags and ways to change them. > > -- > Sincerely, > Vitaly Zaitsev (vit...@easycoding.org) > > -- > > Date: Fri, 5 Jun 2020 12:46:12 +0200 > From: Vitaly Zaitsev > Subject: Re: Fedora 33 System-Wide Change proposal: CompilerPolicy > Change > To: devel@lists.fedoraproject.org > Message-ID: > Content-Type: text/plain; charset=utf-8 > > On 05.06.2020 09:52, Kevin Kofler wrote: > > I am opposed to this change. Chromium and Firefox build fine with GCC. I > > think that a distribution should be built with a consistent toolchain > > wherever possible. > > Clang is much better than GCC nowadays. It has better architecture, > support lots of optimizations and analyzers. > > GCC is a legacy compiler. It should be completely replaced by Clang in > the nearest future. I would disagree strongly with both of those statements, but neither your opinion nor mine on these statements have any relevance to the change proposal. THe change proposal is simple. Compiler selection should flow from the upstream project. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 16:23 +, devel-requ...@lists.fedoraproject.org wrote: > Date: Fri, 5 Jun 2020 11:15:39 -0500 > From: Steven Munroe > Subject: Re: Fedora 33 System-Wide Change proposal: CompilerPolicy > Change > To: devel@lists.fedoraproject.org > Message-ID: > > Content-Type: text/plain; charset="UTF-8" > > I would also add that Clang/LLVM is missing some of the newer C > language revisions at least for the pppc64le target. > > Both IEEE/ISO _Float128 and _Decimalxx support is missing. Ie the > type is not supported or if supported basic arithmetic and math.h > support is missing. Also finding bugs for in-line assembly and missing > constraints needed to work around the missing language features. > > Also Clang's poor support for constant folding makes using __int128 > (and vector __int128) harder than it needs to be. > > This is a significant impact for enabling my project (PVECLIB) for > Clang. As-is a number of project features have been disabled for > Clang. Clearly your upstream project is still using GCC then and as such the Fedora package would continue to use GCC. We're not changing the default here. We're just removing the anti Clang/LLVM policy and allowing upstreams to select the compiler that best suits their needs. Clearly Clang/LLVM is not the right choice for your project. > > I think Clang needs more time to cook. I'd respectfully disagree. There are certain features that GCC supports that Clang does not and vice-versa. But broadly they are comparable. What this means is some projects that are using bleeding edge features may have a hard need for one toolchain or the other. And the proposal I've made accounts for that by allowing the upstream project to select the compiler. In your case it would be GCC. For others it could well be Clang/LLVM. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: devel Digest, Vol 196, Issue 58
On Fri, 2020-06-05 at 10:22 +, devel-requ...@lists.fedoraproject.org wrote: > Date: Fri, 5 Jun 2020 11:19:54 +0200 > > From: Jakub Jelinek > > Subject: Re: Fedora 33 System-Wide Change proposal: CompilerPolicy > > Change > > To: Development discussions related to Fedora > > > > Message-ID: <20200605091954.GH8462@tucnak> > > Content-Type: text/plain; charset=us-ascii > > > > On Fri, Jun 05, 2020 at 09:52:09AM +0200, Kevin Kofler wrote: > > > I do not see why we should allow yet another special case for Firefox, nor > > why we should let random packages make their own choice of compiler and > > risk > > running into hidden binary incompatibilities. We have a system compiler for > > a reason. > > > I'll note that there are some even not really hidden binary > > incompatibilities, where LLVM diverges from the psABI for years, it has been > > reported and nothing has been changed. > > So if a library is built with clang/LLVM and used by GCC built package or > > vice versa, one might very well run into those (this is e.g. about passing > > std::byte or other scoped enums with char/short underlying type by value, or > > in some cases even about passing char/short arguments). > > And of course unknown ABI bugs on both sides. As we both know, GCC has had ABI bugs as well. Both compilers strive to be ABI compatible with each other and we should continue to work together to find and address such issues. SImilarly both compilers are going to have codegen issues, or rejects-valid-code bugs. Ultimately they're just bugs and I don't see that one toolchain or the other is inherently better than the other, particularly WRT ABI issues. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 10:22 +, devel-requ...@lists.fedoraproject.org wrote: > Date: Fri, 5 Jun 2020 11:42:36 +0200 > > From: Vít Ondruch > > Subject: Re: Fedora 33 System-Wide Change proposal: CompilerPolicy > > Change > > To: devel@lists.fedoraproject.org > > Message-ID: <0f69dde1-839e-e0c3-26d8-90af2c5d6...@redhat.com> > > Content-Type: text/plain; charset=utf-8 > > > > > > Dne 05. 06. 20 v 9:52 Kevin Kofler napsal(a): > > > Ben Cotton wrote: > > > == Summary == > > > Fedora has historically forced packages to build with GCC unless the > > > upstream project for the package only supported Clang/LLVM. This > > > change proposal replaces that policy with one where compiler selection > > > for Fedora follows the package's upstream preferences. > > > == Owner == > > > * Name: Jeff Law > > > * Email: l...@redhat.com > > I am opposed to this change. Chromium and Firefox build fine with GCC. I > > think that a distribution should be built with a consistent toolchain > > wherever possible. > > Last I checked, there were several reasons why GCC is preferred over > > Clang/LLVM in Fedora. And if that should ever change (or have changed > > already), then switching the systemwide default (reversing the rules, i.e., > > using GCC only for those packages that do not build with Clang) should be > > envisioned. But as far as I know, that is not the case at this time, > > considering runtime performance, security features, etc. > > I do not see why we should allow yet another special case for Firefox, nor > > why we should let random packages make their own choice of compiler and > > risk > > running into hidden binary incompatibilities. We have a system compiler for > > a reason. > > > > > Just FTR, there are technical (and security) reasons why we might > > consider switching Ruby from GCC to Clang in the future: > > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1721553 Absolutely correct. While this bug contains the first public hint of the proposal we're now discussing, we'd been discussing these issues within the toolchain team for some time before that bug raised its ugly head. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 14:57 +, devel-requ...@lists.fedoraproject.org wrote: > Date: Fri, 5 Jun 2020 07:56:57 -0700 > > From: Tom Stellard > > Subject: Re: Fedora 33 System-Wide Change proposal: CompilerPolicy > > Change > > To: Development discussions related to Fedora > > , Igor Raits > > > > Message-ID: <59de2b7b-e7a3-15c3-0c72-475ef4adb...@redhat.com> > > Content-Type: text/plain; charset=utf-8 > > > > On 06/05/2020 12:09 AM, Igor Raits wrote: > > > On Thu, 2020-06-04 at 16:30 -0400, Ben Cotton wrote: > > > https://fedoraproject.org/wiki/Changes/CompilerPolicy > > > == Summary == > > > Fedora has historically forced packages to build with GCC unless the > > > upstream project for the package only supported Clang/LLVM. This > > > change proposal replaces that policy with one where compiler > > > selection > > > for Fedora follows the package's upstream preferences. > > Sadly some upstreams insist on clang just because they like it more, > > without any technical reason. The question that comes to my mind: > > Should we still try to convince upstreams to use GCC in such cases or > > not? > > Also does this mean that Clang is now fully supported compiler by full- > > time working people @ Red Hat in toolchains team that are also > > contributing to upstream? Just curious if we will be able to "fully > > support" people when we find bugs in the compiler. > > And also, does it mean that we will be following same pattern as with > > GCC to test pre-release versions in rawhide, do the mass rebuild and so > > on? > > > We have been packaging -rc1 release candidates of major Clang/LLVM releases > in rawhide and plan to continue doing this. It's also possible we could > package even early snapshots if this fits better with the Fedora release > schedule, but this is not something we have done before. FWIW, I expect to be poking at this problem from the GCC side in the fall. The way we currently handle GCC in the even numbered Fedora release is (IMHO) insane. Carlos and his team have shown an ability to drop in snapshots of glibc development sources weekly without introducing huge instability. While I'm not shooting for weekly drops of GCC bits into rawhide, I do think we should be dropping in development snapshots to rawhide earlier and more often than we currently do. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 14:11 +, devel-requ...@lists.fedoraproject.org wrote: > Date: Fri, 5 Jun 2020 14:16:58 +0100 > > From: Ian McInerney > > Subject: Re: Fedora 33 System-Wide Change proposal: CompilerPolicy > > Change > > To: Development discussions related to Fedora > > > > Message-ID: > > > > Content-Type: multipart/alternative; > > boundary="1b547505a756148e" > > > > --1b547505a756148e > > Content-Type: text/plain; charset="UTF-8" > > > > On Fri, Jun 5, 2020 at 1:51 PM Stephan Bergmann wrote: > > > > > On 05/06/2020 10:15, Frantisek Zatloukal wrote: > > > [...] Apart from > > > browsers, LibreOffice is going to use LLVM/Clang from Release 7.0 too, > > > so that would potentially be another added work to LibreOffice packagers > > > in the future. > > Just to clarify, upstream LibreOffice supports both GCC and Clang on > > Linux equally well since ~forever, and there is no change coming for LO > > 7.0 that I'm aware of. Upstream Linux binaries are built with GCC, FWIW. > > > I think what they are referring to is that there was a patch [1] a few > > months ago making Clang *preferred* for building the LibreOffice > > rendering code in response to a slow-performance bug [2]. This highlights what I consider probably the most interesting case. Upstream has a preference for Liberoffice (Clang/LLVM), but supports both Clang/LLVM and GCC. In this case I think Fedora policy should be to follow the upstream preference. Jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 07:58 +, devel-requ...@lists.fedoraproject.org wrote: > > Date: Fri, 05 Jun 2020 09:52:09 +0200 > From: Kevin Kofler > Subject: Re: Fedora 33 System-Wide Change proposal: CompilerPolicy > Change > To: devel@lists.fedoraproject.org > Message-ID: > Content-Type: text/plain; charset="ISO-8859-1" > > Ben Cotton wrote: > > == Summary == > > Fedora has historically forced packages to build with GCC unless the > > upstream project for the package only supported Clang/LLVM. This > > change proposal replaces that policy with one where compiler selection > > for Fedora follows the package's upstream preferences. > > > > == Owner == > > * Name: Jeff Law > > * Email: l...@redhat.com > > I am opposed to this change. Chromium and Firefox build fine with GCC. I > think that a distribution should be built with a consistent toolchain > wherever possible. But that's because engineers spend the time to make that work. It's not a trivial task. And while there is some value to that work (it exposes non- portable code in the package or bugs in GCC), I believe Fedora developers' time is better spent elsewhere. > > Last I checked, there were several reasons why GCC is preferred over > Clang/LLVM in Fedora. And if that should ever change (or have changed > already), then switching the systemwide default (reversing the rules, i.e., > using GCC only for those packages that do not build with Clang) should be > envisioned. But as far as I know, that is not the case at this time, > considering runtime performance, security features, etc. The reasons GCC was preferred was because Red Hat's tools team had no Clang/LLVM expertise and thus had no ability to help address issues that might arise. Furthermore when the policy was put into place Clang/LLVM was "only" ~10 years old and the tools team considered it still maturing. It was risk mitigation, pure and simple. Red Hat has slowly staffed up in the Clang/LLVM space which addresses the first problem. And naturally Clang/LLVM has continued to mature addressing the second problem and it has reached the point where we are comfortable with Clang/LLVM as a co-equal toolchain. And if I look at development community trajectories I have no concerns about the viability of Clang/LLVM going forward. We are not proposing a change in the default compiler. Only to allow the compiler selection to be driven by the upstream project. > > I do not see why we should allow yet another special case for Firefox, nor > why we should let random packages make their own choice of compiler and risk > running into hidden binary incompatibilities. We have a system compiler for > a reason. This is not a special case for Firefox or any other package. It is acknowledgment that the toolchains are comparable and each has its own strengths and weaknesses which in turn may cause an upstream project to prefer one over the other. Are the risks, certainly. But that's already the case because packages that are building with Clang/LLVM (they do exist) have to interoperate with libraries that are compiled with GCC. Similarly Clang/LLVM has to work with headers that are supplied by GCC (libstdc++ in particular). I've been doing GCC compiler development for 30 years and have had the "joy" of trying to make GCC compatible with compilers from HP, IBM, Sun, MIPS, as well as various embedded compilers through the years, often without documentation. The ABI compatibility situation between Clang/LLVM and GCC is *far* better than any I've dealt with through the decades and I strongly believe both projects have a vested interest in maintaining ABI compatibility and fixing ABI bugs. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 19:23 +0200, Florian Weimer wrote: > * Jeff Law: > > > I'm not suggesting switching the default. I'm suggesting the compiler > > choice be made by the upstream projects. Some prefer LLVM, others > > prefer GCC. Fedora should get out of the way and use the same tools > > that the upstream projects are using. > > Do we know how many upstream projects actually recommend building with a > recent Clang upstream release? Chromium explicitly does not: Not really. I'm not in a great position to know the set of upstream projects that recommend Clang/LLVM. > > > Chromium ships a prebuilt clang binary. It's just upstream clang built > > at a known-good revision that we bump every two weeks or so. > > > > This is the only supported compiler for building Chromium. > > <https://chromium.googlesource.com/chromium/src/+/master/docs/clang.md> Yea, Tom mentioned this tidbit to me privately. > > Currently, it's this build: > > $ ./src/third_party/llvm-build/Release+Asserts/bin/clang++ --version > clang version 11.0.0 (https://github.com/llvm/llvm-project/ > a6ae333a0c23fc9b0783ca45e2676abac00c6723) > Target: x86_64-unknown-linux-gnu > Thread model: posix > InstalledDir: /home/test/chromium/./src/third_party/llvm- > build/Release+Asserts/bin > > So something pulled recently from the development branch, not a release > branch. > > In my experience, this is pretty typical: Upstreams leaning towards > Clang expect that you use their prebuilt compiler, not a system > compiler. I suspect you're right. > > Using Clang downstream may make things simpler for packagers, but a > system-supplied Clang compiler is rarely a preferred upstream choice > (not even on Macos, where there is no real choice anymore). I'm not sure making this distinction is that useful. If upstream prefers an pre- built clang, then ISTM we'd either be using that pre-built clang or our system clang. Forcing a packager to use GCC seems wrong in this case. > > On the other hand, there is demand from users (not just packagers) for > the LLVM-based toolchain, and increased use by Fedora packagers will > help us to improve the quality of Fedora's version of it. Fortunately, > for this effect, it does not matter *why* Fedora packagers choose Clang. Right. jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 19:31 +0200, Florian Weimer wrote: > * Ben Cotton: > > > https://fedoraproject.org/wiki/Changes/CompilerPolicy > > > > == Summary == > > Fedora has historically forced packages to build with GCC unless the > > upstream project for the package only supported Clang/LLVM. This > > change proposal replaces that policy with one where compiler selection > > for Fedora follows the package's upstream preferences. > > Jeff, would you please clarify in the proposal that libc++ is out of > scope (as already discussed)? Yes. I haven't replied to that part of the thread yet. But yes, libc++ is out of scope. It'd be a compatibility nightmare. > > What about compiler-rt? And lld? Any other LLVM subprojects that could > be relevant? I consider those out of scope as well :-) I think there's some desire to add lld into the alternatives mechanism (along side ld.bfd and ld.gold). But that's about it to the best of my knowledge. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 20:51 +0200, Florian Weimer wrote: > * Jeff Law: > > > As we both know, GCC has had ABI bugs as well. Both compilers strive > > to be ABI compatible with each other and we should continue to work > > together to find and address such issues. SImilarly both compilers > > are going to have codegen issues, or rejects-valid-code bugs. > > Ultimately they're just bugs and I don't see that one toolchain or the > > other is inherently better than the other, particularly WRT ABI > > issues. > > More problematic are not ABI bugs, but the cases where the ABI > divergence is a matter of opinion (more or less). Yes because these are much less likely to get fixed. THankfully these don't pop up nearly as often. > > I think we really should figure out what to do about the alignment of > _Atomic long long on 32-bit. GCC has 4, Clang has 8. Clang seems to be > correct here. This also has implications for the use of libatomic. Yes. I thought we bumped up that bug in the database so that it'd get some attention in the gcc-10 cycle. But I couldn't follow it myself, so I don't know what happened. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
> On Thu, 2020-06-04 at 16:30 -0400, Igor Raits wrote: > ... > > Sadly some upstreams insist on clang just because they like it more, > without any technical reason. The question that comes to my mind: > Should we still try to convince upstreams to use GCC in such cases or > not? It happens (choosing Clang because they like it) and while we can (and have) engaged upstreams on this topic and I suspect we will continue to do so in some cases. But in the end I think we still have to respect the upstream project's choices, even if it's just because they like Clang/LLVM more than GCC. > > Also does this mean that Clang is now fully supported compiler by full- > time working people @ Red Hat in toolchains team that are also > contributing to upstream? Just curious if we will be able to "fully > support" people when we find bugs in the compiler. Yes. Absolutely. Tom S & Serge G directly. Others in a more indirect capacity. > > And also, does it mean that we will be following same pattern as with > GCC to test pre-release versions in rawhide, do the mass rebuild and so > on? Some of that is already happening internally. Tom's got a tester which spins Fedora packages with Clang/LLVM. I'm not sure if he's trying to cover the whole distro, but it is running regularly (it shares a jenkins instance with my tester). > ... > > I think this is not fully true because clang / gcc produce different > binaries in terms of size / performance. So just switching compiler in > some package may result in significat (hopefully not) performance gain > / drop. Certainly switching compilers can result in performance changes. Having been through this kind of disruptive change on the other side (GCC vs various vendor compilers through the 90s), what tends to happen is the compilers get to a point where they're typically +-5% of each other on the vast majority of codes. That's where Clang/LLVM and GCC are today based on the data I've seen. > > Also we probably should mention that -fstack-clash-protection is not > available in clang, so in theory binaries can be less secure due to > that. Clang/LLVM is closing that gap rapidly. I fully expect stack clash protection to be at parity on the relevant platforms except aarch64 by LLVM 11 and a reasonable chance to reach parity on aarch64 by LLVM 12. > > ... > > I think this should mention that annobin plugin should be prepared for > clang. Nick Clifton is already working on that :-) Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
> -BEGIN PGP SIGNED MESSAGE- > Hash: SHA512 > > On Fri, 2020-06-05 at 09:52 +0200, Kevin Kofler wrote: > ... > > Since I was not sure if clang is supported by Red Hat Toolchain team in > the same way as GCC, I've asked this in my reply. If they are supported > in the same manner (maintainers are as well developers in upstream and > work full-time on this, development versions are being tested in > rawhide early, etc…) I do not see a reason to disallow that. Yup. As mentioned in my previous message. Tom S in Red Hat's LLVM lead with Serge G. directly contributing to LLVM and a couple others that are more indirect contributors. > > - From the security features, do you have some specifics in mind? I saw > only from our CFLAGS/LDFLAGS, only the -fstack-clash-protection is not > yet supported, but it is being worked on (already in trunk, though only > for x86). As mentioned elsewhere, LLVM 10 adds x86_64, LLVM 11 should add Power and Z and LLVM 12 adding AArch64. > > ... > > Well, if they are supported in the same way as GCC (in the sense as it > is not just being packaged in Fedora, but developed and properly tested > in Fedora), why not to declare that we have 2 system compilers? > Regarding hidden binary incompatibilities, those are the bugs that > needs to be fixed so I assume if maintainers of clang make commitment, > they will have to fix it because Clang will be 2nd system compiler. Precisely. I think between Tom & Serge with direct upstream work and my contacts into the LLVM groups at IBM and ARM I'm confident we'll be able to deal with issues. Or to look at it another way. Red Hat's supported offerings already include full support for Clang/LLVM (using libstdc++, not libc++) and GCC. They have to work and they have to work together. Red Hat is already in a dual compiler world. Jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
> On 05/06/20 10:26 +0200, Igor Raits wrote: > ... > > Well, upstreams are not necessarily enabling many security features > > or > > optimizations. So you are effectively saying "upstream knows better" > > where I would have to disagree with you. > > Yes, this is a very good point. > > Many of Fedora's packages have upstreams that are not using the latest > compilers, libraries, security features etc. > > Just because upstream hasn't been updated to work with compiler > hardening features doesn't mean we should disable those features. Just > because upstream's code is not portable to more than one compiler > doesn't mean we shouldn't send them bugs (or better still, patches).<> Right. Though I think the security side of this largely belongs in redhat-rpm- config and moving annobin/annocheck into an enforcement role (like we've done with RHEL). We did this for RHEL and while painful, getting the vast majority of packages to honor the flags injection and verification via annobin/annocheck before the resultant packages can be included in the distro has been a big win and enables us to do a lot of useful things knowing that the flags injection works well. Fedora is behind on this. While most packages honor flags injection, we don't actually know which do not (either by accident or design) and we don't have a way to easily find them. So things like CET in enforcing mode by default are going to be harder to achieve in Fedora than in RHEL. But like so many things, I don't have the time to push on something like this for Fedora. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 21:18 +0200, Fabio Valentini wrote: > On Fri, Jun 5, 2020 at 9:11 PM Jeff Law wrote: > > Yes. I thought we bumped up that bug in the database so that it'd get some > > attention in the gcc-10 cycle. But I couldn't follow it myself, so I don't > > know > > what happened. > > Sorry for being off-topic, but can you (Jeff) please not respond to > the mailing list digest and amend the Subject line manually? > I think it screws up the In-Reply-To (?) header in emails, so every > new email from you creates a new thread (for clients that support > this) ... > Right now, this is the 16th thread for this topic in my inbox. It > makes it *really really hard* to follow the discussion. Yea. Dominique and Florian have both reached out to me privately. It's an artifact of consuming the list in -digest form and evolution not having a way to explode the digest for individual replies AFAICT. I'm using Dominique's suggestion and hoping it helps, though it is somewhat painful. Thankfully I don't (usually) have to interact a lot on fedora-devel so just short term pain for me as we figure out if/how to move forward on the compiler stuff. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 22:07 +0200, Igor Raits wrote: > > Just curious, how is it done in RHEL? Just some kind of CI that > analyses all builds or? So we have a step that sits between the build phase and when the resultant packages land in the distro which runs annocheck to validate options used, CET coverage across the binary, PIE, etc etc. If that annocheck run fails, then the packages are not allowed into the distribution, buildroots, etc. We flipped it on a couple years ago in the run up to RHEL 8 and it proved to be quite valuable. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 16:09 -0400, Simo Sorce wrote: > On Fri, 2020-06-05 at 13:58 -0600, Jeff Law wrote: > > On Fri, 2020-06-05 at 21:18 +0200, Fabio Valentini wrote: > > > On Fri, Jun 5, 2020 at 9:11 PM Jeff Law wrote: > > > > Yes. I thought we bumped up that bug in the database so that it'd get > > > > some > > > > attention in the gcc-10 cycle. But I couldn't follow it myself, so I > > > > don't know > > > > what happened. > > > > > > Sorry for being off-topic, but can you (Jeff) please not respond to > > > the mailing list digest and amend the Subject line manually? > > > I think it screws up the In-Reply-To (?) header in emails, so every > > > new email from you creates a new thread (for clients that support > > > this) ... > > > Right now, this is the 16th thread for this topic in my inbox. It > > > makes it *really really hard* to follow the discussion. > > Yea. Dominique and Florian have both reached out to me privately. It's an > > artifact of consuming the list in -digest form and evolution not having a > > way to > > explode the digest for individual replies AFAICT. > > > > I'm using Dominique's suggestion and hoping it helps, though it is somewhat > > painful. Thankfully I don't (usually) have to interact a lot on > > fedora-devel so > > just short term pain for me as we figure out if/how to move forward on the > > compiler stuff. > > I am glad I resisted till this point, as I was going to be the Nth to > contact you privately about it :-) No worries. I'll spare the list my frustrations with MUAs and desire to return to CLI MH :-) > > One way you could use, if it works better, is to import archives and > then reply from there, not sure if this is any less painful then > replying from hyperkitty. > > Another way is to subscribe to the full list instead of digest but have > an aggressive archive policy in evolution to clean your fedora-devel > folder (see auto-archive in folder properties). It may be easiest to just subscribe in the normal way for the duration of this discussion and return to a digest once the issues relevant to me have died down to the usual trickle. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 22:22 +0200, Igor Raits wrote: > On Fri, 2020-06-05 at 14:16 -0600, Jeff Law wrote: > > On Fri, 2020-06-05 at 22:07 +0200, Igor Raits wrote: > > > Just curious, how is it done in RHEL? Just some kind of CI that > > > analyses all builds or? > > So we have a step that sits between the build phase and when the > > resultant > > packages land in the distro which runs annocheck to validate options > > used, CET > > coverage across the binary, PIE, etc etc. If that annocheck run > > fails, then the > > packages are not allowed into the distribution, buildroots, etc. > > > > We flipped it on a couple years ago in the run up to RHEL 8 and it > > proved to be > > quite valuable. > > Seems like a thing we should do in Fedora CI for each update! Possibly. It requires some work on by the packagers to understand the new requirements and how to interpret the annocheck results. Nick & Florian spent countless hours working with RHEL developers to understand and fix packaging issues. But it's awful nice once it's in and everyone's updated their packages. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 21:51 +0200, Igor Raits wrote: > On Fri, 2020-06-05 at 13:36 -0600, Jeff Law wrote: > > > On Thu, 2020-06-04 at 16:30 -0400, Igor Raits wrote: > > > ... > > > > > > Sadly some upstreams insist on clang just because they like it > > > more, > > > without any technical reason. The question that comes to my mind: > > > Should we still try to convince upstreams to use GCC in such cases > > > or > > > not? > > It happens (choosing Clang because they like it) and while we can > > (and have) > > engaged upstreams on this topic and I suspect we will continue to do > > so in some > > cases. > > > > But in the end I think we still have to respect the upstream > > project's choices, > > even if it's just because they like Clang/LLVM more than GCC. > > > > > > > > > Also does this mean that Clang is now fully supported compiler by > > > full- > > > time working people @ Red Hat in toolchains team that are also > > > contributing to upstream? Just curious if we will be able to "fully > > > support" people when we find bugs in the compiler. > > Yes. Absolutely. Tom S & Serge G directly. Others in a more > > indirect capacity. > > > > > And also, does it mean that we will be following same pattern as > > > with > > > GCC to test pre-release versions in rawhide, do the mass rebuild > > > and so > > > on? > > Some of that is already happening internally. Tom's got a tester > > which spins > > Fedora packages with Clang/LLVM. I'm not sure if he's trying to > > cover the whole > > distro, but it is running regularly (it shares a jenkins instance > > with my > > tester). > > > > > > > > > ... > > > > > > I think this is not fully true because clang / gcc produce > > > different > > > binaries in terms of size / performance. So just switching compiler > > > in > > > some package may result in significat (hopefully not) performance > > > gain > > > / drop. > > Certainly switching compilers can result in performance changes. > > Having been > > through this kind of disruptive change on the other side (GCC vs > > various vendor > > compilers through the 90s), what tends to happen is the compilers get > > to a point > > where they're typically +-5% of each other on the vast majority of > > codes. That's > > where Clang/LLVM and GCC are today based on the data I've seen. > > > > > Also we probably should mention that -fstack-clash-protection is > > > not > > > available in clang, so in theory binaries can be less secure due to > > > that. > > Clang/LLVM is closing that gap rapidly. I fully expect stack clash > > protection to > > be at parity on the relevant platforms except aarch64 by LLVM 11 and > > a reasonable > > chance to reach parity on aarch64 by LLVM 12. > > Sure thing! I just asked to have it documented that as of F33 it is not > yet implemented, but is planned to be worked on. So that users do not > expect it to have all security features built-in as on F33. Where do we want this documented? I don't mind adding verbage to the change proposal around this, but I fear that's really not the right place and the info will get lost. Perhaps make it part of the packaging guidelines change if/when we make a change? jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 19:31 +0200, Florian Weimer wrote: > * Ben Cotton: > > > https://fedoraproject.org/wiki/Changes/CompilerPolicy > > > > == Summary == > > Fedora has historically forced packages to build with GCC unless the > > upstream project for the package only supported Clang/LLVM. This > > change proposal replaces that policy with one where compiler selection > > for Fedora follows the package's upstream preferences. > > Jeff, would you please clarify in the proposal that libc++ is out of > scope (as already discussed)? > > What about compiler-rt? And lld? Any other LLVM subprojects that could > be relevant? Done. I've modified the the change request to note that it's just compiler selection and does not apply to runtimes, linkers, debuggers, etc :-) Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 15:04 +0200, Mark Wielaard wrote: > On Fri, 2020-06-05 at 11:19 +0200, Jakub Jelinek wrote: > > On Fri, Jun 05, 2020 at 09:52:09AM +0200, Kevin Kofler wrote: > > > I do not see why we should allow yet another special case for Firefox, > > > nor > > > why we should let random packages make their own choice of compiler and > > > risk > > > running into hidden binary incompatibilities. We have a system compiler > > > for > > > a reason. > > > > I'll note that there are some even not really hidden binary > > incompatibilities, where LLVM diverges from the psABI for years, it has been > > reported and nothing has been changed. > > So if a library is built with clang/LLVM and used by GCC built package or > > vice versa, one might very well run into those (this is e.g. about passing > > std::byte or other scoped enums with char/short underlying type by value, or > > in some cases even about passing char/short arguments). > > And of course unknown ABI bugs on both sides. > > So the new policy is only for non-library packages then? > Packages which provide libraries that can be linked to by other > packages still need to use the default system compiler to make sure > there is no breakage. At least to the other packages. A package that is > build with llvm might subtly break when linked against system > libraries. No, it makes no distinction between library and other packages. Clang/LLVM and GCC are ABI compatible (with the known exception of the alignment issue for atomics) and one should be able to mix and match libraries compiled by one with code compiled by the other just fine. Perhaps you're worried about libc++ (the Clang/LLVM C++ runtime)? We don't use or support libc++ in Fedora because it is not ABI compatible with libstdc++ and an object created by one library can not be used/manipulated by the other (or instantiated templates from the other). If you're aware of ABI compatibility issues between the compilers, certainly chime in here, the appropriate BZ or to me directly. You can even raise it in the Tuesday meetings with the LLVM/GCC teams -- I'm sure that if there's an issue that both Tom and I will want to know about it. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 09:59 +0100, Jonathan Wakely wrote: > On 05/06/20 10:23 +0200, Tomáš Popela wrote: > > On Fri, Jun 5, 2020 at 9:56 AM Kevin Kofler > > wrote: > > > > > I am opposed to this change. Chromium and Firefox build fine with GCC. I > > > think that a distribution should be built with a consistent toolchain > > > wherever possible. > > > > > > > Kevin, that's not true at all. Maybe it looks like it builds fine for you, > > because all the fixes get to you, but I as a co-maintainer of Chromium in > > Fedora and ex-maintainer of Chromium in RHEL can say that most of the time > > I spent fixing GCC problems. Just look at the current SPEC file and search > > for gcc there.. There are several gcc related bugs - > > https://src.fedoraproject.org/rpms/chromium/blob/master/f/chromium.spec. Or > > even better look at > > https://bugs.chromium.org/p/chromium/issues/detail?id=819294 where the GCC > > The very first one there is just a bug in Chromium: using a type > without including the header that defines it. There are several others > like that (relying on unique_ptr bing declared implicitly, relying on > nullptr_t being declared implicitly ...) > > Those are upstream bugs and should be fixed. Period. Yup. And what you're hitting on is a real issue. Monocultures are generally bad. All the world is a vax, all the world is a sparc (running solaris), all the world is x86-linux, everything builds with gcc and so-on. There's a secondary problem you're touching on and that's engagement to get stuff fixed correctly. Finding the balance there can be hard too, particularly if the developer doesn't know *how* to engage a particular community. I run into this myself and it can be frustrating. So while I'd rather see better engagement on bugs, I do understand why a Chromium developer might not have reported issues to the GCC community. In an ideal world, everything would build with both compilers, folks would review and fix all the diagnostics they generate. Selection of which binaries would essentially happen at the end of the build cycle with the decision made on a per- package basis. However, I strongly suspect the pitchforks would be out if we tried to go that far :-) [ ... ] > tl;dr This is a chromium problem, not just a GCC problem. Fixing those > portability problems is an investment and reduces technical debt. It's a balance and judgment call. For Chromium and Firefox the cost of maintaining GCC as a build toolchain has apparently outgrown the cost of non- portable code. Of course the cost of the latter usually does show up until much later in a product's lifecycle, so only time will tell if those projects have made a good decision. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 21:49 +0200, Jakub Jelinek wrote: > On Fri, Jun 05, 2020 at 01:36:37PM -0600, Jeff Law wrote: > > > On Thu, 2020-06-04 at 16:30 -0400, Igor Raits wrote: > > > ... > > > > > > Sadly some upstreams insist on clang just because they like it more, > > > without any technical reason. The question that comes to my mind: > > > Should we still try to convince upstreams to use GCC in such cases or > > > not? > > It happens (choosing Clang because they like it) and while we can (and have) > > engaged upstreams on this topic and I suspect we will continue to do so in > > some > > cases. > > > > But in the end I think we still have to respect the upstream project's > > choices, > > even if it's just because they like Clang/LLVM more than GCC. > > Why? Shouldn't the Fedora maintainers be able to decide this? If they have > been > using GCC for years and it hasn't been an obstackle for them, why should > they switch? Maintainers aren't static and when the maintainer chooses to go a different direction than upstream, then there's a cost to be paid. While it may work for a particular maintainer to do something different for their package, what about the poor person who takes the package over when the original maintainer leaves or someone who has to pick it up to deal with a CVE while the maintainer is on PTO. Not everyone is as toolchain savvy as you, or is able to understand the intent of code as well, or is likely to have the longevity you've had, etc. You're at a level that most developers will never achieve in their career. Think about folks that are less experienced, or with fewer natural gifts, etc and sometimes you come up with different answers. Conceptually, IHMO, packages in Fedora should be as close to upstream as possible. Toolchain selection is a piece of that. The major exception in my mind is flag selection, particularly due to the need to enforce consistent security hardening, distro-wide optimizations, or ISA selection to ensure the bits run everywhere they're supposed to. > If I understand the LO case, it is just that LO sometimes uses the Skia > library which is written by Google and Google likes compiler monoculture and > is using heavily #ifdef __clang__ in it and using the clang variants of the > generic vectors guarded by that, and as fallback just doesn't use simd. > I believe Honza Hubicka had quite some changes for Skia, not sure if they went > upstream already or not. > But the maintainers should be able to choose, build just Skia with clang and > rest of LO with GCC, or everything with GCC and with help from us get Skia > into shape for better portability (that would be ideal, but of course can > mean more hopefully one time work), or build all of it with Clang/LLVM. Again, I'd fall back to the "what does upstream do". DOing something different should require strong technical justification and none of the stuff I've seen discussed around LO, Firefox or Chromium would meet that bar IMHO. jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 16:47 -0500, Steven Munroe wrote: > Jeff Law wrote: > > I'd respectfully disagree. There are certain features that GCC supports > > that Clang does not > > and vice-versa. But broadly they are comparable. What this means is some > > projects that are > > using bleeding edge features may have a hard need for one toolchain or the > > other. And the > > proposal I've made accounts for that by allowing the upstream project to > > select the compiler. > > In your case it would be GCC. For others it could well be Clang/LLVM. > > I respectfully disagree with your definition of bleeding edge ... Perhaps that was a bad choice of words. > > And while you allow that some packages have good reasons to stick with > GCC. Several others on this list have demanded that clang/LLVM replace > GCC as the default compiler. That's not what is being proposed. What's being proposed is far more narrowly scoped. > > I respectfully restate my position that clang/LLVM is incomplete and > not ready for that role. The proposal under debate is to change Fedora policy so that compiler selection comes from the upstream project. For your project clearly GCC is a better choice. Other projects Clang/LLVM is a better choice. Whether or not Clang/LLVM is ready to replace GCC as the default compiler is not on the table and thus isn't terribly interesting (to me) to debate. So I would ask that we keep the discussion to the proposal at hand. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 10:14 +0100, Jonathan Wakely wrote: > On 04/06/20 16:30 -0400, Ben Cotton wrote: > > https://fedoraproject.org/wiki/Changes/CompilerPolicy > [snip] > > == Documentation == > > Several years ago Red Hat's tools team championed for Fedora policy to > > strongly > > discourage the use of LLVM/Clang for package building. Exceptions were > > made for > > packages that could only be built with Clang/LLVM: > > > > https://docs.fedoraproject.org/en-US/packaging-guidelines/#compiler > > > > > > At that point in history Red Hat had no Clang/LLVM engineers or expertise. > > In > > fact, the LLVM packages were actually maintained by an engineer on the > > desktop > > team (they had a hard requirement for llvm-pipe, so they got to own the > > Clang/LLVM bits). The policy essentially was a risk management strategy for > > Fedora. > > > > Times have changed and as a result we should revisit that Fedora policy. > > > > The Red Hat tools team believes that LLVM/Clang and GCC should be > > considered equals from > > a Fedora policy standpoint. Selection of one toolchain over the > > should should be > > s/should should be/other should be/ Fixed. > > > driven by the upstream project's preferences not by Fedora specific policy. > > I think this needs to be clear that we're talking about the compiler > here, not the C++ standard library. Fedora has no libc++ expertise I'm > aware of. > > I think we want to be a lot more cautious about using libc++, because > it would potentially require large parts of the C++ stack to be built > twice and installed in parallel (think Python 2 and Python 3). For > example if your package depends on Boost and you want to use libc++ > then you need Boost rebuilt. Similarly for any C++ library, Qt, gtkmm, > etc. etc. > > And I do not intend to offer C++ support for people who decide to use > Clang + libc++ but don't try to resolve their own build failures :) Agreed 100%. Based on comments from others I've already updated the proposal to hopefully clarify this change is strictly the compiler and does not include the runtime. Trying to support libc++ would be a compatibility nightmare. I do believe one day we'll need to do something there, but now isn't the time to open that discussion. > > > > What that means in practice is that if the project upstream prefers > > Clang/LLVM, > > then Fedora should in turn be using Clang/LLVM to build those packages. As > > a > > concrete example, let's consider Chromium. > > > > Chromium upstream has been building with Clang/LLVM for several years. > > Yet policy > > has forced Fedora package owners to shoulder a significant burden to make it > > build with GCC. Furthermore, Fedora does not get as much benefit at it > > could by > > forcing Chromium to be built with GCC since most other instances are built > > with > > Clang/LLVM. > > > > By changing policy Fedora package maintainers no longer have to waste > > time trying > > to make Chromium build/work with GCC and Fedora gains additional "many > > eyes" and > > "many users" benefits by relying on the same tools to build Chrome as the > > upstream developers and other distributions. > > > > Additionally, if an upstream project currently uses GCC, but switches to > > Clang/LLVM (or vice-versa), then the package in Fedora should switch > > in a similar > > manner. The only policy restriction should be that the compiler must > > exist in Fedora. > > If upstream supports both compilers, that's probably not a good reason > to change anything in Fedora. If they're on equal footing upstream, then no I don't think changing would be the right thing to do. I don't think that's really the case for Firefox or Chromium. I'm less familiar with the LibreOffice situation, so I won't try to give an opinion there. Perhaps in this case we leave it up to the Fedora packager? > > > In some ways this means there is no "default" compiler for Fedora. The > > default > > is whatever the upstream project supports/recommends. However, there are > > probably many packages with upstreams that are ambivalent about their > > compiler > > choice. For those packages I would recommend we keep the status quo at the > > current time. For a package with a dead upstream, the Fedora packager > > should be > > able to select the compiler they want to use for the package. > > I would prefer the policy to have a stronger preference for GCC when > there is no good reason to change. Currently it's worded like keeping > the status quo is Jeff's personal opinion. I would like it to be > policy. So that text was literally copy and pasted from internal discussions. So, yea, it's my opinion and I think it's one of the option questions this body needs to hammer out to more precisely define any updated policy. My preference would be to stick with GCC when upstream has no recommendations/support policy around supported compilers to avoid unnecessary churn. I would also be willing to sup
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Sat, 2020-06-06 at 07:58 +0200, Igor Raits wrote: > The big problem then becomes getting packagers to address the > > diagnostics. I've > > been disappointed at how many packages are ignoring diagnostics > > (particularly > > those with security implications) and I'm actively looking at schemes > > to improve > > this situation :-) > > Just make them error by default and people will have to deal with it :) Easier said than done. Though having something like the annobin/annocheck stuff in place does help -- folks can't simply disable the warning in their package which I've seen happen far too often. One of the big problems is you can end up with a ton of local patches if the upstream project doesn't take this stuff seriously. And every one of those local patches has a cost. Naturally folks object to the initial work and ongoing cost, particularly if upstream isn't on board. So, if we do go forward with some of the ideas, they'll probably be some kind of opt-in with packages where Red Hat's tools team has significant influence taking the lead since the projects we work with regularly do generally take this stuff seriously. I have some thoughts on how to expand the set of packages covered, but I'm not particularly ready to publicize those yet :-) jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Sun, 2020-06-07 at 13:07 -0700, stan via devel wrote: > On Thu, 4 Jun 2020 16:30:09 -0400 > Ben Cotton wrote: > > > https://fedoraproject.org/wiki/Changes/CompilerPolicy > > An obvious example is Firefox. Upstream, the Firefox project builds > > primarily with Clang/LLVM. Yet we force the Fedora package owner to > > find and fix issues building with GCC then either carry those custom > > fixes forward in Fedora or negotiate with upstream to get those > > changes upstreamed. While this process can be helpful in finding > > non-portable code, this is ultimately a poor use of the packager's > > time. > > I compile firefox nightly locally from an hg repository. What I notice > is that although mozilla officially states that it is possible to limit > resource use, even though I set a maximum thread count, or maximum > load, the clang compiler grabs every core and maxes it out to 100%. > Will this change help with that? i.e. is this a clang problem? Or is > my experience because of a problem with the firefox compilation process? It's unclear if it's a problem in Clang/LLVM or the firefox build procedures. Regardless, this proposal shouldn't change/improve your issue in any way. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 23:04 +0200, Mark Wielaard wrote: > Hi Jeff, > > On Fri, 2020-06-05 at 10:07 -0600, Jeff Law wrote: > > On Fri, 2020-06-05 at 15:56 +, devel-requ...@lists.fedoraproject.org > > wrote: > > > One issue I am concerned about here is debuginfo quality. GCC produced > > > pretty bad debuginfo with LTO in older versions. The EarlyDebug work > > > did make it usable. And we needed some work on the DARF consumers to > > > correctly process GCC 10 LTO produced DWARF (I actually have two small > > > patches for elfutils pending so it correctly parses the cross CU- > > > references GCC 10 LTO now produces). > > > > > > I don't know if anybody did any analysis on llvm LTO produced > > > debuginfo. In the past it was observed that llvm doesn't do VTA (Var > > > Tracking Assignments) that is default in GCC. And VTA is really > > > important for debugging and performance/tracing tools like systemtap. > > > So even without LTO I am concerned about the observability of llvm > > > generated binaries. > > > > And these are reasonable concerns (even more so in an LTO world and I > > expect I'll > > be spending most of my summer poking at debuginfo stuff for GCC LTO :-). > > > > IMHO this is one of the issues an upstream project needs to evaluate for > > their > > toolchain choice -- no different than performance, diagnostics, sanitizer > > support, plug-in support, etc etc. The level of observability needed by any > > particular upstream project can vary. ie, the kernel may well have > > different > > needs than the Ruby interpreter. > > I think you place too much faith in upstream and give Fedora and Fedora > packagers not enough credit for creating a well integrated > distribution. For Fedora it is important that what we ship is > observable by our users. We, together with our users, sometimes need to > debug, trace or do performance analysis on the binaries as we ship > them. Some upstream projects don't care about that at all. They will > happily tell their users to just rebuild with some other > tools/optimizations/etc. Or just rebuild their whole world with some > sanitizers enabled. So then an upstream developer might prefer a > different toolchain, simply because they personally would rebuild the > whole world with it. In such cases Fedora should not simply switch to a > non-default system compiler. I put faith in both the upstreams and packagers to do the right thing for their project. Right now Fedora policy does exactly the opposite by forcing everyone to use GCC rather than making an informed, per project, decision. If an upstream has decided that debug-ability isn't a priority, then, the packager for Fedora has to decide if the project is even really suitable for Fedora and take on a degree of responsibility for the package. > > > I have not done any evaluation of LLVM's debug info. I would not be > > surprised if > > GCC is generally better, particularly with Alex's work over the last couple > > years. But again, I think the upstream project is in the best position to > > determine how to balance this against the other factors in toolchain > > selection. > > GCC's is certainly better, it has much better coverage. That doesn't > mean that LLVM's debuginfo is invalid, but it is less useful. At some > point the kernel accidentially disabled VTA and it made the kernel a > lot less observable by systemtap for example. I'm well aware of the kernels' disabling VTA, it was done in response to a codegen bug that was triggered by the VTA changes. Sadly the kernel's response was a bit drastic. But that's water under the bridge. > > And even if it is correct it might still not be fully supported by all > our tools. For example clang used to default to not generate > .debug_aranges, which is valid, but means a DWARF consumer needs to > construct its own aranges table by reading the all the CUs. elfutils > doesn't support that, and so some address/line lookups simply fail with > clang generated binaries. I could cerainly add support for missing > .debug_aranges, but there is so much more to do (luckily I could > convince them to switch the default to generate .debug_aranges, at > least for rust code). > > In general when someone reports an issue with an elfutils or valgrind > based tool and it is against code not compiled with the default gcc > toolchain I simply have to tell people to please use the system > compiler. There are just not enough time to also try to resolve any > debuginfo issues because people use an alternative toolchain. All valid
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 21:49 +0200, Jakub Jelinek wrote: > On Fri, Jun 05, 2020 at 01:36:37PM -0600, Jeff Law wrote: > > > On Thu, 2020-06-04 at 16:30 -0400, Igor Raits wrote: > > > ... > > > > > > Sadly some upstreams insist on clang just because they like it more, > > > without any technical reason. The question that comes to my mind: > > > Should we still try to convince upstreams to use GCC in such cases or > > > not? > > It happens (choosing Clang because they like it) and while we can (and have) > > engaged upstreams on this topic and I suspect we will continue to do so in > > some > > cases. > > > > But in the end I think we still have to respect the upstream project's > > choices, > > even if it's just because they like Clang/LLVM more than GCC. > > Why? Shouldn't the Fedora maintainers be able to decide this? If they have > been > using GCC for years and it hasn't been an obstackle for them, why should > they switch? The experts in a particular codebase are going to be the upstream maintainers, not Fedora packagers. Sometimes those are the same developers, but often they are not. In the former case, obviously the right thing will "just happen". In the latter the upstream is the right place to be making this decision. > If I understand the LO case, it is just that LO sometimes uses the Skia > library which is written by Google and Google likes compiler monoculture and > is using heavily #ifdef __clang__ in it and using the clang variants of the > generic vectors guarded by that, and as fallback just doesn't use simd. > I believe Honza Hubicka had quite some changes for Skia, not sure if they went > upstream already or not. If I read that patch correctly, they were prefering Clang/LLVM just in the skia library and not across the whole project -- which also seems like a valid case to me. If part of a project wants to use Clang/LLVM and another part is using GCC, then that's the project's decision. That decision has consequences (like the inability to LTO across those components), but again, the upstream project is going to be in the best position to make that decision. > But the maintainers should be able to choose, build just Skia with clang and > rest of LO with GCC, or everything with GCC and with help from us get Skia > into shape for better portability (that would be ideal, but of course can > mean more hopefully one time work), or build all of it with Clang/LLVM. I would agree if you inserted "upstream" before "maintainers" here :-) I do think that we could/should have an exception policy where the Fedora packager could make a case to do something different from upstream WRT toolchain selection. But that should be an exceptional case, not the norm. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Fri, 2020-06-05 at 23:16 +0200, Jakub Jelinek wrote:
> On Fri, Jun 05, 2020 at 03:03:18PM -0600, Jeff Law wrote:
> > Clang/LLVM and GCC are ABI compatible (with the known exception of the
> > alignment
> > issue for atomics) and one should be able to mix and match libraries
> > compiled by
> > one with code compiled by the other just fine.
>
> They are known not to be ABI compatible, see e.g.
> https://bugs.llvm.org/buglist.cgi?quicksearch=42439%2C19909%2C44228%2C12207
> That is just one arch, has anyone e.g. tried gcc
> make check-gcc check-c++-all RUN_ALL_COMPAT_TESTS=1 ALT_CC_UNDER_TEST=clang \
> ALT_CXX_UNDER_TEST=clang++ RUNTESTFLAGS=struct-layout-1.exp
> on different architectures?
> That might come up with more.
>
> Is e.g. C++
> struct A {};
> struct B { [[no_unique_address]] A a; double b; };
> passed by value the same between g++ 10 and latest clang++?
It's a bug and should be treated as such. Given it's a c++20 feature I wouldn't
consider it terribly concerning at this stage. Hell, gcc's been all over the
map
with this stuff varying release-to-release, particularly for the empty baseclass
stuff.
Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Mon, 2020-06-08 at 12:21 -0400, Robbie Harwood wrote: > Igor Raits writes: > > > On Fri, 2020-06-05 at 23:11 -0600, Jeff Law wrote: > > > On Fri, 2020-06-05 at 10:14 +0100, Jonathan Wakely wrote: > > > > On 04/06/20 16:30 -0400, Ben Cotton wrote: > > > > > > > > > https://fedoraproject.org/wiki/Changes/CompilerPolicy > > > > > > > > > > In some ways this means there is no "default" compiler for Fedora. > > > > > The default is whatever the upstream project supports/recommends. > > > > > However, there are probably many packages with upstreams that are > > > > > ambivalent about their compiler choice. For those packages I would > > > > > recommend we keep the status quo at the current time. For a > > > > > package with a dead upstream, the Fedora packager should be able to > > > > > select the compiler they want to use for the package. > > > > > > > > Ideally we'd have CI building (nearly) everything with *both* GCC > > > > and Clang, and finding and fixing problems in packages and in both > > > > compilers. But that's probably not realistic (yet?). > > > > > > You may remember me advocating for this in our meeting in Montreal :- > > > ) So, yea, I'd be totally on board with something like this. I think > > > Tillman was also interested and even floated the idea of finding > > > additional Fedora builder resources to facilitate this kind of > > > scheme. > > > > > > The big problem then becomes getting packagers to address the > > > diagnostics. I've been disappointed at how many packages are > > > ignoring diagnostics (particularly those with security implications) > > > and I'm actively looking at schemes to improve this situation :-) > > > > Just make them error by default and people will have to deal with it > > :) > > (I know you weren't seriously proposing that we do this, but it's an > idea I've seen seriously proposed elsewhere and have experienced.) > > Please do not do this for non-security diagnostics. Supporting this > across a matrix of different compiler versions (and compilers) is truly > awful - especially managing semantics changes between versions, and > pragma stew, and behavior of compilers when asked to ignore flags they > don't understand... My desire is to only target security related diagnostics and quite possibly only a small subset initially. I don't see even trying to move on this until after F33 is done. jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 33 System-Wide Change proposal: CompilerPolicy Change
On Tue, 2020-06-09 at 08:57 +0200, Nicolas Mailhot wrote: > Le lundi 08 juin 2020 à 09:48 -0600, Jeff Law a écrit : > > I put faith in both the upstreams and packagers to do the right thing > > for their project. Right now Fedora policy does exactly the opposite > > by forcing everyone to use GCC rather than making an informed, per > > project, decision. > > Look, 9∕10th of the times the "informed" upstream decision is just one > poor dev who slapped some build automation using the first compiler he > though of in quick and dirty mode so he could get back to coding as > fast as possible (sometimes, for another system than Linux, that was > grandfathered for Linux builds later, because who cares about those > things in real life). That certainly happens. I get to see both the good and the bad in terms of code quality, build systems, etc for Fedora. I won't call out any packages by name, but some of the stuff I've seen is of such poor quality that the author should be embarrassed. Thankfully these packages are likely only being used by one person and when it crashes because they've written high school quality code the impact is minimal. In other cases the upstream code is exceedingly good and thorough in terms of trying to do the right thing. So much in fact that it obscures the intent of the code so much that GCC can't actually analyze it fully and GCC spits out a false positive diagnostic. I've seen code put in to shut up things like Coverity that is horribly wrong and introduces new buffer overflows and undefined behavior. But I'm also experienced enough to know there's a lot of selection bias that goes on here. I see the old crappy code because it breaks and breaks often as we update from one version of gcc to the next. I see the well written, security sensitive code as well because gcc tries hard to issue diagnostics for potentially problematic code -- and it's precisely those codes that end up being complex as the developer tries to do the right thing and avoid undefined behavior in the first place. > > Sifting through build systems, possible compilers and compiler options > takes an enormous amount of care and knowledge which is lacking in the > average upstream project. Only giants like Chrome or Mozilla can afford > to do this level of analysis. I’m not even putting LibreOffice in this > class – they switched compiler for a subset of their codebase, probably > just to workaround a perf regression quickly in the hope that someone > would analyse the regression causes later. Not even sure they bothered > to check if the regression happened with all possible build flags > combinations, one report of "it’s faster using compiler XXX" (one > version of one compiler on one system with one set of build flags), is > usually sufficient for this kind of switch. No doubt. It does take work, often significant amounts of work. I get to see it all because of the work I do to try and make sure GCC and Fedora work well together results in me looking at hundreds or thousands of build issues every cycle. I have to figure out the build system, intent of the sources or testcases, then debug what's going on in the package and what should be going on in whatever part is failing, then make a decision about whether or not the issue is a package problem or a compiler problem, then fix the problem or give someone else enough state to do so. Rinse and repeat. > > Arguably, in Fedora the average packager can not afford it either – he > relies 100% on the default compiler flags defined by the gcc team. And > the gcc team can afford to look at those closely only because of the > mass effect of a whole distribution codebase that is fully built using > the same compiler and flags everywhere. And that's not changing. The GCC team still selects default flags, we still analyze build issues to ensure things go as smoothly as possible, we still look for ways to make things work faster, compiler to smaller code, etc. We also work closely with our LLVM team to make sure we're doing the right thing for the Clang/LLVM toolchain as well. > > “Upstream knows better” – for upstream’s own code sure, but for > everything else (legalities, the tird party code they depend on, build > systems and automation) certainly not. I'd disagree, particularly around build systems and the compiler components within build systems. THere's always exceptions, but by in large I believe upstreams are in the best position to select the compiler toolchains. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Very strange compiler/linker related build failures in rawhide
On Fri, 2020-07-24 at 10:31 +0200, Fabio Valentini wrote:
> Hi all,
>
> I'm starting to see various very strange kinds of build failures in
> rawhide, that seem to have started with either of these updates (or a
> combination of them):
>
> - annobin 9.21-1.fc33 → 9.22-1.fc33
> - binutils 2.34.0-6.fc33 → 2.34.0-7.fc33
> - elfutils 0.179-2.fc33 → 0.180-2.fc33
> - glibc 2.31.9000-13.fc33 → 2.31.9000-14.fc33
>
> These rawhide updates all happened at roughly the same time, so it's
> difficult to say which one of them is to blame (if any of them).
>
> One error I've seen in libreoffice is a gcc / annobin segfault:
>
> [build CXX] vcl/unx/gtk3/gtk3gtkinst.cxx
> *** WARNING *** there are active plugins, do not report this as a bug
> unless you can reproduce it without enabling any plugins.
> Event| Plugins
> PLUGIN_FINISH_UNIT | annobin: Generate final annotations
> PLUGIN_START_UNIT| annobin: Generate global annotations
> PLUGIN_ALL_PASSES_START | annobin: Generate per-function annotations
> PLUGIN_ALL_PASSES_END| annobin: Register per-function end symbol
> /builddir/build/BUILD/libreoffice-6.4.5.2/vcl/workben/vcldemo.cxx: In
> destructor 'virtual DemoWin::RenderThread::~RenderThread()':
> /builddir/build/BUILD/libreoffice-6.4.5.2/vcl/workben/vcldemo.cxx:1733:18:
> internal compiler error: Segmentation fault
> 1733 | join();
This sounds like a compiler bug. Can you try adding
"%define _lto_cflags %{nil}"
To the .spec file and see if that gets you over the hump? I've seen one failure
of this nature in my LTO testing and haven't gotten around to producing a
bugreport suitable for upstream (but the affected package has LTO disabled to
keep it from failing its builds). My tester reports that it's never got a clean
control build of libreoffice, so I've never dug into it for any LTO specific
failures.
> | ^
>
> Other errors look like this one from switchboard-plug-onlineaccounts:
>
> src/libonline-accounts.so.p/Authentification/Server.c: In function
> ‘online_accounts_server_on_bus_acquired’:
> src/libonline-accounts.so.p/Authentification/Server.c:498:2: error:
> function ‘__errno_location’ is initialized like a variable
> 498 | gint errno = 0;
> | ^~~~
>
> Where errno is neither __errno_location, nor a function, but a gint??
This is more likely related to the glibc update.
>
> Other failures I've seen end up with linker failures, line these, from
> postgresql:
>
> ld: undefined reference to `postgresql_subtrans__checkpoint__start_semaphore'
This is a known interaction between stap/dtrace probes and LTO. I've already
fixed postgresql to avoid LTO until we fix this issue on the GCC side.
jeff
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Very strange compiler/linker related build failures in rawhide
On Fri, 2020-07-24 at 17:44 +0200, Fabio Valentini wrote:
> On Fri, Jul 24, 2020 at 5:11 PM Jeff Law wrote:
> > > One error I've seen in libreoffice is a gcc / annobin segfault:
> > >
> > > [build CXX] vcl/unx/gtk3/gtk3gtkinst.cxx
> > > *** WARNING *** there are active plugins, do not report this as a bug
> > > unless you can reproduce it without enabling any plugins.
> > > Event| Plugins
> > > PLUGIN_FINISH_UNIT | annobin: Generate final annotations
> > > PLUGIN_START_UNIT| annobin: Generate global annotations
> > > PLUGIN_ALL_PASSES_START | annobin: Generate per-function
> > > annotations
> > > PLUGIN_ALL_PASSES_END| annobin: Register per-function end
> > > symbol
> > > /builddir/build/BUILD/libreoffice-6.4.5.2/vcl/workben/vcldemo.cxx: In
> > > destructor 'virtual DemoWin::RenderThread::~RenderThread()':
> > > /builddir/build/BUILD/libreoffice-6.4.5.2/vcl/workben/vcldemo.cxx:1733:18:
> > > internal compiler error: Segmentation fault
> > > 1733 | join();
> > This sounds like a compiler bug. Can you try adding
> > "%define _lto_cflags %{nil}"
> >
> > To the .spec file and see if that gets you over the hump? I've seen one
> > failure
> > of this nature in my LTO testing and haven't gotten around to producing a
> > bugreport suitable for upstream (but the affected package has LTO disabled
> > to
> > keep it from failing its builds). My tester reports that it's never got a
> > clean
> > control build of libreoffice, so I've never dug into it for any LTO specific
> > failures.
>
> I added this %define _lto_cflags %{nil} to the top of the libreoffice
> .spec file, and compiled it in mock locally.
> And it spits out the same GCC crash error message without LTO.
THanks for checking. That'll make things easier ;-)
Add -save-temps to the compile line and also build with V=1 so you can get the
full command line. Pass along the .ii file and that full command line and I'll
take a peek at what's going on inside GCC.
jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Very strange compiler/linker related build failures in rawhide
On Fri, 2020-07-24 at 17:59 +0200, Fabio Valentini wrote:
> On Fri, Jul 24, 2020 at 5:48 PM Jeff Law wrote:
> > On Fri, 2020-07-24 at 17:44 +0200, Fabio Valentini wrote:
> > > On Fri, Jul 24, 2020 at 5:11 PM Jeff Law wrote:
> > > > > One error I've seen in libreoffice is a gcc / annobin segfault:
> > > > >
> > > > > [build CXX] vcl/unx/gtk3/gtk3gtkinst.cxx
> > > > > *** WARNING *** there are active plugins, do not report this as a bug
> > > > > unless you can reproduce it without enabling any plugins.
> > > > > Event| Plugins
> > > > > PLUGIN_FINISH_UNIT | annobin: Generate final annotations
> > > > > PLUGIN_START_UNIT| annobin: Generate global
> > > > > annotations
> > > > > PLUGIN_ALL_PASSES_START | annobin: Generate per-function
> > > > > annotations
> > > > > PLUGIN_ALL_PASSES_END| annobin: Register per-function end
> > > > > symbol
> > > > > /builddir/build/BUILD/libreoffice-6.4.5.2/vcl/workben/vcldemo.cxx: In
> > > > > destructor 'virtual DemoWin::RenderThread::~RenderThread()':
> > > > > /builddir/build/BUILD/libreoffice-6.4.5.2/vcl/workben/vcldemo.cxx:1733:18:
> > > > > internal compiler error: Segmentation fault
> > > > > 1733 | join();
> > > > This sounds like a compiler bug. Can you try adding
> > > > "%define _lto_cflags %{nil}"
> > > >
> > > > To the .spec file and see if that gets you over the hump? I've seen
> > > > one failure
> > > > of this nature in my LTO testing and haven't gotten around to producing
> > > > a
> > > > bugreport suitable for upstream (but the affected package has LTO
> > > > disabled to
> > > > keep it from failing its builds). My tester reports that it's never
> > > > got a clean
> > > > control build of libreoffice, so I've never dug into it for any LTO
> > > > specific
> > > > failures.
> > >
> > > I added this %define _lto_cflags %{nil} to the top of the libreoffice
> > > .spec file, and compiled it in mock locally.
> > > And it spits out the same GCC crash error message without LTO.
> > THanks for checking. That'll make things easier ;-)
> >
> > Add -save-temps to the compile line and also build with V=1 so you can get
> > the
> > full command line. Pass along the .ii file and that full command line and
> > I'll
> > take a peek at what's going on inside GCC.
> >
> > jeff
> >
>
> Looks like somebody already did that and attached the .ii file to the RHBZ.
> https://bugzilla.redhat.com/show_bug.cgi?id=1859588
Unfortunately neither Marek nor I can reproduce with the compilers we've tested.
Is it possible the OOM killer is killing the process? Is there anything in the
system logs that might be relevant?
Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Very strange compiler/linker related build failures in rawhide
On Fri, 2020-07-24 at 20:52 +0200, Vít Ondruch wrote:
> The LTO break Ruby on various platforms.
>
>
> https://koji.fedoraproject.org/koji/taskinfo?taskID=47582573
>
> vs
>
> https://koji.fedoraproject.org/koji/taskinfo?taskID=47621733
>
> (Note these are my experimental builds testing single test case).
I haven't gotten a clean ruby build with or without LTO. So I haven't
investigated Ruby for any LTO specific failures.
>
>
> The only difference is redhat-rpm-config 162-1.fc33 => 163-1.fc33.
> Adding `%define _lto_cflags %{nil}` helped to recover, but if it was at
> least behaving the same on all platforms :/
>
>
> And this is Koschei failure: https://koschei.fedoraproject.org/package/ruby
>
> Looking at the full test suite, it seems it causes some troubles to
> SIGSEV signal handler (Ruby spawns subprocess and kills it).
Does the signal handler modify any global variables? That's been a common
source
of issues I've seen.
jeff
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: The case of LTO when produced enlarged binaries
On Fri, 2020-07-24 at 19:12 +, Artem Tim wrote: > Hi. In rare cases building packages with LTO producing binaries or libraries > which have bigger size then if they have built without LTO. For example > 'kitty' package: > > * with LTO: > - koji task https://koji.fedoraproject.org/koji/taskinfo?taskID=47762998 > 1.79 MB glfw-wayland.so > 1.99 MB glfw-x11.so > 4.78 MB fast_data_types.so > 8.56 MB total > > * no LTO > - koji https://koji.fedoraproject.org/koji/taskinfo?taskID=47769854 > 1.65 MB glfw-wayland.so > 1.84 MB glfw-x11.so > 4.51 MB fast_data_types.so > 8.00 MB total > > Difference is 7%. What we should do in such case? Should we disable LTO for > such packages? Or there is still could be gains from faster code execution > speed? I'd tend to leave LTO on, but it's totally your call. Without looking at the binaries, sources and compiler dumps I'd hazard a guess you're getting a lot of cross module inlining, but very little identical code folding. THe former tends to make things bigger, but faster. The latter tends to shrink code with little impact on runtime performance. Unfortunately comparing this stuff in an LTO world is much harder than in a non- LTO world. You can't just bisect it to a .o or function that's larger :( jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: z3 soname bump
On Fri, 2020-07-24 at 13:15 -0600, Jerry James wrote:
> I will soon push a change to the z3 package, in Rawhide only, which
> will result in an soname bump. The actual contents of libz3 will not
> change, however. The only Fedora consumer outside of the z3 package
> itself is cppcheck, which currently fails to build due to the recent
> cmake change. If the cppcheck maintainers want me to try to fix that,
> I will do so; otherwise, I am happy to let them fix it themselves.
>
> The gory details for those interested:
>
> The z3 project has two build systems: an old one, based on generating
> Makefiles with python scripts, and a new one based on cmake. We have
> been using the old one, because the cmake build system does not
> support building z3's OCaml interface. However, the old build system
> has a number of ... features ... that we had to work around, leading
> to a fair amount of uncleanness in the spec file.
>
> I have decided to take the plunge and switch to using the cmake build
> system, with manual steps afterward to build the OCaml interface. The
> old build system gave the z3 library an soname of "libz3.so", which
> the spec file modified to be "libz3.so.0", with a versioned library
> libz3.so.0.0.0. The cmake build system gives the library an soname of
> "libz3.so.4.8" (where 4 and 8 are the major and minor version numbers,
> respectively), with a versioned library libz3.so.4.8.8.0. This lets
> me throw out a bunch of cruft, while introducing a much smaller amount
> of cruft due to the OCaml interface. I think it's a win.
Just a note on z3.
I've been trying to track down what I think is an uninstantiated template issue
that's exposed by LTO. I've been chasing it on/off over the last day or two
without success. So if you get a build failure that looks like this:
/usr/bin/ld: /tmp/z3.16EntH.ltrans1.ltrans.o:(.data.rel.ro+0x110): undefined
reference to `lp::lp_solver::get_variable_name[abi:cxx11](unsigned int) const'
/usr/bin/ld: /tmp/z3.16EntH.ltrans1.ltrans.o:(.data.rel.ro+0x168): undefined
reference to `lp::lp_solver::get_variable_name[abi:cxx11](unsigned int) const'
/usr/bin/ld: /tmp/z3.16EntH.ltrans1.ltrans.o:(.data.rel.ro+0x1b0): undefined
reference to `lp::lp_solver::get_variable_name[abi:cxx11](unsigned int) const'
collect2: error: ld returned 1 exit status
Just disable LTO via the usual mechanism (%define _lto_cflags %{nil}). We'll
revisit any opt-outs again between F33 and F34 and re-evaluate them.
Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: The case of LTO when produced enlarged binaries
On Fri, 2020-07-24 at 22:24 +0200, Igor Raits wrote: > On Fri, 2020-07-24 at 13:29 -0600, Jeff Law wrote: > > On Fri, 2020-07-24 at 19:12 +, Artem Tim wrote: > > > Hi. In rare cases building packages with LTO producing binaries or > > > libraries which have bigger size then if they have built without > > > LTO. For example 'kitty' package: > > > > > > * with LTO: > > > - koji task > > > https://koji.fedoraproject.org/koji/taskinfo?taskID=47762998 > > > 1.79 MB glfw-wayland.so > > > 1.99 MB glfw-x11.so > > > 4.78 MB fast_data_types.so > > > 8.56 MB total > > > > > > * no LTO > > > - koji > > > https://koji.fedoraproject.org/koji/taskinfo?taskID=47769854 > > > 1.65 MB glfw-wayland.so > > > 1.84 MB glfw-x11.so > > > 4.51 MB fast_data_types.so > > > 8.00 MB total > > > > > > Difference is 7%. What we should do in such case? Should we disable > > > LTO for such packages? Or there is still could be gains from faster > > > code execution speed? > > I'd tend to leave LTO on, but it's totally your call. Without > > looking at the > > binaries, sources and compiler dumps I'd hazard a guess you're > > getting a lot of > > cross module inlining, but very little identical code folding. THe > > former tends > > to make things bigger, but faster. The latter tends to shrink code > > with little > > impact on runtime performance. > > From what I see in this case, -ffat-lto-objects generates code that is > bigger than without -flto. -flto alone generates smaller code than > without -flto. The fat-lto-objects bits are not used during an LTO link. They exist solely to cover the case where there's a .o/.a that ends up installed. jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: The case of LTO when produced enlarged binaries
On Fri, 2020-07-24 at 22:30 +0200, Igor Raits wrote: > On Fri, 2020-07-24 at 14:27 -0600, Jeff Law wrote: > > On Fri, 2020-07-24 at 22:24 +0200, Igor Raits wrote: > > > On Fri, 2020-07-24 at 13:29 -0600, Jeff Law wrote: > > > > On Fri, 2020-07-24 at 19:12 +, Artem Tim wrote: > > > > > Hi. In rare cases building packages with LTO producing binaries > > > > > or > > > > > libraries which have bigger size then if they have built > > > > > without > > > > > LTO. For example 'kitty' package: > > > > > > > > > > * with LTO: > > > > > - koji task > > > > > https://koji.fedoraproject.org/koji/taskinfo?taskID=47762998 > > > > > 1.79 MB glfw-wayland.so > > > > > 1.99 MB glfw-x11.so > > > > > 4.78 MB fast_data_types.so > > > > > 8.56 MB total > > > > > > > > > > * no LTO > > > > > - koji > > > > > https://koji.fedoraproject.org/koji/taskinfo?taskID=47769854 > > > > > 1.65 MB glfw-wayland.so > > > > > 1.84 MB glfw-x11.so > > > > > 4.51 MB fast_data_types.so > > > > > 8.00 MB total > > > > > > > > > > Difference is 7%. What we should do in such case? Should we > > > > > disable > > > > > LTO for such packages? Or there is still could be gains from > > > > > faster > > > > > code execution speed? > > > > I'd tend to leave LTO on, but it's totally your call. Without > > > > looking at the > > > > binaries, sources and compiler dumps I'd hazard a guess you're > > > > getting a lot of > > > > cross module inlining, but very little identical code folding. > > > > THe > > > > former tends > > > > to make things bigger, but faster. The latter tends to shrink > > > > code > > > > with little > > > > impact on runtime performance. > > > > > > From what I see in this case, -ffat-lto-objects generates code that > > > is > > > bigger than without -flto. -flto alone generates smaller code than > > > without -flto. > > The fat-lto-objects bits are not used during an LTO link. They exist > > solely to > > cover the case where there's a .o/.a that ends up installed. > > Well, I tell what I see :) > > Compiling kitty with settings below produces this big > /usr/lib64/kitty/kitty/fast_data_types.so: > > * Without any LTO-related flags: 4.52 MB > * With -flto: 4.30 MB > * With -flto -ffat-lto-objects: 4.79 MB > > Well, I did not run compilation multiple times but don't think it will > change much. That's quite bizarre. I'll put it on the list of things to investigate. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Very strange compiler/linker related build failures in rawhide
On Fri, 2020-07-24 at 15:00 -0600, Jerry James wrote: > On Fri, Jul 24, 2020 at 2:49 PM Eric Sandeen wrote: > > Hm, is this related? (libtool segfault building xfsprogs) > > > > https://kojipkgs.fedoraproject.org//work/tasks/9149/47779149/build.log > > > > /bin/sh ../libtool --quiet --tag=CC --mode=link gcc -Wl,-z,relro > > -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld > > -Wl,-z,relro -Wl,--as-needed -Wl,-z,now > > -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -rpath /lib64 -version-info > > 0:0:0 -static -o libfrog.la avl64.lo bitmap.lo bulkstat.lo convert.lo > > crc32.lo fsgeom.lo list_sort.lo linux.lo logging.lo paths.lo projects.lo > > ptvar.lo radix-tree.lo scrub.lo topology.lo util.lo workqueue.lo > > ../libtool: line 1085: 137776 Segmentation fault (core dumped) ar cru > > .libs/libfrog.a avl64.o bitmap.o bulkstat.o convert.o crc32.o fsgeom.o > > list_sort.o linux.o logging.o paths.o projects.o ptvar.o radix-tree.o > > scrub.o topology.o util.o workqueue.o > > gmake[2]: *** [../include/buildrules:71: libfrog.la] Error 139 > > gmake[1]: *** [include/buildrules:36: libfrog] Error 2 > > make: *** [Makefile:92: default] Error 2 > > error: Bad exit status from /var/tmp/rpm-tmp.fcTM51 (%build) > > RPM build errors: > > Bad exit status from /var/tmp/rpm-tmp.fcTM51 (%build) > > Child return code was: 1 > > EXCEPTION: [Error()] > > Isn't that ar segfaulting? I just saw the same thing (an ar segfault) > while trying to do a mock build of z3 with --enablerepo=local. More likely the binutils update if that's gone in. Nick's probably already wrapped for the day. Jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: ar (binutils) segfaulting in Rawhide - known bug?
On Fri, 2020-07-24 at 22:29 +0100, Richard W.M. Jones wrote: > Just upgraded a development machine to: > > binutils-2.34.0-10.fc33.x86_64 > gcc-10.1.1-2.fc33.x86_64 > glibc-2.31.9000-21.fc33.x86_64 > > and a very simple C compile (non-LTO) is now segfaulting: > > make[3]: Entering directory '/home/rjones/d/nbdkit/common/protocol' > /bin/sh ../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. > -I../..-Wall -Wshadow -Wvla -Werror -O0 -g -Wp,-U_FORTIFY_SOURCE -MT > libprotocol_la-protostrings.lo -MD -MP -MF > .deps/libprotocol_la-protostrings.Tpo -c -o libprotocol_la-protostrings.lo > `test -f 'protostrings.c' || echo './'`protostrings.c > libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../.. -Wall -Wshadow -Wvla > -Werror -O0 -g -Wp,-U_FORTIFY_SOURCE -MT libprotocol_la-protostrings.lo -MD > -MP -MF .deps/libprotocol_la-protostrings.Tpo -c protostrings.c -fPIC -DPIC > -o .libs/libprotocol_la-protostrings.o > libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../.. -Wall -Wshadow -Wvla > -Werror -O0 -g -Wp,-U_FORTIFY_SOURCE -MT libprotocol_la-protostrings.lo -MD > -MP -MF .deps/libprotocol_la-protostrings.Tpo -c protostrings.c -o > libprotocol_la-protostrings.o >/dev/null 2>&1 > mv -f .deps/libprotocol_la-protostrings.Tpo > .deps/libprotocol_la-protostrings.Plo > /bin/sh ../../libtool --tag=CC --mode=link gcc -Wall -Wshadow -Wvla > -Werror -O0 -g -Wp,-U_FORTIFY_SOURCE -O0 -g -Wp,-U_FORTIFY_SOURCE -o > libprotocol.la libprotocol_la-protostrings.lo > libtool: link: ar cru .libs/libprotocol.a .libs/libprotocol_la-protostrings.o > ../../libtool: line 1734: 2572327 Segmentation fault (core dumped) ar > cru .libs/libprotocol.a .libs/libprotocol_la-protostrings.o > > Core was generated by `ar cru .libs/libprotocol.a > .libs/libprotocol_la-protostrings.o'. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 0x in ?? () > binutils-2.34.0-10.fc33.x86_64 > (gdb) bt > Missing separate debuginfos, use: dnf debuginfo-install#0 0x > in ?? () > #1 0x7f15bd3e03d0 in make_relative_prefix_1.part () >from /lib64/libbfd-2.34.0.20200522.so > #2 0x7f15bd3d22db in bfd_plugin_object_p.lto_priv () >from /lib64/libbfd-2.34.0.20200522.so > #3 0x7f15bd3401ce in bfd_check_format_matches () >from /lib64/libbfd-2.34.0.20200522.so > #4 0x7f15bd340e7a in _bfd_write_archive_contents () >from /lib64/libbfd-2.34.0.20200522.so > #5 0x7f15bd348b2a in bfd_close () from /lib64/libbfd-2.34.0.20200522.so > #6 0x559ee83994b6 in write_archive () > #7 0x559ee8396ac3 in main () > > I can't find any BZ for this. Any ideas what it could be? Hmm, what's interesting here is that it's binutils-2.34, so it's not the update that Nick was doing to do today. I've seen a couple folks trip over this today and just saw it in a couple of my builds. I'll take a look. I'm not much of a binutils hacker these days, but it's just code. jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: ar (binutils) segfaulting in Rawhide - known bug?
On Fri, 2020-07-24 at 22:40 +0100, Richard W.M. Jones wrote: > On Fri, Jul 24, 2020 at 03:37:05PM -0600, Jeff Law wrote: > > Hmm, what's interesting here is that it's binutils-2.34, so it's not > > the update that Nick was doing to do today. I've seen a couple > > folks trip over this today and just saw it in a couple of my builds. > > I believe it's the version that nickc just built in Rawhide this > afternoon. Yea, but I'm probably to blame :-) > > > I'll take a look. I'm not much of a binutils hacker these days, but > > it's just code. > > Even simpler reproducer ... > > $ ar cru test.a /dev/null > Segmentation fault (core dumped) Sweet. I'm on it. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: ar (binutils) segfaulting in Rawhide - known bug?
On Fri, 2020-07-24 at 22:40 +0100, Richard W.M. Jones wrote: > On Fri, Jul 24, 2020 at 03:37:05PM -0600, Jeff Law wrote: > > Hmm, what's interesting here is that it's binutils-2.34, so it's not > > the update that Nick was doing to do today. I've seen a couple > > folks trip over this today and just saw it in a couple of my builds. > > I believe it's the version that nickc just built in Rawhide this > afternoon. > > > I'll take a look. I'm not much of a binutils hacker these days, but > > it's just code. > > Even simpler reproducer ... > > $ ar cru test.a /dev/null > Segmentation fault (core dumped) But that's not triggering for me:( Let me try with the nbdkit bits jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: ar (binutils) segfaulting in Rawhide - known bug?
On Fri, 2020-07-24 at 17:55 -0400, Stephen John Smoogen wrote: > On Fri, 24 Jul 2020 at 17:51, Jeff Law wrote: > > On Fri, 2020-07-24 at 22:40 +0100, Richard W.M. Jones wrote: > > > On Fri, Jul 24, 2020 at 03:37:05PM -0600, Jeff Law wrote: > > > > Hmm, what's interesting here is that it's binutils-2.34, so it's not > > > > the update that Nick was doing to do today. I've seen a couple > > > > folks trip over this today and just saw it in a couple of my builds. > > > > > > I believe it's the version that nickc just built in Rawhide this > > > afternoon. > > > > > > > I'll take a look. I'm not much of a binutils hacker these days, but > > > > it's just code. > > > > > > Even simpler reproducer ... > > > > > > $ ar cru test.a /dev/null > > > Segmentation fault (core dumped) > > But that's not triggering for me:( Let me try with the nbdkit bits > > > > Would it help if the people having the problem gave a list of what > RPMs they have installed and what kernel they are running? That could > help cut down some of the guessing. No need. I've reproduced it with the nbdkit package. Proceeding to debugging. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: ar (binutils) segfaulting in Rawhide - known bug?
On Fri, 2020-07-24 at 17:55 -0400, Stephen John Smoogen wrote: > On Fri, 24 Jul 2020 at 17:51, Jeff Law wrote: > > On Fri, 2020-07-24 at 22:40 +0100, Richard W.M. Jones wrote: > > > On Fri, Jul 24, 2020 at 03:37:05PM -0600, Jeff Law wrote: > > > > Hmm, what's interesting here is that it's binutils-2.34, so it's not > > > > the update that Nick was doing to do today. I've seen a couple > > > > folks trip over this today and just saw it in a couple of my builds. > > > > > > I believe it's the version that nickc just built in Rawhide this > > > afternoon. > > > > > > > I'll take a look. I'm not much of a binutils hacker these days, but > > > > it's just code. > > > > > > Even simpler reproducer ... > > > > > > $ ar cru test.a /dev/null > > > Segmentation fault (core dumped) > > But that's not triggering for me:( Let me try with the nbdkit bits > > > > Would it help if the people having the problem gave a list of what > RPMs they have installed and what kernel they are running? That could > help cut down some of the guessing. What would help would be if someone could untag that version of binutils so that it doesn't show up in the buildroots anymore. It's clearly fubar'd. What exceedingly weird is it looks like like we've got a call through the PLT to a routine that should be defined, but the PLT entry is zero. Naturally that causes bad things to happen. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: ar (binutils) segfaulting in Rawhide - known bug?
On Fri, 2020-07-24 at 22:29 +0100, Richard W.M. Jones wrote: > Just upgraded a development machine to: > > binutils-2.34.0-10.fc33.x86_64 > gcc-10.1.1-2.fc33.x86_64 > glibc-2.31.9000-21.fc33.x86_64 > > and a very simple C compile (non-LTO) is now segfaulting: > > make[3]: Entering directory '/home/rjones/d/nbdkit/common/protocol' > /bin/sh ../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. > -I../..-Wall -Wshadow -Wvla -Werror -O0 -g -Wp,-U_FORTIFY_SOURCE -MT > libprotocol_la-protostrings.lo -MD -MP -MF > .deps/libprotocol_la-protostrings.Tpo -c -o libprotocol_la-protostrings.lo > `test -f 'protostrings.c' || echo './'`protostrings.c > libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../.. -Wall -Wshadow -Wvla > -Werror -O0 -g -Wp,-U_FORTIFY_SOURCE -MT libprotocol_la-protostrings.lo -MD > -MP -MF .deps/libprotocol_la-protostrings.Tpo -c protostrings.c -fPIC -DPIC > -o .libs/libprotocol_la-protostrings.o > libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../.. -Wall -Wshadow -Wvla > -Werror -O0 -g -Wp,-U_FORTIFY_SOURCE -MT libprotocol_la-protostrings.lo -MD > -MP -MF .deps/libprotocol_la-protostrings.Tpo -c protostrings.c -o > libprotocol_la-protostrings.o >/dev/null 2>&1 > mv -f .deps/libprotocol_la-protostrings.Tpo > .deps/libprotocol_la-protostrings.Plo > /bin/sh ../../libtool --tag=CC --mode=link gcc -Wall -Wshadow -Wvla > -Werror -O0 -g -Wp,-U_FORTIFY_SOURCE -O0 -g -Wp,-U_FORTIFY_SOURCE -o > libprotocol.la libprotocol_la-protostrings.lo > libtool: link: ar cru .libs/libprotocol.a .libs/libprotocol_la-protostrings.o > ../../libtool: line 1734: 2572327 Segmentation fault (core dumped) ar > cru .libs/libprotocol.a .libs/libprotocol_la-protostrings.o > > Core was generated by `ar cru .libs/libprotocol.a > .libs/libprotocol_la-protostrings.o'. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 0x in ?? () > binutils-2.34.0-10.fc33.x86_64 > (gdb) bt > Missing separate debuginfos, use: dnf debuginfo-install#0 0x > in ?? () > #1 0x7f15bd3e03d0 in make_relative_prefix_1.part () >from /lib64/libbfd-2.34.0.20200522.so > #2 0x7f15bd3d22db in bfd_plugin_object_p.lto_priv () >from /lib64/libbfd-2.34.0.20200522.so > #3 0x7f15bd3401ce in bfd_check_format_matches () >from /lib64/libbfd-2.34.0.20200522.so > #4 0x7f15bd340e7a in _bfd_write_archive_contents () >from /lib64/libbfd-2.34.0.20200522.so > #5 0x7f15bd348b2a in bfd_close () from /lib64/libbfd-2.34.0.20200522.so > #6 0x559ee83994b6 in write_archive () > #7 0x559ee8396ac3 in main () > > I can't find any BZ for this. Any ideas what it could be? After banging my head on the wall for a few hours, I think I see what's happening here. So at a high level ar makes a call to lrealpath. That naturally goes through the PLT. The PLT stub loads the value out of the GOT and jumps to it. The problem is the entry in the GOT is *zero* when it should be pointing to the resolver. Now lrealpath is provided by libiberty and a copy is in libbfd.so and the GOT entry in libbfd.so looked right. But by the time the program has hit main, the GOT entry has been reset to zero. Naturally that's happening inside the dynamic linker (as expected, confirmed with a watchpoint). If you've ever had to debug ld.so, you'll know it's an insanely painful experience, but it proved fruitful. The key was finding out that we were not using the libbfd.so linker map to resolve lrealpath, instead we were using the linker map for the main program (ar in this case). So natrually it's time to look a bit more closely at the symbol table for ar. The main symbol table for ar it doesn't mention lrealpath. But that's just a confusing byproduct of having two symbol tables. What matters to ld.so is the *dynamic* symbol table. And ar has lrealpath in its dynamic symbol table. And here's the kicker, it's an absolute symbol with the value 0: A lrealpath A symbol in the main program takes precedence over a symbol in a DSO. So the dynamic linker was actually doing the right thing given the input it was provided. Now why (*&@#$ does ar have lrealpath as an absolute symbol? It's got to be related to the fact that when we link ar we pull in another copy of libiberty. In fact, ar links against libiberty twice. Once via -liberty then again against libiberty.a (and kindof a 3rd time indirectly via libbfd). BUt even so that shouldn't be creating an absolute symbol. That's just weird. This smells like a linker bug to me. Not surprisingly if I force the system to use ld.gold, then I don't see the bogus absolute symbol and the resultant ar works just fine. It's late and I'll dig further over the weekend, but right now this looks like a linker bug to me. I may turn off LTO globally or in the various instances of binutils -- I need to sleep on that. Jeff ___ devel mailing list -- devel@lists.fed
Re: We have to talk about annobin... again
On Sat, 2020-07-25 at 10:46 +0100, Peter Robinson wrote: > On Sat, Jul 25, 2020 at 10:35 AM Neal Gompa wrote: > > Hey all, > > > > So I was trying to update libseccomp last night, and I was able to > > build it for everything except aarch64 on Rawhide because it says the > > compiler can't build executables[1]. > > > > Looking a bit closer, it looks like the compiler stack is out of sync > > again with annobin. > > > > Is there anything that can be done to keep the compiler teams from > > submitting gcc into rawhide without doing the required rebuild cycle > > to make it so annobin works? > > > > And we're going to have the same problem with clang now that annobin > > grew a clang plugin, so I would want neither LLVM nor GCC to land in > > Rawhide unless those teams are literally ensuring that annobin isn't > > breaking the compiler afterward. > > > > I'm personally very tired of having the compiler break so frequently > > because of that plugin. Either some kind of mechanism to hold back GCC > > builds until annobin works is implemented, or I'd much rather see the > > whole thing go away. Obviously, you could just *bundle* annobin into > > the GCC package and build it together to ensure it never broke, but > > that option was discarded already[2]. > > > > Somebody fix it. ASAP. > > I've kicked off an annobin build for the short term issue. > > I agree here, we used to have the same issue with gcc-plugins back in > the day, sadly this is nothing new :-/ :( I'll raise it again with Nick and Jakub, it's a sore point for everyone I think and it causes far more friction than just what we see here in Fedora. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: ar (binutils) segfaulting in Rawhide - known bug?
On Sun, 2020-07-26 at 09:39 -0600, Jerry James wrote: > On Fri, Jul 24, 2020 at 6:41 PM Kevin Fenzi wrote: > > On Fri, Jul 24, 2020 at 04:55:31PM -0600, Jeff Law wrote: > > > What would help would be if someone could untag that version of binutils > > > so that > > > it doesn't show up in the buildroots anymore. It's clearly fubar'd. > > > > Done. > > H. Yet my most recent build attempt, just now, failed with a > linker segfault on all arches: > > https://koji.fedoraproject.org/koji/buildinfo?buildID=1546752 > > This is with: > annobin-9.24.2-fc33 > binutils-2.35-1.fc33 > gcc-10.2.1-1.fc33 > glibc-2.31.9000-21.fc33 Urggh. Probably the same problem showing up in the 2.35 build which must have recently landed. We need that untagged too. jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: ar (binutils) segfaulting in Rawhide - known bug?
On Sun, 2020-07-26 at 09:39 -0600, Jerry James wrote: > On Fri, Jul 24, 2020 at 6:41 PM Kevin Fenzi wrote: > > On Fri, Jul 24, 2020 at 04:55:31PM -0600, Jeff Law wrote: > > > What would help would be if someone could untag that version of binutils > > > so that > > > it doesn't show up in the buildroots anymore. It's clearly fubar'd. > > > > Done. > > H. Yet my most recent build attempt, just now, failed with a > linker segfault on all arches: > > https://koji.fedoraproject.org/koji/buildinfo?buildID=1546752 > > This is with: > annobin-9.24.2-fc33 > binutils-2.35-1.fc33 > gcc-10.2.1-1.fc33 > glibc-2.31.9000-21.fc33 As Kevin mentioned in a followup, he's untagged the 2.35 build so this should be working again. I think I see the root cause in the linker now. It's probably an uncommon scenario, but I doubt binutils is the only affected package. The even better news is I think we can go ahead and green light the mass rebuild for Monday. Two reasons. One, I expect the preconditions necessary to trip the bug to be uncommon. Two, I think we can reliably detect a broken binary by the existence of absolute symbols in the dynamic symbol table. The latter in particular means we've got a method where we can find affected packages while Nick and I iterate on the linker fix. So even if the bug leaks into packages, we can find them and do targeted rebuilds. I'll find the fesco issue and add some notes there along with my recommendation. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: [pam_radius] aarch64 GCC failures during ./configure's working compiler step?
On Mon, 2020-07-27 at 08:39 -0400, Alex Scheel wrote: > - Original Message - > > From: "Dan Čermák" > > To: "Alex Scheel" , "Development discussions related to > > Fedora" > > Sent: Monday, July 27, 2020 8:25:27 AM > > Subject: Re: [pam_radius] aarch64 GCC failures during ./configure's working > > compiler step? > > > > Hi Alex, > > > > > > This is most likely due to annobin being broken in rawhide on aarch64 > > (see Neal's email to devel from Saturday). > > Ah, thanks Dan and Peter! I must've missed it for another thread. I'll > be patient. :D I just did a scratch build of pam_radius which seems to have worked fine for aarch64. Presumably that means all the buildroots have updated :-) jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: ar (binutils) segfaulting in Rawhide - known bug?
On Mon, 2020-07-27 at 13:32 +, Zbigniew Jędrzejewski-Szmek wrote: > On Sun, Jul 26, 2020 at 11:03:58PM -0600, Jeff Law wrote: > > On Sun, 2020-07-26 at 09:39 -0600, Jerry James wrote: > > > On Fri, Jul 24, 2020 at 6:41 PM Kevin Fenzi wrote: > > > > On Fri, Jul 24, 2020 at 04:55:31PM -0600, Jeff Law wrote: > > > > > What would help would be if someone could untag that version of > > > > > binutils so that > > > > > it doesn't show up in the buildroots anymore. It's clearly fubar'd. > > > > > > > > Done. > > > > > > H. Yet my most recent build attempt, just now, failed with a > > > linker segfault on all arches: > > > > > > https://koji.fedoraproject.org/koji/buildinfo?buildID=1546752 > > > > > > This is with: > > > annobin-9.24.2-fc33 > > > binutils-2.35-1.fc33 > > > gcc-10.2.1-1.fc33 > > > glibc-2.31.9000-21.fc33 > > As Kevin mentioned in a followup, he's untagged the 2.35 build so this > > should be > > working again. > > > > I think I see the root cause in the linker now. It's probably an uncommon > > scenario, but I doubt binutils is the only affected package. > > > > The even better news is I think we can go ahead and green light the mass > > rebuild > > for Monday. Two reasons. One, I expect the preconditions necessary to > > trip the > > bug to be uncommon. Two, I think we can reliably detect a broken binary by > > the > > existence of absolute symbols in the dynamic symbol table. > > > > The latter in particular means we've got a method where we can find affected > > packages while Nick and I iterate on the linker fix. So even if the bug > > leaks > > into packages, we can find them and do targeted rebuilds. > > The problem with that is that if broken builds land in the buildroot of > other packages, those dependent packages might either a) fail to build, > b) be built incorrectly, for example because feature detection fails. > Situation a) happens in mass rebuilds quite a lot anyway, so it's not > a big issue, since the build would just be repeated. But b) is more serious. > Even if you detect that a package was faulty and needs to be rebuilt, > we might have to also rebuild all packages using that faulty package > as a build dependency, recursively. This quickly becomes messy :( I'm aware of that potential. I think the odds of stumbling into this are small. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: ar (binutils) segfaulting in Rawhide - known bug?
On Mon, 2020-07-27 at 18:20 +0200, Nikola Forró wrote:
> On Sat, 2020-07-25 at 01:11 -0600, Jeff Law wrote:
> > So at a high level ar makes a call to lrealpath. That naturally goes
> > through the
> > PLT. The PLT stub loads the value out of the GOT and jumps to it. The
> > problem
> > is the entry in the GOT is *zero* when it should be pointing to the
> > resolver.
> >
> > Now lrealpath is provided by libiberty and a copy is in libbfd.so and the
> > GOT
> > entry in libbfd.so looked right. But by the time the program has hit main,
> > the
> > GOT entry has been reset to zero. Naturally that's happening inside the
> > dynamic
> > linker (as expected, confirmed with a watchpoint). If you've ever had to
> > debug
> > ld.so, you'll know it's an insanely painful experience, but it proved
> > fruitful.
> >
> > The key was finding out that we were not using the libbfd.so linker map to
> > resolve lrealpath, instead we were using the linker map for the main
> > program (ar
> > in this case). So natrually it's time to look a bit more closely at the
> > symbol
> > table for ar.
> >
> > The main symbol table for ar it doesn't mention lrealpath. But that's just
> > a
> > confusing byproduct of having two symbol tables. What matters to ld.so is
> > the
> > *dynamic* symbol table. And ar has lrealpath in its dynamic symbol table.
> > And
> > here's the kicker, it's an absolute symbol with the value 0:
> >
> > A lrealpath
> >
> > A symbol in the main program takes precedence over a symbol in a DSO. So
> > the
> > dynamic linker was actually doing the right thing given the input it was
> > provided.
> >
> > Now why (*&@#$ does ar have lrealpath as an absolute symbol? It's got to be
> > related to the fact that when we link ar we pull in another copy of
> > libiberty.
> > In fact, ar links against libiberty twice. Once via -liberty then again
> > against
> > libiberty.a (and kindof a 3rd time indirectly via libbfd). BUt even so that
> > shouldn't be creating an absolute symbol. That's just weird.
> >
> > This smells like a linker bug to me. Not surprisingly if I force the
> > system to
> > use ld.gold, then I don't see the bogus absolute symbol and the resultant ar
> > works just fine.
> >
> > It's late and I'll dig further over the weekend, but right now this looks
> > like a
> > linker bug to me. I may turn off LTO globally or in the various instances
> > of
> > binutils -- I need to sleep on that.
>
> I'm seeing the same behavior with man-db, more specifically with accessdb
> linking to libmandb:
>
> $ nm -D accessdb | grep xmalloc
> A xmalloc
>
> Obviously it segfaults, unless I disable LTO.
>
> Is there a bugzilla for that linker bug?
I don't think so. Nick was trying to pull together a simpler testcase and open
a
discussion with the other binutils developers on a path forward. He's aware of
the impacts, so I'm sure he's working diligently on it.
In the immediate term, disabling LTO seems reasonable.
%define _lto_cflags %{nil}
We're going to go through all the opt-outs at some point after the mass rebuild,
so we can re-enable once the ld bug is fixed.
jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: z3 soname bump
On Fri, 2020-07-24 at 13:39 -0600, Jerry James wrote:
> On Fri, Jul 24, 2020 at 1:35 PM Jeff Law wrote:
> > Just a note on z3.
> >
> > I've been trying to track down what I think is an uninstantiated template
> > issue
> > that's exposed by LTO. I've been chasing it on/off over the last day or two
> > without success. So if you get a build failure that looks like this:
> >
> > /usr/bin/ld: /tmp/z3.16EntH.ltrans1.ltrans.o:(.data.rel.ro+0x110):
> > undefined reference to `lp::lp_solver > double>::get_variable_name[abi:cxx11](unsigned int) const'
> > /usr/bin/ld: /tmp/z3.16EntH.ltrans1.ltrans.o:(.data.rel.ro+0x168):
> > undefined reference to `lp::lp_solver > double>::get_variable_name[abi:cxx11](unsigned int) const'
> > /usr/bin/ld: /tmp/z3.16EntH.ltrans1.ltrans.o:(.data.rel.ro+0x1b0):
> > undefined reference to `lp::lp_solver > double>::get_variable_name[abi:cxx11](unsigned int) const'
> > collect2: error: ld returned 1 exit status
> >
> > Just disable LTO via the usual mechanism (%define _lto_cflags %{nil}).
> > We'll
> > revisit any opt-outs again between F33 and F34 and re-evaluate them.
>
> Thanks for the heads-up, Jeff. I have not seen that in a mock build,
> but if there hasn't been a successful Rawhide compose since the LTO
> bits landed, that is no surprise. I have a vague memory of asking z3
> upstream to add some explicit template instantiations around the time
> gcc 10 landed in Rawhide, so there is precedent if we need to do so
> again.
Whatever you committed in the last couple days seems to have fixed the issue. I
just got a clean build of z3 on x86_64.
jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 32 aarch64 build failures on copr
On Tue, 2020-07-28 at 14:51 -0500, Brandon Nielsen wrote:
> I've been seeing build failures for Fedora 32[0][1][2], always the same
> failure: "xgcc: fatal error: Killed signal terminated program cc1plus"
>
> I can't find any more detail than that. These builds succeed locally
> with mock. The copr failures are reproducible so I'm sure I'm doing
> something wrong, but I can't figure out what. Any ideas?
>
> [0] -
> https://copr.fedorainfracloud.org/coprs/nielsenb/msp430-development-tools/build/1577708/
> [1] -
> https://copr.fedorainfracloud.org/coprs/nielsenb/msp430-development-tools/build/1565812/
> [2] -
> https://copr.fedorainfracloud.org/coprs/nielsenb/msp430-development-tools/build/1565363/
If this is a new failure (say in the last week), it could be an out of memory
scenario. Try disabling LTO. The standard way to do that is
%define _lto_cflags %{nil}
In your %build stanza in the spec file.
Heff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 32 aarch64 build failures on copr
On Tue, 2020-07-28 at 15:29 -0500, Michael Catanzaro wrote:
> On Tue, Jul 28, 2020 at 2:01 pm, Jeff Law wrote:
> > If this is a new failure (say in the last week), it could be an out
> > of memory
> > scenario. Try disabling LTO. The standard way to do that is
> >
> > %define _lto_cflags %{nil}
> >
> > In your %build stanza in the spec file.
> >
> > Heff
>
> I agree, it's almost certainly OOM because it says "fatal error:
> Killed." I've never seen that happen for any reason other than OOM.
I've seen it happen for a variety of reasons. Please test with LTO disabled and
let me know if that helps.
jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: DWARF version 0 unhandled
On Tue, 2020-07-28 at 16:26 -0600, Jerry James wrote: > Here's a bit of fallout from the mass rebuild. The alt-ergo build failed: > > https://koji.fedoraproject.org/koji/buildinfo?buildID=1548139 [ ... ] You might try without LTO. I'm just shooting in the dark, but LTO does have a significant impact on how we generate debuginfo as well as impacting the structure of the resultant debug info. While I haven't seen anything like what you've shown it can't hurt to at least see if LTO is playing a role here. jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Disable LTO for now ?
On Wed, 2020-07-29 at 14:13 +0100, ser...@serjux.com wrote: > Hello opencv [1] build also failed around LTO > What is your advise ? What is your advice? In general I want to have a very good indicator the issue is LTO related before I disable. THe build you referenced doesn't have any good indicators that LTO is the problem. For ppc64le is that the build was done with p8, but there is one function (__builtin_altivec_vadub) that requires p9. This seems like a package bug at first glance, not an LTO issue. I would try a ppc64le test build with and without LTO, my guess is both are going to fail with similar problems. For s390 I would guess that it got aborted -- there's nothing useful in the build.log. So I'd try an LTO test build of that. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: No debugsource generated, weird DWARF errors
On Wed, 2020-07-29 at 13:15 +0100, Richard W.M. Jones wrote: > https://koji.fedoraproject.org/koji/taskinfo?taskID=48101965 fails > with: > > error: Empty %files file > /builddir/build/BUILD/hevea-2.34/debugsourcefiles.list > > However it works when I build locally: > > $ rpm -qlp > /home/rjones/d/fedora/hevea/master/x86_64/hevea-debugsource-2.34-7.fc33.x86_64.rpm > /usr/src/debug/hevea-2.34-7.fc33.x86_64 > /usr/src/debug/hevea-2.34-7.fc33.x86_64/_build > /usr/src/debug/hevea-2.34-7.fc33.x86_64/_build/auxx.ml > /usr/src/debug/hevea-2.34-7.fc33.x86_64/_build/bibhva.ml > /usr/src/debug/hevea-2.34-7.fc33.x86_64/_build/color.ml > [etc] > > There's an earlier error which occurs in the Koji build which doesn't > occur for me locally but looks related: > > Dwarf Error: wrong version in compilation unit header (is 0, should be 2, > 3, 4 or 5) [in module > /builddir/build/BUILDROOT/hevea-2.34-7.fc33.x86_64/usr/bin/hacha] > gdb-add-index: No index was created for > /builddir/build/BUILDROOT/hevea-2.34-7.fc33.x86_64/usr/bin/hacha > gdb-add-index: [Was there no debuginfo? Was there already an index?] > > Did something change with DWARF or gdb in Rawhide that we should be > aware of? Obviously these are not C code and OCaml has its own DWARF > generator. I think mjw is tracking something like this yesterday, but it was aarch64. I'm going to be offline for a couple hours, but you might want to reach out to him and see if there's any useful state (on cc) Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: ceph -> qemu -> lots failing in the mass rebuild
On Wed, 2020-07-29 at 09:16 -0700, Kevin Fenzi wrote: > On Wed, Jul 29, 2020 at 02:40:18PM +0100, Richard W.M. Jones wrote: > > qemu is uninstallable at the moment because ceph is uninstallable > > because fmt was upgraded from 6 to 7 in the middle of the build > > (resulting in an soname bump - it was announced). > > > > There are quite a few things failing in the mass rebuild as a result. > > > > I've kicked off a new build of ceph, and will follow up with qemu if > > that's necessary, but note that ceph takes ~ 24 hours to build, so > > it'll be a while ... > > We are planning on doing a resubmit on all the mass rebuild failures > after the mass rebuild is done. Obviously we should wait for this too. Thanks for doing that. I was trying to walk through the existing failures looking for LTO issues (and even found a few), but there's so much noise from the s390 and cmake issues that it doesn't seem worth the time until that second round of builds has started. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: s390 builds failing with "All mirrors were tried"
On Wed, 2020-07-29 at 09:19 -0700, Kevin Fenzi wrote: > On Wed, Jul 29, 2020 at 10:31:32AM -0400, Scott Talbert wrote: > > On Wed, 29 Jul 2020, Richard W.M. Jones wrote: > > > > > libnbd failed in the mass rebuild. I kicked off a second build by > > > hand, and it failed in the exact same way: > > > > > > https://koji.fedoraproject.org/koji/taskinfo?taskID=48118058 > > > > > > DEBUG util.py:623: Downloading Packages: > > > DEBUG util.py:621: Error: Error downloading packages: > > > DEBUG util.py:621:Cannot download > > > toplink/packages/ocaml/4.11.0/0.4.dev2.fc33/s390x/ocaml-compiler-libs-4.11.0-0.4.dev2.fc33.s390x.rpm: > > > All mirrors were tried > > > > > > Is this a generic s390 problem of is there really something wrong with > > > the ocaml-compiler-libs package? If it's generic then I guess this > > > sort of error could be affecting a lot of other packages in the mass > > > rebuild. > > > > Seems to be a generic s390x problem (with networking / connectivity?). At > > least one of my packages failed in a similar was on s390x. > > Yes, the local to the s390x builders cache had issues last night. > I disabled all builders and rebooted it, but likely it failed a lot of > builds. > > Additionally, it seems like from time to time it just gets swamped. > > We are planning on resubmitting the failed builds after the mass rebuild > is complete. Or you can feel free to resubmit any of yours that failed > in this way before then. > > I already have some ideas on how to make the s390x setup more robust, > but I'm not going to make changes in the middle of the mass rebuild, but > hopefully that will help for the next one. That reminds me. My local testing has shown that -flto=auto works reasonably well and can help significantly with build times (it parallelizes the compiler invocations that occur at link time). It'd planned to enable it after the mass rebuild, but I find myself wondering if I should go ahead and update redhat-rpm-config now so that the second pass over the packages will be faster. Thoughts? jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 32 aarch64 build failures on copr
On Wed, 2020-07-29 at 14:24 -0500, Brandon Nielsen wrote:
> On 7/28/20 4:39 PM, Jeff Law wrote:
> > On Tue, 2020-07-28 at 15:29 -0500, Michael Catanzaro wrote:
> > > On Tue, Jul 28, 2020 at 2:01 pm, Jeff Law wrote:
> > > > If this is a new failure (say in the last week), it could be an out
> > > > of memory
> > > > scenario. Try disabling LTO. The standard way to do that is
> > > >
> > > > %define _lto_cflags %{nil}
> > > >
> > > > In your %build stanza in the spec file.
> > > >
> > > > Heff
> > >
> > > I agree, it's almost certainly OOM because it says "fatal error:
> > > Killed." I've never seen that happen for any reason other than OOM.
> > I've seen it happen for a variety of reasons. Please test with LTO
> > disabled and
> > let me know if that helps.
> >
> > jeff
>
> Disabling LTO with '%define _lto_cflags %{nil}' fixed it.
ACK. I don't see msp430-development-tools in the standard fedora repos. So
I'll
leave it to you to fix the package in whatever repo it lives in.
Also note, you may ultimately be better off getting msp430 added to the cross-
{binutils,gcc} packages rather than creating a new package.
Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Disable LTO for now ?
On Wed, 2020-07-29 at 20:39 +0100, Sérgio Basto wrote: > On Wed, 2020-07-29 at 12:19 -0500, Steven Munroe wrote: > > Jeff Law wrote: > > > > > For ppc64le is that the build was done with p8, but there is one > > > function (__builtin_altivec_vadub) that requires p9. This seems > > > like a package bug at first glance, not an LTO issue. > > > > Yup __builtin_altivec_vadub is POWER9_VECTOR only. > > > > For P8/6 use something like this: > > > > vmin = vec_min (a, b); > > vmax = vec_max (a, b); > > result = vec_sub (vmax, vmin); > > > > executes in 4 cycles. > > but why it built 18 day ago, opencv-4.3.0-7 [2], > with same source code and rpm spec [1] ? All kinds of possibilities and speculating wouldn't be terribly helpful here. What remains is that there's code in there that requires p9, but the build is specifying p8 only and thus you get an error. jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 32 aarch64 build failures on copr
On Wed, 2020-07-29 at 17:14 -0500, Brandon Nielsen wrote:
> On 7/29/20 4:40 PM, Jeff Law wrote:
> > ACK. I don't see msp430-development-tools in the standard fedora repos.
> > So I'll
> > leave it to you to fix the package in whatever repo it lives in.
> >
> > Also note, you may ultimately be better off getting msp430 added to the
> > cross-
> > {binutils,gcc} packages rather than creating a new package.
> >
> > Jeff
>
> It's a work in progress[0].
>
> As for cross-gcc and friends, the description notes those are for
> building kernels only, is that not the case?
It shouldn't matter for cross-binutils and cross-gcc.
>
> There's also some vanilla upstream / TI upstream compatibility weirdness
> to consider...
That issue should be resolved by getting those patches upstreamed to the GCC
project. Your best contact point for that would be Jozef Lawrynowicz <
joze...@mittosystems.com>
Jeff
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Disabled LTO on qemu
On Thu, 2020-07-30 at 11:06 +0100, Richard W.M. Jones wrote: > I just disabled LTO for qemu. > > It caused what are best described as "weird" assert failures > in the test suite. > > For comparison here's a good build (without LTO): > https://koji.fedoraproject.org/koji/taskinfo?taskID=48188577 > > And here's a bad build (with LTO): > https://koji.fedoraproject.org/koji/taskinfo?taskID=48182910 > > At some point we'll need to look at what's going on, but right now > disabling LTO was the easiest way to work around what might be > toolchain bugs. Agreed 100%. qemu was one of the dozen or so things left on my "to review" list that I couldn't get to with the binutils panic Fri/Sat. I suspect there's an asm gone awry, but we won't know for sure until we dig deeply into it. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Can we use emulation of other architectures to run integration tests?
On Thu, 2020-07-30 at 13:48 +0200, Nikos Mavrogiannopoulos wrote: > On Thu, Jul 30, 2020 at 12:25 PM Aleksandra Fedorova > wrote: > > Hi, all, > > > > I'd like to get some understanding on the current state of emulation > > of other architectures. > > > > In the current CI infra we have infinite(*) access to x86_64 compute > > resources, but we haven't yet got our hands on any non x86_64 > > hardware. > > > > As COPR has recently got support for s390 builds, the question is: if > > emulation is good enough for building packages, can we use it for > > testing? What are the limitations there? Is it worth it? > > Few years ago we transformed the gnutls' upstream CI from baremetal > h/w to qemu-user [0] (reasoning was pretty much what you mention, we > had x86-64 systems for free, and we had to pay for everything else). > This eliminated the need for such dedicated hardware, and in practice > the years it was in use I believe it eliminated issues in > compatibility with non-x86-64 architectures and also helped catch > problems in new code (such as alignment issues). For an upstream test > suite it was totally worth it, and I believe it eliminated all issues > we were getting with non-x86 hardware support. The only drawback that > was noticed is that it could not be used to test some special features > of these CPUs, but that's also a problem with dedicated hardware > (e.g., when it doesn't support the particular instruction set you'd > like to introduce). We use it heavily in the GCC upstream tester for ancient targets (think m68k, hppa, alpha and the like). It works amazingly well. Interestingly enough I've found those old targets to work better than, say, s390. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: s390 builds failing with "All mirrors were tried"
On Wed, 2020-07-29 at 10:11 -0700, Kevin Fenzi wrote: > On Wed, Jul 29, 2020 at 10:25:24AM -0600, Jeff Law wrote: > > On Wed, 2020-07-29 at 09:19 -0700, Kevin Fenzi wrote: > > > On Wed, Jul 29, 2020 at 10:31:32AM -0400, Scott Talbert wrote: > > > > On Wed, 29 Jul 2020, Richard W.M. Jones wrote: > > > > > > > > > libnbd failed in the mass rebuild. I kicked off a second build by > > > > > hand, and it failed in the exact same way: > > > > > > > > > > https://koji.fedoraproject.org/koji/taskinfo?taskID=48118058 > > > > > > > > > > DEBUG util.py:623: Downloading Packages: > > > > > DEBUG util.py:621: Error: Error downloading packages: > > > > > DEBUG util.py:621:Cannot download > > > > > toplink/packages/ocaml/4.11.0/0.4.dev2.fc33/s390x/ocaml-compiler-libs-4.11.0-0.4.dev2.fc33.s390x.rpm: > > > > > All mirrors were tried > > > > > > > > > > Is this a generic s390 problem of is there really something wrong with > > > > > the ocaml-compiler-libs package? If it's generic then I guess this > > > > > sort of error could be affecting a lot of other packages in the mass > > > > > rebuild. > > > > > > > > Seems to be a generic s390x problem (with networking / connectivity?). > > > > At > > > > least one of my packages failed in a similar was on s390x. > > > > > > Yes, the local to the s390x builders cache had issues last night. > > > I disabled all builders and rebooted it, but likely it failed a lot of > > > builds. > > > > > > Additionally, it seems like from time to time it just gets swamped. > > > > > > We are planning on resubmitting the failed builds after the mass rebuild > > > is complete. Or you can feel free to resubmit any of yours that failed > > > in this way before then. > > > > > > I already have some ideas on how to make the s390x setup more robust, > > > but I'm not going to make changes in the middle of the mass rebuild, but > > > hopefully that will help for the next one. > > That reminds me. My local testing has shown that -flto=auto works > > reasonably > > well and can help significantly with build times (it parallelizes the > > compiler > > invocations that occur at link time). > > > > It'd planned to enable it after the mass rebuild, but I find myself > > wondering if > > I should go ahead and update redhat-rpm-config now so that the second pass > > over > > the packages will be faster. > > > > Thoughts? > > If all it does is parallelize, I'd say enable it anytime... Done. jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: super-drafty F28 and F29 schedules
On 07/12/2017 06:10 AM, Jakub Jelinek wrote: > On Tue, Jul 11, 2017 at 09:36:28PM -0400, Carlos O'Donell wrote: >> On 07/06/2017 09:15 PM, Matthew Miller wrote: >>> https://fedoraproject.org/wiki/Releases/28/Schedule >> >> I encourage Jeff Law and Jakub Jelinek to review these schedules >> for compiler related issues. >> >> This is just a perfunctory review from the glibc perspective with >> regard to base ABI and API issues in this core runtime. >> >> 2018-01-10 MASS REBUILD >> - This date is ahead of the glibc release on 2018-02-01 >> but after the 2018-01-01 ABI freeze date for glibc. >> Therefore you will be largely assured a stable ABI upon >> which to base Fedora, _but_ there is a vanishingly small >> chance you may need a mass rebuild or a targeted rebuild >> on 2018-02-01 if something has to be reverted. > > 2018-01-10 is way too early for a mass rebuild from GCC point of view, > even if we perform the test mass rebuild over the Christmas break, > there won't be enough time to analyze it and fix any GCC issues revealed > during that time. Right. Optimistically, we'd let the test mass rebuild run over the break giving the GCC team a nice pile of issues to look at in the start of the new year. But even in that scenario I can't in good conscience recommend Fedora start mass rebuilds on 01-10 -- it's just not enough time to triage the issues from the mass rebuild and get them fixed. 01-25 is probably the earliest I'd recommend scheduling a mass rebuild from GCC's point of view. Even that may be overly optimistic. GCC bugfixing usually starts around November 15th and > needs some time before it can be fixed enough to use widely. > Moving the mass rebuild one week earlier than last time > is maybe doable, but not one month. And not rebuilding F27 with GCC 8 > is a bad idea for both the compiler and Fedora. FWIW I would like to see everything in the GCC process shift earlier by a couple weeks. I haven't had much success convincing folks to make that shift though. > > As for F29, there is no GCC release around that time, so from GCC POV the > schedule doesn't matter that much. Right. The odd Fedora releases tend to just pick up whatever the latest point release of GCC is available. They don't tend to cause much work or stress from a toolchain standpoint. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: super-drafty F28 and F29 schedules
On 07/12/2017 02:54 PM, Jakub Jelinek wrote: > On Wed, Jul 12, 2017 at 04:45:56PM -0400, Matthew Miller wrote: >> On Wed, Jul 12, 2017 at 04:30:12PM -0400, Matthew Miller wrote: >>> So, "one week earlier than last time" would be January 31st. (Or 30th? >>> Depends if we want that on a Tuesday like everything else or Wednesday >>> like this time around, if it matters.) Is that enough to help? Let me >>> rework it and let's see what that looks like... >> >> So, what does updated >> https://fedoraproject.org/wiki/Releases/28/Schedule#Key_Milestones look >> like to you? That puts the mass rebuild on January 31st. > > LGTM. Likewise. We should keep this schedule in mind as we work our way through stage3 into stage4. Ideally Marek would start the test builds prior to the Christmas break, even if we're still in stage3. We'll review on the 9th and 16th with go/no-go decisions on the 23rd and 30th. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: super-drafty F28 and F29 schedules
On 07/13/2017 09:01 AM, Matthew Miller wrote: > On Wed, Jul 12, 2017 at 03:14:24PM -0600, Jeff Law wrote: >>> LGTM. >> Likewise. We should keep this schedule in mind as we work our way >> through stage3 into stage4. Ideally Marek would start the test builds >> prior to the Christmas break, even if we're still in stage3. >> >> We'll review on the 9th and 16th with go/no-go decisions on the 23rd and >> 30th. > > Which months are these? :) January 2018 as we approach the mass rebuild. We'll want to review the state of the test builds as well as other incoming bugs for things that are most important leading into the official mass rebuild of F28. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: super-drafty F28 and F29 schedules
On 07/13/2017 09:27 AM, Matthew Miller wrote: > On Thu, Jul 13, 2017 at 09:23:58AM -0600, Jeff Law wrote: >>>> Likewise. We should keep this schedule in mind as we work our way >>>> through stage3 into stage4. Ideally Marek would start the test builds >>>> prior to the Christmas break, even if we're still in stage3. >>>> We'll review on the 9th and 16th with go/no-go decisions on the 23rd and >>>> 30th. >>> Which months are these? :) >> January 2018 as we approach the mass rebuild. We'll want to review the >> state of the test builds as well as other incoming bugs for things that >> are most important leading into the official mass rebuild of F28. > > You mentioned possibly shifting the GCC process a little bit earlier. > Is it possible to maybe do that as well here, to give a little more > breathing room? Maybe even not for 2018 but for beyond, as we intend to > keep to this schedule for the foreseeable future. That's my hope -- I'll be pushing internally and externally to shift things by a couple weeks. Whether or not the release managers and the project as a whole will support that remains to be seen :-) jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: gcc 10 plans for Fedora 32?
On Wed, 2019-12-11 at 14:53 +0100, Miro Hrončok wrote: > Hello, > > what are the gcc 10 plans for Fedora 32? Will there be a change > proposal for > that? Is Fedora 32 the target for gcc 10? Plan is for gcc-10 to be the compiler for F32. We coordinate with the Fedora leaders on this each year as part of green-lighting the spring mass rebuild (which doesn't start until after the gcc update). Typically rawhide gets the GCC update in late Jan/early Feb. > > I remember that gcc was updated to 9 during Fedora 30 cycle without a > change > proposal and the change proposal was submitted later, after the > deadline and > after gcc was already updated. > > I'd like to avoid repeating this situation. Updating gcc yearly for the spring Fedora release is standard procedure. However, we should still be filing the change proposals. > If you already have a place to test our packages with gcc 10 (such as > a copr > with gcc snapshot), please do tell, I'd like to see what Python > versions are > impacted. I have very rough packages I use in my tester. I'm happy to point you to them, but be aware they (for example) claim to be gcc-9 at the RPM level. file.rdu.redhat.com:~law/public_html/repo.Snapshot has the latest bits. I typically update it Sunday nights to the latest gcc snapshot. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: devel Digest, Vol 190, Issue 186
On Thu, 2019-12-19 at 21:56 +, devel- requ...@lists.fedoraproject.org wrote: > Neal, > > I'm generally happy with this idea. I'm one of the maintainers of > rpm-config-SUSE (the equivalent of redhat-rpm-config for SUSE > distributions) and I somewhat saw the development of this feature > across rpm, rpm-config-SUSE, and the implementation in openSUSE > Tumbleweed. Understood. FWIW, I work closely with Martin L, Martin J, Jan and Richi @ SuSE. Martin L and I in particular coordinate on mass build testing & related failure analysis and bugfixing. > > However, the implementation of LTO in openSUSE caused major problems > for "weaker" architectures like ARM and RISC-V. In Fedora, ARM is > co-primary with the rest of the architectures (ARM is allowed to be > broken in openSUSE from time to time). The major problems we > encountered was increased miscompilation errors and timeouts due to > builds taking even longer with LTO straining ARM build environments. Yes. The 32bit architectures in particular are expected to be slightly problematical due to the limited address space. Even with Jan's work in this area, I expect things like firefox to simply be too big to compile/link using LTO on the 32bit platforms. When I discussed this with Martin L and one of the Ubuntu engineers (Matthias) back in Sept, the general plan we agreed on was to first get the Fedora test builds in reasonable shape on x86_64 that we could use as a baseline (and we're just about there). Then do an i686 build for comparison purposes. For packages we find problematical on the 32bit platforms, we'll be able to trivially opt-out of LTO for that package on those 32bit platforms. It's a one-liner in the .spec file. > > And with RISC-V, is LTO even implemented well in GCC? In OpenMandriva > (where I help maintain rpm-openmandriva-setup, its equivalent of > redhat-rpm-config), we're still using GCC for RISC-V since Clang is > still horribly broken there, and we had to disable LTO to get a usable > port going. Fedora RISC-V is going to be in for a world of pain if > we're enabling it without any consideration for them. GCC's implementation of LTO is target independent. RISC-V really isn't on my radar at this point. Though if we have issues specific to the code generator on RISC-V, I'm sure Jim Wilson (SiFive) and I can work it out. We've been working together on GCC for nearly 30 years at this point. I would _not_ expect RISC-V's code generator to be as solid as other platforms simply because it hasn't been used much yet. > > So what's the state of things for non-x86? Generally speaking, my > experience indicates everything is broken... My tester is x86_64 based. There's no inherent reason it couldn't work on another platform other than needing lots of machines to turn around builds quickly enough to matter. My hope has always been to spin up i686 and ppc64le as I can get access to reasonable hardware for those targets. I've spoken with ARM's & IBM's GCC engineers about this upcoming change, they're aware and supportive of the plan. We typically lean heavily on them to take care of any GCC issues that are specific to their targets. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: devel Digest, Vol 190, Issue 187
On Thu, 2019-12-19 at 22:14 +, devel- requ...@lists.fedoraproject.org wrote: Igor, > Send devel mailing list submissions to > > It would be very nice to get more specific analysis data. Like running > some benchmarks of big applications, size comparisons (of binaries and > libraries) and compile time. Jan (from SuSE) probably has the best data on this. http://www.ucw.cz/~hubicka/slides/opensuse2018-e.pdf https://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-procedural.html I don't have his 2019 Cauldron slides which discuss the most recent improvements. I'll put those links into the change proposal. > > > This change also brings us back on-par with SuSE who enabled LTO by > > default for their free distribution earlier in 2019. > > You probably should have said openSUSE rather here. Yea, openSUSE is more appropriate here. I'll update that. > > > == Scope == > > * Proposal owners: > > The primary change is to redhat-rpm-config to add LTO to the default > > compile/link flags as well as a conditional which allows easy opt-out > > on a package by package basis. Additionally the post-build scripts > > need to strip the LTO bytecodes from any installed .o/.a files. > > What does that mean? Which post-build scripts are needed and where > that needs to be done? How does it affect build time? There's one post-build script which strips the LTO bytecode. Essentially it's just like stripping debug info or the static symbol table (like we already do), except we have to apply it to all the .o/.a files. The openSUSE guys already have these bits, odds are we'll just copy 'em. I wouldn't expect the stripping to be noticeable on the build time. LTO itself will certainly make things slower at build time. > > > Additionally, we know there are many packages with configure scripts > > that are compromised by LTO. I have tweaks to the %configure macro in > > redhat-rpm-config which fixes the vast majority of these problems with > > a few simple sed scripts on the generated output. Like the basic > > support for injecting the LTO flags, this will require coordination > > with the redhat-rpm-config maintainers. Packages which call configure > > directly and have compromised tests will need a one line change to > > their .spec files to fix their configure scripts. > > "simple sed scripts" are probably not so simple :) There's 5 sed scripts I'm using within %configure. Trust me, that's dramatically easier than trying to fix every package with broken configure scripts -- particularly when some of those configure files were built with versions of autoconf that are 12+ years old. Only two of the 5 sed scripts are, IMHO as a non-sed person, complex. This avoids the need to fix several hundred packages. There's certainly some straggler packages that have configure bits that are unique to those packages rather than shared across many packages. Those I expect we'll just fix on a per-package basis (it's measured in a dozen or two in the end I suspect). I have a tester which allows me to build all the Fedora binary packages. So I build once with gcc-10+LTO, capture the generated config.h files, then build again with the standard gcc-9 compiler. Then the resultant config.h files are compared. If there's any differences, the build is failed. > > > Some packages will need to opt-out of using LTO at this time. The > > most common case are packages that use symbol versioning or toplevel > > ASM statements. While there is a new mechanism to make LTO work with > > symbol versioning, I don't think any packages have been updated to use > > that mechanism. This will require a one line change to 50-75 packages > > (my script to find these is still running). > > Hmm, I think we have bunch of packages (more than 75 of them) which > use symbol versioning. Would be useful to give some links in the > change proposal to "how to update to use that mechanism". Yup. I've already got a (growing) list of 'em. I've got ~50 on my list and ~125 failures still to evaluate. Ideally the opt-out mechanism for a package's .spec file would look like: %undefine _lto Simple enough? :-) Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: devel Digest, Vol 190, Issue 186
On Thu, 2019-12-19 at 16:24 -0600, Neal Gompa wrote: > On Thu, Dec 19, 2019 at 4:14 PM Jeff Law wrote: > > On Thu, 2019-12-19 at 21:56 +, devel- > > requ...@lists.fedoraproject.org wrote: > > > > Neal, > > > > > > > I'm generally happy with this idea. I'm one of the maintainers of > > > rpm-config-SUSE (the equivalent of redhat-rpm-config for SUSE > > > distributions) and I somewhat saw the development of this feature > > > across rpm, rpm-config-SUSE, and the implementation in openSUSE > > > Tumbleweed. > > Understood. FWIW, I work closely with Martin L, Martin J, Jan and > > Richi @ SuSE. Martin L and I in particular coordinate on mass build > > testing & related failure analysis and bugfixing. > > > > Martin Liška is the one I've primarily interfaced with throughout the > implementation. > > I don't know if you know about this, but there's a tracker bug for > LTO-related failures: https://bugzilla.opensuse.org/1133084 I am aware of it and have referenced it several times in my own work. > > We should probably make sure this is cross-referenced as LTO is > implemented in Fedora. Sure. That's easy enough to do. Some of the information is dated, but there still useful nuggets in there. They punted lots of stuff though. In particular they punted the configure problem and middle-end issued diagnostics, which was terribly unfortunate. > > > > However, the implementation of LTO in openSUSE caused major problems > > > for "weaker" architectures like ARM and RISC-V. In Fedora, ARM is > > > co-primary with the rest of the architectures (ARM is allowed to be > > > broken in openSUSE from time to time). The major problems we > > > encountered was increased miscompilation errors and timeouts due to > > > builds taking even longer with LTO straining ARM build environments. > > Yes. The 32bit architectures in particular are expected to be slightly > > problematical due to the limited address space. Even with Jan's work > > in this area, I expect things like firefox to simply be too big to > > compile/link using LTO on the 32bit platforms. > > > > When I discussed this with Martin L and one of the Ubuntu engineers > > (Matthias) back in Sept, the general plan we agreed on was to first get > > the Fedora test builds in reasonable shape on x86_64 that we could use > > as a baseline (and we're just about there). Then do an i686 build for > > comparison purposes. > > > > For packages we find problematical on the 32bit platforms, we'll be > > able to trivially opt-out of LTO for that package on those 32bit > > platforms. It's a one-liner in the .spec file. > > > > > > Are we going to use the same %_lto_cflags mechanism that was > implemented in openSUSE, or are we going to do something different? Ideally the same. I don't own redhat-rpm-config, but certainly my preference is use the same mechanisms. > > > I'd like to see at *least* AArch64 spun to see how well this goes. I'm > reasonably confident that POWER8+ would be fine, since we don't have > ppc64be anymore (and I *know* that one was broken, since I reported it > years ago: https://bugzilla.redhat.com/1515934). Obviously aarch64 will be built and any issues resolved as is the case with other architectures. We do this every spring cycle with the introduction of a major GCC release and we work closely with the GCC engineers at ARM and IBM along the way. Building 9000 packages 3X each (gcc-10+LTO, gcc-10, gcc-9 baseline) takes significant hardware. If there's hardware I can use (and you should be thinking on the order of dozens of dedicated machines), then I'm happy to use them. I mentioned ppc64le simply because I can get access to a goodly number of them. Thanks for that reference. Big endian ppc. Ugh. But in the end it's just code. Jeff > > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 32 System-Wide Change proposal: LTO by default for package builds
On Fri, 2019-12-20 at 11:52 +0100, Miro Hrončok wrote: > On 19. 12. 19 22:41, Ben Cotton wrote: > > https://fedoraproject.org/wiki/LTOByDefault > > > > == Contingency Plan == > > * Contingency mechanism: Revert the LTO flags injection > > * Contingency deadline: Beta freeze, but shooting for prior to mass > > rebuilds starting > > * Blocks release? No > > * Blocks product? No > > > > Most critically, if we don't address the GDB testsuite issue noted > > above, our fallback position would be to simply disable the LTO > > injection globally and re-evaluate for Fedora 33, similarly if we were > > to find some show-stopping LTO issue. > > Should the contingency plan include a second mass rebuild in case our > packages > successfully built with lto during the mass rebuild, but are broken at > runtime? > > Or do we safely assume that it's good as long as it builds fine? It would really depend on what we find and it's the kind of decision we deal with semi-regularly with the yearly compiler update in Fedora -- what to do if we find something horrifically wrong, whether it be an unexpected ABI change or a particularly nasty codegen issue. When we encounter these situations we often end up instrumenting the compiler to get a sense of how pervasive the problem is or building a scanner to look for problematic code sequences across the builds done with the problematic compiler/options. To-date I don't think we've ever had to do something like back out a major compiler update or request a mass rebuild in Fedora as a result of a compiler issue. We've had some "phew" moments for issues we thought were going to have wide impact, but for various reasons didn't. So while I wish I could give you a concrete answer, it's really something we (Jakub, Marek, Jason, Jon, myself and others) evaluate on a case by case basis looking at size/scope and determine a path forward, knowing that an unexpected mass rebuild would be catastrophic. What gives me a relatively high degree of confidence in the introduction of LTO is that openSUSE made this change for their distro in the spring of this year confirming the base LTO technology on a wide scale, combined with the testing we're already doing for Fedora with gcc-10. By *far* the biggest issue with LTO is the configure test snippets that are so horribly written that they can be easily compromised by aggressive optimization enabled by LTO and the widespread nature of that horrific code. Hence the desire to fix the most common and egregious issues by in-place editing of the already-generated configure files within %configure. And just to be clear, these code snippets would never actually run and nobody would ever write code this way expecting it to be executed -- they're primarily compile/link tests to look for features. Concerns over mis-configuring packages due to these issues is what led to the introduction of the capability to capture the generated config.h files and compare them against builds with standard tools in our tester. If we note any differences in the generated config.h files, the build is considered a failure and needs to be examined and the issue fixed. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: digested list etiquette [ was: devel Digest, Vol 190, Issue 186 ]
On Fri, 2019-12-20 at 12:59 -0500, Robbie Harwood wrote: > Hi, > > When replying to digests, I'd appreciate if you could please make an > effort to have the posts thread properly for the rest of us. Fedora > mailing list guidance on this: > https://fedoraproject.org/wiki/Mailing_list_guidelines#Replying_to_Digests Will do. Sorry for making things harder on everyone in this regard. jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: devel Digest, Vol 190, Issue 186
On Sat, 2019-12-21 at 10:52 -0500, Josh Boyer wrote: > On Thu, Dec 19, 2019 at 5:48 PM Jeff Law wrote: > > On Thu, 2019-12-19 at 16:24 -0600, Neal Gompa wrote: > > > On Thu, Dec 19, 2019 at 4:14 PM Jeff Law wrote: > > > > On Thu, 2019-12-19 at 21:56 +, devel- > > > > requ...@lists.fedoraproject.org wrote: > > > > > > > > Neal, > > > > > > > > > > > > > I'm generally happy with this idea. I'm one of the maintainers of > > > > > rpm-config-SUSE (the equivalent of redhat-rpm-config for SUSE > > > > > distributions) and I somewhat saw the development of this feature > > > > > across rpm, rpm-config-SUSE, and the implementation in openSUSE > > > > > Tumbleweed. > > > > Understood. FWIW, I work closely with Martin L, Martin J, Jan and > > > > Richi @ SuSE. Martin L and I in particular coordinate on mass build > > > > testing & related failure analysis and bugfixing. > > > > > > > > > > Martin Liška is the one I've primarily interfaced with throughout the > > > implementation. > > > > > > I don't know if you know about this, but there's a tracker bug for > > > LTO-related failures: https://bugzilla.opensuse.org/1133084 > > I am aware of it and have referenced it several times in my own work. > > > > > We should probably make sure this is cross-referenced as LTO is > > > implemented in Fedora. > > Sure. That's easy enough to do. Some of the information is dated, but > > there still useful nuggets in there. > > > > They punted lots of stuff though. In particular they punted the > > configure problem and middle-end issued diagnostics, which was terribly > > unfortunate. > > > > > > > However, the implementation of LTO in openSUSE caused major problems > > > > > for "weaker" architectures like ARM and RISC-V. In Fedora, ARM is > > > > > co-primary with the rest of the architectures (ARM is allowed to be > > > > > broken in openSUSE from time to time). The major problems we > > > > > encountered was increased miscompilation errors and timeouts due to > > > > > builds taking even longer with LTO straining ARM build environments. > > > > Yes. The 32bit architectures in particular are expected to be slightly > > > > problematical due to the limited address space. Even with Jan's work > > > > in this area, I expect things like firefox to simply be too big to > > > > compile/link using LTO on the 32bit platforms. > > > > > > > > When I discussed this with Martin L and one of the Ubuntu engineers > > > > (Matthias) back in Sept, the general plan we agreed on was to first get > > > > the Fedora test builds in reasonable shape on x86_64 that we could use > > > > as a baseline (and we're just about there). Then do an i686 build for > > > > comparison purposes. > > > > > > > > For packages we find problematical on the 32bit platforms, we'll be > > > > able to trivially opt-out of LTO for that package on those 32bit > > > > platforms. It's a one-liner in the .spec file. > > > > > > > > > > > > > > Are we going to use the same %_lto_cflags mechanism that was > > > implemented in openSUSE, or are we going to do something different? > > Ideally the same. I don't own redhat-rpm-config, but certainly my > > preference is use the same mechanisms. > > > > > > > > I'd like to see at *least* AArch64 spun to see how well this goes. I'm > > > reasonably confident that POWER8+ would be fine, since we don't have > > > ppc64be anymore (and I *know* that one was broken, since I reported it > > > years ago: https://bugzilla.redhat.com/1515934). > > Obviously aarch64 will be built and any issues resolved as is the case > > with other architectures. We do this every spring cycle with the > > introduction of a major GCC release and we work closely with the GCC > > engineers at ARM and IBM along the way. > > > > Building 9000 packages 3X each (gcc-10+LTO, gcc-10, gcc-9 baseline) > > takes significant hardware. If there's hardware I can use (and you > > should be thinking on the order of dozens of dedicated machines), then > > I'm happy to use them. I mentioned ppc64le simply because I can get > > access to a goodly number of them. > > Is there a reason you couldn't use aarch64 AWS instances? I've actually applied for aws credits which could be used for this. THe instances in the free tier are too small to really be useful. jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 32 System-Wide Change proposal: GCC10
On Thu, 2020-01-02 at 19:48 +, devel- requ...@lists.fedoraproject.org wrote: > Date: Thu, 2 Jan 2020 12:52:03 -0500 > From: Kaleb Keithley > Subject: Re: Fedora 32 System-Wide Change proposal: GCC10 > To: Development discussions related to Fedora > > Message-ID: > > Content-Type: multipart/alternative; > boundary="6bf9f6059b2bdac6" > > --6bf9f6059b2bdac6 > Content-Type: text/plain; charset="UTF-8" > Content-Transfer-Encoding: quoted-printable > > One (the only) thing I've noticed so far about gcc-10 is that (sloppily) > defined variables in header files that lack an extern qualifier and that > don't have an explicit defn in a .c file are no longer 'common' or .comm > but are now .global .bss and cause link errors due to duplicate definitions= > . > > This very well might be because I made a mistake in the way I built gcc-10. > I'm not sure I have any way to know. > > If it's not a mistake on my part then this change has revealed a few bugs > in the other applications that I work on. > > These bugs should be fixed of course, but it's something to be aware of > when considering a major change like this, this late in the f32/rawhide > development cycle. Almost certainly a lot of other things will have similar > bugs. This is a known and desirable change in the compiler. GCC-10 defaults to -fno-common for C which is a change relative to gcc- 9 (C++ has been -fno-common for eons, possibly forever). You really should get your sources fixed to adhere to modern C standards. This affects ~450 packages in Fedora, openSUSE and other distributions. I have already reached out to ffesti (without a response :( to get an opt-out mechanism for redhat-rpm-config that would broken packages to opt-out of this behavior. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: gcc 10: Default to -fno-common, multiple definitions of ...
> > > > Date: Tue, 21 Jan 2020 11:05:37 -0500 > From: Steve Grubb > Subject: Re: gcc 10: Default to -fno-common, multiple definitions of > ... > To: devel@lists.fedoraproject.org > Cc: Jakub Jelinek > Message-ID: <4127758.jL2Gs7s9Fr@x2> > Content-Type: text/plain; charset="UTF-8" > > On Tuesday, January 21, 2020 7:35:03 AM EST Miro Hrončok wrote: > > This is a known thing in gcc 10: > > > > https://gcc.gnu.org/gcc-10/porting_to.html#common > > > > "Default to -fno-common > > > > A common mistake in C is omitting extern when declaring a global variable > > in a header file. If the header is included by several files it results in > > multiple definitions of the same variable. In previous GCC versions this > > error is ignored. GCC 10 defaults to -fno-common, which means a linker > > error will now be reported. To fix this, use extern in header files when > > declaring global variables, and ensure each global is defined in exactly > > one C file. As a workaround, legacy C code can be compiled with -fcommon. > > > > > >int x; // tentative definition - avoid in header files > > > >extern int y; // correct declaration in a header file" > > So, for those of us using F31 as the development / test environment, is there > a macro that we can add -fno-common to in ~/.rpmmacros without placing it in > the %__global_compiler_flags in /usr/lib/rpm/redhat/macros ? > > Looking for an easy way to reproduce this without modifying root owned files. > > -Steve > > > -- > > Date: Tue, 21 Jan 2020 17:27:20 +0100 > From: Miro Hrončok > Subject: Re: gcc 10: Default to -fno-common, multiple definitions of > ... > To: Steve Grubb , devel@lists.fedoraproject.org > Message-ID: <0f655139-44f5-a0c5-b600-66614c8bd...@redhat.com> > Content-Type: text/plain; charset=utf-8; format=flowed > > On 21. 01. 20 17:05, Steve Grubb wrote: > > On Tuesday, January 21, 2020 7:35:03 AM EST Miro Hrončok wrote: > > > This is a known thing in gcc 10: > > > > > > https://gcc.gnu.org/gcc-10/porting_to.html#common > > > > > > "Default to -fno-common > > > > > > A common mistake in C is omitting extern when declaring a global variable > > > in a header file. If the header is included by several files it results in > > > multiple definitions of the same variable. In previous GCC versions this > > > error is ignored. GCC 10 defaults to -fno-common, which means a linker > > > error will now be reported. To fix this, use extern in header files when > > > declaring global variables, and ensure each global is defined in exactly > > > one C file. As a workaround, legacy C code can be compiled with -fcommon. > > > > > > > > > int x; // tentative definition - avoid in header files > > > > > > extern int y; // correct declaration in a header file" > > > > So, for those of us using F31 as the development / test environment, is > > there > > a macro that we can add -fno-common to in ~/.rpmmacros without placing it in > > the %__global_compiler_flags in /usr/lib/rpm/redhat/macros ? > > > > Looking for an easy way to reproduce this without modifying root owned > > files. > > You should be able to redefine %__global_compiler_flags in ~/.rpmmacros. > It still means you need to copy paste the current flags, but you won't need > to > touch root owned files. I proposed a change to redhat-rpm-config to handle this case by allowing package to add a single line to their .spec file to turn off the new common symbol handling. Igor rejected that change arguing that the packages themselves should be fixed. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: gcc 10: Default to -fno-common, multiple definitions of
> > -- > > > Date: Tue, 21 Jan 2020 17:44:37 + > From: Peter Robinson > Subject: Re: gcc 10: Default to -fno-common, multiple definitions of > ... > To: l...@redhat.com, Development discussions related to Fedora > > Message-ID: > > Content-Type: text/plain; charset="UTF-8" > > > > I proposed a change to redhat-rpm-config to handle this case by > > allowing package to add a single line to their .spec file to turn off > > the new common symbol handling. Igor rejected that change arguing that > > the packages themselves should be fixed. > > Ultimately yes they should be fixed, but giving a means of easily > mitigating the problem while people work with upstream to fix the > issues is also useful, it's been a problem for ever and expecting > people to fix it in short order is problematic especially when they'll > start to be deluged in FTBFS spam within moments of the mass rebuild. That was the idea. Provide a trivial opt-out so that upstreams had time to fix properly. I even volunteered to add the opt-out marker where appropriate to minimize the FTBFS issues. I also volunteered to help with the packages that don't honor flags injection. > > It might also be handy for there to be a heads up bug when the > toolchain team first discovers these sorts of issues to give a longer > runway for people to be able to engage with upstream to get a fix. I contacted ffesti about 6 weeks ago -- shortly after the commit went upstream GCC and I'd consulted with the SuSE engineers about how they were planning to handle this situation. Sadly I got no response whatsoever from ffesti. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: gcc 10: Default to -fno-common, multiple definitions of
On Tue, 2020-01-21 at 13:33 -0500, Steve Grubb wrote: > On Tuesday, January 21, 2020 1:16:00 PM EST Miro Hrončok wrote: > > > > > > I proposed a change to redhat-rpm-config to handle this case by > > > > > > > > > > allowing package to add a single line to their .spec file to turn off > > > > > the new common symbol handling. Igor rejected that change arguing > > > > > that > > > > > the packages themselves should be fixed. > > > > > > > > Ultimately yes they should be fixed, but giving a means of easily > > > > mitigating the problem while people work with upstream to fix the > > > > issues is also useful, it's been a problem for ever and expecting > > > > people to fix it in short order is problematic especially when they'll > > > > start to be deluged in FTBFS spam within moments of the mass rebuild. > > > > > > That was the idea. Provide a trivial opt-out so that upstreams had > > > time to fix properly. I even volunteered to add the opt-out marker > > > where appropriate to minimize the FTBFS issues. I also volunteered to > > > help with the packages that don't honor flags injection. > > > > This sounds reasonable to me. > > > > The PR is > > https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/77 > > > > I have reopened it. > > > > The mass rebuild will notify the maintainrs and they can use that macro to > > short-workaround the problem. It' also easier to grep from the specs than > > custom patches and whatnots > > I need something like this for suricata. It has about 45 variables causing > this. And it's not a simple "extern" addition because it looks like in many > cases the variable was never placed in a C file. Simply adding extern keyword > leads undefined reference errors. This will take a while to sort out with > upstream. You also have to be careful if you're building shared libraries -- I was looking at a package (I forget which) which built multiple DSOs. If you're not careful you can end up with some DSOs which wouldn't have the definition -- worse yet, you're not going to get an undefined symbol when you build those DSOs. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: gcc 10: Default to -fno-common, multiple definitions of
On Tue, 2020-01-21 at 19:16 +0100, Miro Hrončok wrote: > On 21. 01. 20 19:04, Jeff Law wrote: > > > -- > > > > > > > > > Date: Tue, 21 Jan 2020 17:44:37 + > > > From: Peter Robinson > > > Subject: Re: gcc 10: Default to -fno-common, multiple definitions of > > > ... > > > To: l...@redhat.com, Development discussions related to Fedora > > > > > > Message-ID: > > > > > > Content-Type: text/plain; charset="UTF-8" > > > > > > > > I proposed a change to redhat-rpm-config to handle this case by > > > > allowing package to add a single line to their .spec file to turn off > > > > the new common symbol handling. Igor rejected that change arguing that > > > > the packages themselves should be fixed. > > > > > > Ultimately yes they should be fixed, but giving a means of easily > > > mitigating the problem while people work with upstream to fix the > > > issues is also useful, it's been a problem for ever and expecting > > > people to fix it in short order is problematic especially when they'll > > > start to be deluged in FTBFS spam within moments of the mass rebuild. > > That was the idea. Provide a trivial opt-out so that upstreams had > > time to fix properly. I even volunteered to add the opt-out marker > > where appropriate to minimize the FTBFS issues. I also volunteered to > > help with the packages that don't honor flags injection. > > This sounds reasonable to me. > > The PR is https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/77 > > I have reopened it. > > The mass rebuild will notify the maintainrs and they can use that macro to > short-workaround the problem. It' also easier to grep from the specs than > custom > patches and whatnots. Yes, the ability to inspect a .spec file for "violations" was also a benefit -- both in terms of what packages are in need of TLC WRT common symbol handling and in terms of finding packages that are not honoring flags injection. So the question is who makes the final call WRT the change to redhat- rpm-config, particularly given Igor's position? jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: gcc 10: Default to -fno-common, multiple definitions of ...
> Date: Tue, 21 Jan 2020 14:00:30 +0100 > From: Miro Hrončok > Subject: Re: gcc 10: Default to -fno-common, multiple definitions of > ... > To: devel@lists.fedoraproject.org, Jakub Jelinek > Message-ID: > Content-Type: text/plain; charset=utf-8; format=flowed > > On 21. 01. 20 13:47, Jakub Jelinek wrote: > > On Tue, Jan 21, 2020 at 01:42:25PM +0100, Fabio Valentini wrote: > > > I've seen this issue pop up in some other packages, as well. > > > > > > My elementary-files package is affected, and I think it broke > > > rubygem-ffi, too (which is blocking the ruby 2.7 rebuild, breaking a > > > lot of ruby packages; though I can't access the build log for the > > > failed rubygem-ffi build, due to a koji bug relating to side tags). > > > > Yes, it should affect around 400 packages in the distro from what I remember > > from Jeff's test mass rebuild > > Can you please share the list? I'd liek to now how many Python packages are > there and analyze whether extension modules built with distutils are affected > or > not. Sent offlist. Happy to provide to anyone that wants the data, but didn't figure everyone would want it on devel@. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: gcc 10: Default to -fno-common, multiple definitions of ...
On Tue, 2020-01-21 at 13:47 +0100, Jakub Jelinek wrote: > On Tue, Jan 21, 2020 at 01:42:25PM +0100, Fabio Valentini wrote: > > I've seen this issue pop up in some other packages, as well. > > > > My elementary-files package is affected, and I think it broke > > rubygem-ffi, too (which is blocking the ruby 2.7 rebuild, breaking a > > lot of ruby packages; though I can't access the build log for the > > failed rubygem-ffi build, due to a koji bug relating to side tags). > > Yes, it should affect around 400 packages in the distro from what I remember > from Jeff's test mass rebuild, other distros are aware of it too and the > fixes just need to be pushed upstream or grabber from upstream if they are > there already. > Adding -fcommon is a temporary workaround but fixing the packages is > preferred, especially if those fixes end up upstream. Right. It was ~450. The SuSE team in particular has had reasonable success getting fixes upstreamed -- I think Martin L indicated they'd gotten a few dozen fixed upstream. ISTM that fixing upstream is by far the best choice followed by fixing with a Fedora patch. Working around via an opt-out or modifying the flags in a .spec file is the least desirable. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
gcc-10 Fortran argument mismatch issue
So this is another issue that's going to be seen with gcc-10. I'd been hoping to get the time to fix packages correctly, but I think it's ultimately going to have to fall to the package maintainers. gcc has traditionally allowed certain type mismatches for arguments in Fortran code. GCC would issue a diagnostic under -Wargument-mismatch for these cases, but the issue was not considered fatal. Starting with gcc-10, these are now fatal errors which look something like this: Error: Rank mismatch in argument ‘array’ at (1) (rank-1 and scalar) While gcc-10 has an option to disable this diagnostic, the potential for codegen issues in this space was significant enough that I didn't think an opt-out or advertising magic flag workaround was advisable. A partial list of the affected packages (generated back in October from a partial run of my tester): R-deldir R atlas cgnslib cp2k elk elpa exciting ga getdata grib_api hdf libccp4 mpich hwchem psblas3 qrmumps qrupdate quantum-espresso scalapack scipy scorep wannier90 wsjtx xfoil xrotor One of the upstream GCC developers looked at wannier90. It turns out we're using a fairly old version (2.0.1, 2015). Newer versions (3.0 Feb 2019) already have this problem fixed. So consider this a heads-up that roughly 30-40 Fortran packages are going to start failing. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Re: gcc 10: Default to -fno-common, multiple definitions of ...
> Date: Wed, 22 Jan 2020 07:34:01 -0600 > From: Justin Forbes > Subject: Re: gcc 10: Default to -fno-common, multiple definitions of > ... > To: Development discussions related to Fedora > > Message-ID: > > Content-Type: text/plain; charset="UTF-8" > > On Wed, Jan 22, 2020 at 5:42 AM Miro Hrončok wrote: > > On 22. 01. 20 12:33, Fabio Valentini wrote: > > > On Wed, Jan 22, 2020 at 12:12 PM Kevin Kofler > > > wrote: > > > > Miro Hrončok wrote: > > > > > https://gcc.gnu.org/gcc-10/porting_to.html#common > > > > > > > > > > "Default to -fno-common > > > > > > > > > > A common mistake in C is omitting extern when declaring a global > > > > > variable > > > > > in a header file. If the header is included by several files it > > > > > results in > > > > > multiple definitions of the same variable. In previous GCC versions > > > > > this > > > > > error is ignored. GCC 10 defaults to -fno-common, which means a linker > > > > > error will now be reported. To fix this, use extern in header files > > > > > when > > > > > declaring global variables, and ensure each global is defined in > > > > > exactly > > > > > one C file. As a workaround, legacy C code can be compiled with > > > > > -fcommon. > > > > > > > > > > > > > > > int x; // tentative definition - avoid in header files > > > > > > > > > > extern int y; // correct declaration in a header file" > > > > > > > > I fail to see how this kind of incompatible change that breaks hundreds > > > > of > > > > packages (and certainly a lot more software out there) is an > > > > improvement. > > > > > > > > Kevin Kofler > > > > > > Honestly, it would have been good to mention that GCC 10 contains such > > > a breaking change, either on the GCC10 Change proposal wiki page, or > > > in the FESCo ticket. > > > Neither was the case. :( > > > > > > Now we still don't have a publicly available list of broken packages > > > (which, apparently, was known?) - can we please get the list? > > > I don't care where, but adding a link to it to either the Change wiki > > > page or the FESCo ticket would be great, so we can better deal with > > > it. > > > > This is what I got offlist. > > > > "This was generated in early December. There may be minor differences > > due to package updates, deprecations, etc. But this should be 95% > > complete." > > > The list is certainly incomplete. Kernel and kernel-tools are not on > it, but don't build due to it. And, the issues don't seem to be "due > to updates" rather code that has been there for quite some time. So to give everyone a bit of background on the tester. It's primary purpose is to identify GCC issues by building all the Fedora packages with the weekly GCC development snapshot. My time is limited, so to avoid digging into issues that aren't really related to the GCC development snapshot under testing the process works like this: 1. Build a package using the development snapshot (using mock and a priority repo so that my GCC package is selected before the standard tools). If that build succeeds, then we're done and happy. 2. If the build in step #1 fails, then we try the build again, but this time without the priority repo. So we're building with the standard Fedora tools. If this build succeeds, then the package is considered as failing and gets investigated as it's highly likely there's a GCC bug that needs addressing. However, if this control build also fails, then there's a problem of some sort with the package itself or other tools in the buildroot. And given my time is limited and the purpose of my tester is to find GCC development bugs, if the control build fails, then it's not typically investigated. Typically there's around ~500 packages that fail their control build at any given time. I know I've seen kernel and kernel-tools in that bucket from time to time. My logs don't go back far enough to know the state of kernel and kernel-tools in early December when that list was generated. In the run with last week's GCC snapshot, both the kernel and kernel-tools worked with gcc-10 (they failed LTO, but that's totally separate from the common symbol issues we're discussing now). Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re:gcc-10 Fortran argument mismatch issue
On Tue, 2020-01-21 at 20:41 +, devel- requ...@lists.fedoraproject.org wrote: > -- > > Date: Tue, 21 Jan 2020 21:27:47 +0100 > From: Dominik 'Rathann' Mierzejewski > Subject: Re: gcc-10 Fortran argument mismatch issue > To: devel@lists.fedoraproject.org > Message-ID: <20200121202747.ga5...@sakura.greysector.net> > Content-Type: text/plain; charset=utf-8 > > Hi, Jeff. > > On Tuesday, 21 January 2020 at 21:16, Jeff Law wrote: > > > > So this is another issue that's going to be seen with gcc-10. I'd been > > hoping to get the time to fix packages correctly, but I think it's > > ultimately going to have to fall to the package maintainers. > > > > gcc has traditionally allowed certain type mismatches for arguments in > > Fortran code. GCC would issue a diagnostic under -Wargument-mismatch > > for these cases, but the issue was not considered fatal. > > > > Starting with gcc-10, these are now fatal errors which look something > > like this: > > > > > > > > Error: Rank mismatch in argument ‘array’ at (1) (rank-1 and scalar) > > Could you give some example(s) how to fix such errors? Is there an entry > for this in the gcc-10 porting guide? > > > While gcc-10 has an option to disable this diagnostic, the potential > > for codegen issues in this space was significant enough that I didn't > > think an opt-out or advertising magic flag workaround was advisable. > > > > A partial list of the affected packages (generated back in October from > > a partial run of my tester): > > Are there any build logs with errors available? Not easily for a variety of reasons. We're working on that, but I doubt it'll be ready during the F32 cycle. But I can give you specific examples pretty easily. wannier90 is probably the best because there's a single instance and we know it's already fixed in newer versions of wannier90. > gfortran -O2 -flto -ffat-lto-objects -g -pipe -Wall > -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS > -fexceptions -fstack-protector-strong -grecord-gcc-switches > -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 > -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic > -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fPIC > -c ../postw90/comms.F90 > f951: Warning: '-Werror=' argument '-Werror=format-security' is not valid for > Fortran > ../postw90/comms.F90:893:29: > > 893 | call my_icopy(localcount,rootglobalarray,1,array,1) > | 1 > Error: Rank mismatch in argument 'zx' at (1) (rank-1 and scalar) What GCC is complaining about is argument types at the call site do not match the arguments at the function definition. In this case, at the call site "rootglobalarray" is a simple integer: > integer, intent(inout):: rootglobalarray Yet if we look at the definition of my_icopy, the corresponding parameter (ZX) is an array. subroutine my_ICOPY(N,ZX,INCX,ZY,INCY) > ! .. Scalar Arguments .. > integer INCX,INCY,N > ! .. > ! .. Array Arguments .. > integer ZX(*),ZY(*) Thus the types at the call site do not match the types at the definition site and gcc complains. Now if we look at wanner-3.0 from upstream we see the how this was fixed. > integer, dimension(:), intent(inout) :: rootglobalarray > The type at the call site was changed to have an array type. Note the same fix was applied for the "array" argument at the call site (boy was that horribly named!) Note that I'm _not_ a Fortran guy, it's well out of my area of expertise. So while I could help here, particularly for trivial cases like this, it's probably better for package owners to step in and own fixing the Fortran packages. The set of packages I know are affected by these kinds of issues based on last week's builds: arpack cgnslib cp2k dl_poly elk elpa exciting ga getdata grib_api hdf libccp4 mpich MUMPS ncl netcdf-fortran psblas3 qrmumps qrupdate quantum-espresso R R-deldir scalapack scipy scorep wannier90 wsjtx xfoil xrotor Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: cjdns with gcc-10 in Fedora
On Wed, 2020-01-22 at 22:03 +, devel- requ...@lists.fedoraproject.org wrote: > Send devel mailing list submissions to > devel@lists.fedoraproject.org > > To subscribe or unsubscribe via email, send a message with subject or > body 'help' to > devel-requ...@lists.fedoraproject.org > > You can reach the person managing the list at > devel-ow...@lists.fedoraproject.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of devel digest..." > > Today's Topics: > >1. Re: gcc 10: Default to -fno-common, multiple definitions of ... > (Stephen John Smoogen) >2. Re: gcc 10: Default to -fno-common, multiple definitions of ... > (Christoph Junghans) >3. Re: gcc 10: Default to -fno-common, multiple definitions of ... > (Fabio Valentini) >4. Re: gcc 10: Default to -fno-common, multiple definitions of ... > (Michael Cronenworth) >5. Orphaning argyllcms (Richard Hughes) > > > -- > > Date: Wed, 22 Jan 2020 15:30:45 -0500 > From: Stephen John Smoogen > Subject: Re: gcc 10: Default to -fno-common, multiple definitions of > ... > To: Development discussions related to Fedora > > Message-ID: > > Content-Type: text/plain; charset="UTF-8" > > On Wed, 22 Jan 2020 at 14:54, Christoph Junghans wrote: > > On Wed, Jan 22, 2020 at 4:42 AM Miro Hrončok wrote: > > > jtaylornetsniff-ng ocaml-lablgtk suricata > > > junghans gasnet > > > > Sorry, how do I install gcc10 on Rawhide? > > "dnf upgrade --advisory=FEDORA-2020-a165791b6f" doesn't seem to do it. > > > > Christoph > > > > Going from a similar discussion in #fedora-devel. There has been no > current compose of rawhide so it can not be synced out of the mirror. > In order to upgrade/install gcc10 you will need to enable the local > repo in mock or replicate it on your system. > > 2020-01-21 18:33:05 fedora-32 is still at gcc-9.2.1, but I got an > em > ail saying cjdns no longer builds with gcc-10. How do I rebuild with gcc-10 > to > get this fixed? I assume the goal is to get most packages ready for gcc-10 > befo > re upgrading rawhide. > 2020-01-21 18:58:03 sdgathman: rawhide/f32 already has gcc-10... > the > re just hasn't been a successfull rawhide compose syncing it out to mirrors. > You > can do scratch builds or you could enable the 'local' repo in mock to use > the k > oji buildroot... Note that cjdns, according to my tester, is building fine with gcc-10 with and without LTO enabled as Jan 18th. I know there was a bug in GCC that Martin Sebor fixed on Jan 13th that caused gcc-10 to issue a false positive out of bounds array index warning that in turn caused cjdns to fail. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: cjdns with gcc-10 in Fedora
On Wed, 2020-01-22 at 16:11 -0700, Jeff Law wrote: > On Wed, 2020-01-22 at 22:03 +, devel- > requ...@lists.fedoraproject.org wrote: > > Send devel mailing list submissions to > > devel@lists.fedoraproject.org > > > > To subscribe or unsubscribe via email, send a message with subject or > > body 'help' to > > devel-requ...@lists.fedoraproject.org > > > > You can reach the person managing the list at > > devel-ow...@lists.fedoraproject.org > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of devel digest..." > > > > Today's Topics: > > > >1. Re: gcc 10: Default to -fno-common, multiple definitions of ... > > (Stephen John Smoogen) > >2. Re: gcc 10: Default to -fno-common, multiple definitions of ... > > (Christoph Junghans) > >3. Re: gcc 10: Default to -fno-common, multiple definitions of ... > > (Fabio Valentini) > >4. Re: gcc 10: Default to -fno-common, multiple definitions of ... > > (Michael Cronenworth) > >5. Orphaning argyllcms (Richard Hughes) > > > > > > -- > > > > Date: Wed, 22 Jan 2020 15:30:45 -0500 > > From: Stephen John Smoogen > > Subject: Re: gcc 10: Default to -fno-common, multiple definitions of > > ... > > To: Development discussions related to Fedora > > > > Message-ID: > > > > Content-Type: text/plain; charset="UTF-8" > > > > On Wed, 22 Jan 2020 at 14:54, Christoph Junghans wrote: > > > On Wed, Jan 22, 2020 at 4:42 AM Miro Hrončok wrote: > > > > jtaylornetsniff-ng ocaml-lablgtk suricata > > > > junghans gasnet > > > > > > Sorry, how do I install gcc10 on Rawhide? > > > "dnf upgrade --advisory=FEDORA-2020-a165791b6f" doesn't seem to do it. > > > > > > Christoph > > > > > > > Going from a similar discussion in #fedora-devel. There has been no > > current compose of rawhide so it can not be synced out of the mirror. > > In order to upgrade/install gcc10 you will need to enable the local > > repo in mock or replicate it on your system. > > > > 2020-01-21 18:33:05 fedora-32 is still at gcc-9.2.1, but I got > > an em > > ail saying cjdns no longer builds with gcc-10. How do I rebuild with > > gcc-10 to > > get this fixed? I assume the goal is to get most packages ready for gcc-10 > > befo > > re upgrading rawhide. > > 2020-01-21 18:58:03 sdgathman: rawhide/f32 already has > > gcc-10... the > > re just hasn't been a successfull rawhide compose syncing it out to > > mirrors. You > > can do scratch builds or you could enable the 'local' repo in mock to use > > the k > > oji buildroot... > Note that cjdns, according to my tester, is building fine with gcc-10 > with and without LTO enabled as Jan 18th. > > I know there was a bug in GCC that Martin Sebor fixed on Jan 13th that > caused gcc-10 to issue a false positive out of bounds array index > warning that in turn caused cjdns to fail. Actually, I spoke with Martin today. THe patch wasn't committed as Jakub raised some concerns. So, yes, this issue is still a concern. jeff ps. My tester has Martin's patch, which explains why it's working... ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: gcc 10: Default to -fno-common, multiple definitions of
On Thu, 2020-01-23 at 17:23 -0500, Steve Grubb wrote: > On Tuesday, January 21, 2020 1:52:54 PM EST Jeff Law wrote: > > > > > That was the idea. Provide a trivial opt-out so that upstreams had > > > > > time to fix properly. I even volunteered to add the opt-out marker > > > > > where appropriate to minimize the FTBFS issues. I also volunteered > > > > > to help with the packages that don't honor flags injection. > > > > > > > > This sounds reasonable to me. > > > > > > > > The PR is > > > > https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/77 > > > > > > > > I have reopened it. > > > > > > > > The mass rebuild will notify the maintainrs and they can use that macro > > > > to short-workaround the problem. It' also easier to grep from the specs > > > > than custom patches and whatnots > > > > > > I need something like this for suricata. It has about 45 variables > > > causing this. And it's not a simple "extern" addition because it looks > > > like in many cases the variable was never placed in a C file. Simply > > > adding extern keyword leads undefined reference errors. This will take a > > > while to sort out with upstream. > > > > You also have to be careful if you're building shared libraries -- I > > was looking at a package (I forget which) which built multiple DSOs. > > If you're not careful you can end up with some DSOs which wouldn't have > > the definition -- worse yet, you're not going to get an undefined > > symbol when you build those DSOs. > > So, I have a weird one. I'm compiling audit and getting multiple definitions. > > I have > > /usr/bin/ld: ausearch-checkpt.o:/home/sgrubb/working/BUILD/audit-3.0/src/ > ausearch-common.h:53: multiple definition of `event_node_list'; ausearch.o:/ > home/sgrubb/working/BUILD/audit-3.0/src/ausearch-common.h:53: first defined > here > > $ cat ausearch-checkpt.c | grep event_node_list > > Hmm. Nothing found. > > $ objdump --syms ausearch-checkpt.o | grep event_node_list > 0040 g O .bss 0008 event_node_list > > It's in the object file? In ausearch-checkpt.c we have #include "ausearch- > checkpt.h which does not define it. That file has #include "ausearch-llist.h" > which does not define it. That file has #include "ausearch-common.h" which > does > define it as: > > extern slist *event_node_list; > > So, what appears to be happening is all these C files are creating a > definition > based on this and they don't even use this variable. It seems like they would > create a spot in their table *if and only if* it was referenced somewhere in > the code. I'm not sure if this was intended but it will be a big problem for > everyone. Use -save-temps and post the .i file. An "extern blah blah blah" should not be doing that. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 32 Mass Rebuild
On Mon, 2020-01-27 at 16:34 +0100, Jakub Jelinek wrote: > On Mon, Jan 27, 2020 at 10:25:50AM -0500, Mohan Boddu wrote: > > Hi all, > > > > Per the Fedora 32 schedule[1] we will be starting a mass rebuild for > > Fedora 32 today. We are doing a mass rebuild for Fedora 32 for all the > > changes: > > > > https://fedoraproject.org/wiki/Changes/GLIBC231 > > https://fedoraproject.org/wiki/Releases/32/ChangeSet#Binutils_2.33 > > https://fedoraproject.org/wiki/Changes/FontLangProvidesToLangpacks > > https://fedoraproject.org/wiki/Changes/golang1.14 > > https://fedoraproject.org/wiki/Changes/GCC10 > > https://fedoraproject.org/wiki/Changes/Free_Pascal_Compiler_3.2.0 > > > > we will start the mass rebuild on 2020-01-27 > > I thought the mass rebuild should be also for > https://fedoraproject.org/wiki/LTOByDefault > but the required changes AFAIK haven't landed into redhat-rpm-config yet. > > CCing Jeff on the current state. After finally starting looking at the LTO vs GDB issues last week, I think we should defer LTO until F33. There's just too many GDB failures when used with LTO that aren't understood yet. I have asked that the LTO bytecode stripping change get into redhat- rpm-config immediately though. We already know some packages have enabled LTO and we want to make sure they don't leak the LTO bytecodes into the distro. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 32 Mass Rebuild
On Mon, 2020-01-27 at 21:23 +0100, Fabio Valentini wrote: > On Mon, Jan 27, 2020 at 4:46 PM Jeff Law wrote: > > On Mon, 2020-01-27 at 16:34 +0100, Jakub Jelinek wrote: > > > On Mon, Jan 27, 2020 at 10:25:50AM -0500, Mohan Boddu wrote: > > > > Hi all, > > > > > > > > Per the Fedora 32 schedule[1] we will be starting a mass rebuild for > > > > Fedora 32 today. We are doing a mass rebuild for Fedora 32 for all the > > > > changes: > > > > > > > > https://fedoraproject.org/wiki/Changes/GLIBC231 > > > > https://fedoraproject.org/wiki/Releases/32/ChangeSet#Binutils_2.33 > > > > https://fedoraproject.org/wiki/Changes/FontLangProvidesToLangpacks > > > > https://fedoraproject.org/wiki/Changes/golang1.14 > > > > https://fedoraproject.org/wiki/Changes/GCC10 > > > > https://fedoraproject.org/wiki/Changes/Free_Pascal_Compiler_3.2.0 > > > > > > > > we will start the mass rebuild on 2020-01-27 > > > > > > I thought the mass rebuild should be also for > > > https://fedoraproject.org/wiki/LTOByDefault > > > but the required changes AFAIK haven't landed into redhat-rpm-config yet. > > > > > > CCing Jeff on the current state. > > (snip) > > > After finally starting looking at the LTO vs GDB issues last week, I > > think we should defer LTO until F33. There's just too many GDB > > failures when used with LTO that aren't understood yet. > > Since the mass rebuild starts today (or has already started), you're > cutting it pretty close there ... > > So you'll either need to postpone to F33 or request a second mass > rebuild - soon. > > Can you comment on the FESCo ticket with the current status? Just to > keep us in the loop Umm, Ben and I were already discussing the situation this morning. I believe the page and fesco ticket both have current state (deferring to F33). jeff > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Non-responsive maintainer check for libffi maintainer
On Mon, 2020-02-24 at 13:02 +, devel- requ...@lists.fedoraproject.org wrote: > > Date: Mon, 24 Feb 2020 12:10:49 +0100 > From: Miro Hrončok > Subject: Re: Non-responsive maintainer check for libffi maintainer > To: Anthony Green > Cc: Development discussions related to Fedora > > Message-ID: > Content-Type: text/plain; charset=utf-8; format=flowed > > On 24. 02. 20 1:46, Anthony Green wrote: > >I would be very happy if somebody could pick up the libffi packaging > > responsibility from me. > > Suggestion: Try announcing on devel list that you are seeking co-maintainers > (from a separate thread). If that doesn't help, orphan the package and see > who > cares enough to take it. > > I've also CC'ed the RHEL Bugzilla default assignee, DJ Delorie. > > There are plenty of components that need libffi including various Python > versions and Ruby. At this point it probably makes sense to hand libffi off to DJ (who took it over in RHEL fairly recently). I can give DJ a heads-up. Jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: subarchitectures and RISC-V, was Re: F39 proposal: RPM 4.19 (System-Wide Change proposal)
On 4/11/23 19:14, przemek klosowski via devel wrote: On 4/4/23 10:28, Dan Čermák wrote: Chris Adams writes: Yeah, it'd be back to the i386/i586/i686 days... which was a bit of a PITA sometimes. But cramming multiple architectures of core libraries into a single RPM would be bad for disk space, image size, downloads, etc. But something that didn't exist in the i386/i686 days is containers - whether base images like for podman or full things like Flatpaks. Before going too deep into multi-level architectures, that should be taken into account. Afaik at least container runtimes do not support really support x86_64 subarchitectures: https://github.com/containers/podman/discussions/15256 The situation in the RISC-V universe is even more complicated because of all the extensions https://en.wikichip.org/wiki/risc-v/standard_extensions It'll be challenging to write and package software that is portable between all those variants---the fat binaries are just not practical due to combinatorial complexity, so it'll have to be either install-time or link-time configuration. Whatever standard scheme Fedora uses for x86 will hopefully be very useful for RiSC-V era that is apparently coming, with Linux-capable SBC boards like VisionFive ($65) and Pine64 ($70 and $8 (!) ox64) just becoming available, and a lot of activity on the horizon. Rather than trying to track all the individual extensions and combinations thereof, I would suggest focusing on RVI defined profiles. Essentially they provide a set of mandatory features the architecture must support (to be compliant with the profile). That may rule out certain processors, but it ultimately provides a higher performing baseline architecture for systems that are (hopefully) going to be good performing parts rather than embedded focused parts. jeff ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
