On Fri, May 13, 2022 at 3:38 AM Philipp Tomsich <philipp.toms...@vrull.eu> wrote:
> On Fri, 13 May 2022 at 12:00, Christoph Müllner <cmuell...@gcc.gnu.org> > wrote: > > > > On Wed, May 11, 2022 at 2:02 AM Palmer Dabbelt <pal...@dabbelt.com> > wrote: > > > > > > [Sorry for cross-posting to a bunch of lists, I figured it'd be best to > > > have all the discussions in one thread.] > > > > > > We currently only support what is defined by official RISC-V > > > specifications in the various GNU toolchain projects. There's > certainly > > > some grey areas there, but in general that means not taking code that > > > relies on drafts or vendor defined extensions, even if that would > result > > > in higher performance or more featured systems for users. > > > > > > The original goal of these policies were to steer RISC-V implementers > > > towards a common set of specifications, but over the last year or so > > > it's become abundantly clear that this is causing more harm that good. > > > All extant RISC-V systems rely on behaviors defined outside the > official > > > specifications, and while that's technically always been the case we've > > > gotten to the point where trying to ignore that fact is impacting real > > > users on real systems. There's been consistent feedback from users > that > > > we're not meeting their needs, which can clearly be seen in the many > out > > > of tree patch sets in common use. > > > > > > There's been a handful of discussions about this, but we've yet to have > > > a proper discussion on the mailing lists. From the various discussions > > > I've had it seems that folks are broadly in favor of supporting vendor > > > extensions, but the devil's always in the details with this sort of > > > thing so I thought it'd be best to write something up so we can have a > > > concrete discussion. > > > > > > The idea is to start taking code that depends on vendor-defined > behavior > > > into the core GNU toolchain ports, as long as it meets the following > > > criteria: > > > > > > * An ISA manual is available that can be redistributed/archived, > defines > > > the behaviors in question as one or more vendor-specific extensions, > > > and is clearly versioned. The RISC-V foundation is setting various > > > guidelines around how vendor-defined extensions and instructions > > > should be named, we strongly suggest that vendors follow those > > > conventions whenever possible (this is all new, though, so exactly > > > what's necessary from vendor specifications will likely evolve as we > > > learn). > > > * There is a substantial user base that depends on the behavior in > > > question, which probably means there is hardware in the wild that > > > implements the extensions and users that require those extensions in > > > order for that hardware to be useful for common applications. This > is > > > always going to be a grey area, but it's essentially the same spot > > > everyone else is in. > > I must take exception to the "There is a substantial user base" rule, > as this conflicts with the goal of avoiding fragmentation: the support > for vendor-defined extensions should ideally have landed in an > upstream release before the silicon is widely released. The "substantial user base" standard is specious. Other recent emails have suggested that phrase is a euphemism for "widely available consumer-programmable product." It fails to account for RISC-V's most important constituency: people using open-source toolchains to program non-mass-market parts. Our existing regime of "no custom extension support in standard software" is draconian, but at least it's self-consistent. I'd support retaining it. But, if we do change it, it's preposterous to reject extensions like XVentanaCondOps on the basis that their silicon isn't available on AliExpress. This would > see these extensions being sent upstream significantly before > widespread sampling (and sometimes around the time of the announcement > of a device on the roadmap). Simply put: I want everyone defining > vendor extensions to contribute to our mainline development efforts > instead of extending their own ancient forks. > > I suspect that this rule is intended to ensure that experimental, > purely academic, or "closed" (as in: even if you have the silicon, it > will be so deeply embedded that no one can run their own software — > e.g. radio baseband controllers) extensions don't make the maintenance > work harder. If that is the case: could we use wording such as (a > native speaker might wordsmith something more accurate) "accessible to > run user-developed software" and "intended for a wider audience"? > > > > * There is a mechanism for testing the code in question without direct > > > access to hardware, which in practice means a QEMU port (or whatever > > > simulator is relevant in the space and that folks use for testing) or > > > some community commitment to long-term availability of the hardware > > > for testing (something like the GCC compile farm, for example). > > > * It is possible to produce binaries that are compatible with all > > > upstream vendors' implementations. That means we'll need mechanisms > > > to allow extensions from multiple vendors to be linked together and > > > then probed at runtime. That's not to say that all binaries will be > > > compatible, as users are always free to skip the compatibility code > > > and there will be conflicting definitions of instruction encodings, > > > but we can at least provide users with the option of compatibility. > > We today have: > - Tag_RISCV_arch (see > > https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/master/riscv-elf.adoc#tag_riscv_arch-5-ntbssubarch > ) > - ifunc support > > Admittedly, there's some loose ends in the end-to-end story (e.g. > Unified Discovery -> DTB -> glibc ifunc initialisation): we know just > too well how this plays out as there are optimised string/memory > functions (Zbb, Zicboz, cache-block-length, …) in our pipeline as well > as OpenSSL support for Zbb and Zbc. However, this is a known gap and > will be fully addressed in the future. > > Is there something specific beyond this that you'd be looking for? > > Thanks, > Philipp. > > > > > > > These are pretty loosely written on purpose, both because this is all > > > new and because each project has its own set of contribution > > > requirements so it's going to be all but impossible to have a single > > > concrete set of rules that applies everywhere -- that's nothing > specific > > > to the vendor extensions (or even RISC-V), it's just life. > Specifically > > > a major goal here is to balance the needs of users, both in the short > > > term (ie, getting new hardware to work) and the long term (ie, the long > > > term stability of their software). We're not talking about taking code > > > that can't be tested, hasn't been reviewed, isn't going to be supported > > > long-term, or doesn't have a stable ABI; just dropping the specific > > > requirement that a specification must be furnished by the RISC-V > > > foundation in order to accept code. > > > > > > Nothing is decided yet, so happy to hear any thought folks have. This > > > is certainly a very different development methodology than what we've > > > done in the past and isn't something that should be entreated into > > > lightly, so any comments are welcome. > > > > I'd like to add two points to this topic and raise two questions. > > > > 1) Accepting vendor extensions = avoidance of fragmentation > > > > RISC-V implementors are actively encouraged to implement their > > own ISA extensions. To avoid fragmentation in the SW ecosystem > > (every vendor maintains a fork of tools, distros and binaries) there > > needs to be a principle acceptance to get vendor extension support > > upstream. > > > > 2) Leading upstream maintainers already agreed on supporting > vendor-extensions > > > > We have discussed the topic of vendor extensions in many forums last > year. > > This topic was also part of the discussions at the Linux Plumbers > conference. > > Further, there exists a place for documenting toolchain conventions of > > the RISC-V > > ecosystem ([1]), which everyone in the RISC-V ecosystem is aware about. > > > > As a result of the discussions last year, a PR ([2]) has been crafted to > clarify > > the rules for upstreaming vendor extension support. These RISC-V > > toolchain conventions recently added a section for vendor extensions > > which covers important aspects like: > > > > * naming schemes > > * assembly mnemonic prefixes > > * links to the documentation and version information > > > > The PR even includes an explicit rule to clarify that maintainers decide > on > > upstream inclusion: > > """ > > Open source toolchain maintainer has final say on accepting vendor > extension, > > comply with this conventions isn't guarantee upstream will accept. > > """ > > > > A lot of people (maintainers and active developers) were notified > > (including you) > > and many also actively contributed to the PR. In the end there was an > agreement > > of how to document vendor extensions (as a requirement for upstreaming). > > > > I believe that your set of rules is compatible with what is specified > there. > > Note however, that you could have mentioned them during the PRs review > > process as you were notified when the PR was created. > > > > My questions are now the following: > > > > * Where to document the requirements? > > > > Most RISC-V upstream maintainers are accepting the > riscv-toolchains-convention > > repository. Where if not there should we document requirements for the > tool's > > conventions? Why not accept what has already been agreed upon? > > > > * Where to track the status? > > > > You mentioned testing requirements (e.g. QEMU support). > > I've created a wiki page to show the adoption status of all the > > RISC-V extensions ([3]) > > last year. As the chair of the Toolchains SIG, I'm willing to create > > and maintain one for > > vendor extensions as well. This allows users to see which projects > > support which > > extensions upstream. However, a wiki is a joint effort. So > > maintainers that merge > > changes upstream need to update the page. Will you support this? > > If not what is your proposal to track the status of the upstream > > extension support? > > > > BR > > Christoph > > > > [1] > https://github.com/riscv-non-isa/riscv-toolchain-conventions#conventions-for-vendor-extension > > [2] https://github.com/riscv-non-isa/riscv-toolchain-conventions/pull/17 > > [3] > https://wiki.riscv.org/display/HOME/RISC-V+extension+and+feature+support+in+the+Open+Source+SW+Ecosystem >
