I want to write a new calling convention into the GNU g++ compiler, specifically for the x86_64 instruction set.
On MS-Windows, the x64 calling convention puts the first argument in RCX, and puts the return value in RAX. I wish to write a new calling convention which uses RAX for both the first argument and the return value, to be used as follows: unsigned Func(unsigned const a) __attribute__((gotham_call)) { return a * a; } Use of this new calling convention would shave a few CPU cycles off some function invocations, as the above function which previously was compiled to: imul rcx, rcx mov rax, rcx ret would now be compiled to: imul rax, rax ret Could someone please point me to the GNU compiler source files in which I would add a new calling convention? I would appreciate a few pointers on how to add a new calling convention to the GNU g++ compiler.