On Mon, Jul 15, 2024 at 09:16:29AM +0200, Richard Biener wrote:
> > Nick has implemented a new .base64 directive in gas (to be shipped in
> > the upcoming binutils 2.43; big thanks for that).
> > See https://sourceware.org/bugzilla/show_bug.cgi?id=31964
> >
> > The following patch adjusts default_elf_asm_output_ascii (i.e.
> > ASM_OUTPUT_ASCII elfos.h implementation) to use it if it detects binary
> > data and gas supports it.
> >
> > Without this patch, we emit stuff like:
> > .string "\177ELF\002\001\001\003"
> > .string ""
> > .string ""
> > .string ""
> > .string ""
> > .string ""
> > .string ""
> > .string ""
> > .string "\002"
> > .string ">"
> > ...
> > .string "\324\001\236
> > 0FS\202\002E\n0@\203\004\005&\202\021\337)\021\203C\020A\300\220I\004\t\b\206(\234\0132l\004b\300\bK\006\220$0\303\020P$\233\211\002D\f"
> > etc., with this patch more compact
> > .base64
> > "f0VMRgIBAQMAAAAAAAAAAAIAPgABAAAAABf3AAAAAABAAAAAAAAAAACneB0AAAAAAAAAAEAAOAAOAEAALAArAAYAAAAEAAAAQAAAAAAAAABAAEAAAAAAAEAAQAAAAAAAEAMAAAAAAAAQAwAAAAAAAAgAAAAAAAAAAwAAAAQAAABQAwAAAAAAAFADQAAAAAAAUANAAAAAAAAcAAAAAAAAABwAAAAAAAAAAQAAAAAAAAABAAAABAAAAAAAAAAAAAAAAABAAAAAAAAAAEAAAAAAADBwOQAAAAAAMHA5AAAAAAAAEAAAAAAAAAEAAAAFAAAAAIA5AAAAAAAAgHkAAAAA"
> > .base64
> > "AACAeQAAAAAAxSSgAgAAAADFJKACAAAAAAAQAAAAAAAAAQAAAAQAAAAAsNkCAAAAAACwGQMAAAAAALAZAwAAAADMtc0AAAAAAMy1zQAAAAAAABAAAAAAAAABAAAABgAAAGhmpwMAAAAAaHbnAwAAAABoducDAAAAAOAMAQAAAAAA4MEeAAAAAAAAEAAAAAAAAAIAAAAGAAAAkH2nAwAAAACQjecDAAAAAJCN5wMAAAAAQAIAAAAAAABAAgAAAAAAAAgAAAAAAAAABAAAAAQAAABwAwAAAAAAAHADQAAAAAAAcANAAAAAAABAAAAAAAAAAEAAAAAAAAAACAAAAAAA"
> > .base64
> > "AAAEAAAABAAAALADAAAAAAAAsANAAAAAAACwA0AAAAAAACAAAAAAAAAAIAAAAAAAAAAEAAAAAAAAAAcAAAAEAAAAaGanAwAAAABoducDAAAAAGh25wMAAAAAAAAAAAAAAAAQAAAAAAAAAAgAAAAAAAAAU+V0ZAQAAABwAwAAAAAAAHADQAAAAAAAcANAAAAAAABAAAAAAAAAAEAAAAAAAAAACAAAAAAAAABQ5XRkBAAAAAw/WAMAAAAADD+YAwAAAAAMP5gDAAAAAPy7CgAAAAAA/LsKAAAAAAAEAAAAAAAAAFHldGQGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
> > .base64
> > "AAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAUuV0ZAQAAABoZqcDAAAAAGh25wMAAAAAaHbnAwAAAACYGQAAAAAAAJgZAAAAAAAAAQAAAAAAAAAvbGliNjQvbGQtbGludXgteDg2LTY0LnNvLjIAAAAAAAQAAAAwAAAABQAAAEdOVQACgADABAAAAAEAAAAAAAAAAQABwAQAAAAJAAAAAAAAAAIAAcAEAAAAAwAAAAAAAAAEAAAAEAAAAAEAAABHTlUAAAAAAAMAAAACAAAAAAAAAAOAAACsqAAAgS0AAOJWAAAjNwAAXjAAAAAAAAAAAAAAF1gAAHsxAABBBwAA"
> > .base64
> > "G0kAALGmAACwoAAAAAAAAAAAAACQhAAAAAAAAOw1AACNYgAAAAAAAFQoAAAAAAAAx3UAALZAAAAAAAAAiIUAALGeAABBlAAAWEsAAPmRAACmOgAAAAAAADh3AAAAAAAAlCAAAAAAAABymgAAaosAAMIjAAAKMQAAMkIAADU0AAAAAAAA5ZwAAAAAAAAAAAAAAAAAAFIdAAAIGQAAAAAAAMFbAAAoTQAAGDcAAIRgAAA6HgAAlxwAAAAAAADOlgAAAAAAAEhPAAARiwAAMGgAAOVtAADMFgAAAAAAAAAAAACrjgAAYl4AACZVAAA/HgAAAAAAAAAAAABqPwAAAAAA"
> > The patch attempts to juggle between readability and compactness, so
> > if it detects some hunk of the initializer that would be shorter to be
> > emitted as .string/.ascii directive, it does so, but if it previously
> > used .base64 directive it switches mode only if there is a 16+ char
> > ASCII-ish string.
> >
> > On my #embed testcase from yesterday
> > unsigned char a[] = {
> > #embed "cc1plus"
> > };
> > without this patch it emits 2.4GB of assembly, while with this
> > patch 649M.
> > Compile times (trunk, so yes,rtl,extra checking) are:
> > time ./xgcc -B ./ -S -std=c23 -O2 embed-11.c
> >
> >
> >
> >
> >
> > real 0m13.647s
> >
> >
> > user 0m7.157s
> >
> >
> > sys 0m2.597s
> >
> >
> > time ./xgcc -B ./ -c -std=c23 -O2 embed-11.c
> >
> >
> >
> >
> >
> > real 0m28.649s
> >
> >
> > user 0m26.653s
> >
> >
> > sys 0m1.958s
> >
> >
> > without the patch and
> > time ./xgcc -B ./ -S -std=c23 -O2 embed-11.c
> >
> > real 0m4.283s
> > user 0m2.288s
> > sys 0m0.859s
> > time ./xgcc -B ./ -c -std=c23 -O2 embed-11.c
> >
> > real 0m6.888s
> > user 0m5.876s
> > sys 0m1.002s
> > with the patch, so that feels like significant improvement.
> > The resulting embed-11.o is identical between the two ways of expressing
> > the mostly binary data in the assembly. But note that there are portions
> > like:
> > .base64
> > "nAAAAAAAAAAvZRcAIgAOAFAzMwEAAAAABgAAAAAAAACEQBgAEgAOAFBHcAIAAAAA7AAAAAAAAAAAX19nbXB6X2dldF9zaQBtcGZyX3NldF9zaV8yZXhwAG1wZnJfY29zaABtcGZyX3RhbmgAbXBmcl9zZXRfbmFuAG1wZnJfc3ViAG1wZnJfdGFuAG1wZnJfc3RydG9mcgBfX2dtcHpfc3ViX3VpAF9fZ21wX2dldF9tZW1vcnlfZnVuY3Rpb25zAF9fZ21wel9zZXRfdWkAbXBmcl9wb3cAX19nbXB6X3N1YgBfX2dtcHpfZml0c19zbG9uZ19wAG1wZnJfYXRh"
> > .base64
> > "bjIAX19nbXB6X2RpdmV4YWN0AG1wZnJfc2V0X2VtaW4AX19nbXB6X3NldABfX2dtcHpfbXVsAG1wZnJfY2xlYXIAbXBmcl9sb2cAbXBmcl9hdGFuaABfX2dtcHpfc3dhcABtcGZyX2FzaW5oAG1wZnJfYXNpbgBtcGZyX2NsZWFycwBfX2dtcHpfbXVsXzJleHAAX19nbXB6X2FkZG11bABtcGZyX3NpbmgAX19nbXB6X2FkZF91aQBfX2dtcHFfY2xlYXIAX19nbW9uX3N0YXJ0X18AbXBmcl9hY29zAG1wZnJfc2V0X2VtYXgAbXBmcl9jb3MAbXBmcl9zaW4A"
> > .string "__gmpz_ui_pow_ui"
> > .string "mpfr_get_str"
> > .string "mpfr_acosh"
> > .string "mpfr_sub_ui"
> > .string "__gmpq_set_ui"
> > .string "mpfr_set_inf"
> > ...
> > .string "GLIBC_2.14"
> > .string "GLIBC_2.11"
> > .base64
> > "AAABAAIAAQADAAMAAwADAAMAAwAEAAUABgADAAEAAQADAAMABwABAAEAAwADAAMAAwAIAAEAAwADAAEAAwABAAMAAwABAAMAAQADAAMAAwADAAMAAwADAAYAAwADAAEAAQAIAAMAAwADAAMAAwABAAMAAQADAAMAAQABAAEAAwAIAAEAAwADAAEAAwABAAMAAQADAAEABgADAAMAAQAHAAMAAwADAAMAAwABAAMAAQABAAMAAwADAAkAAQABAAEAAwAKAAEAAwADAAMAAQABAAMAAwALAAEAAwADAAEAAQADAAMAAwABAAMAAwABAAEAAwADAAMABwABAAMAAwAB"
> > .base64
> > "AAEAAwADAAEAAwABAAMAAQADAAMAAwADAAEAAQABAAEAAwADAAMAAQABAAEAAQABAAEAAQADAAMAAwADAAMAAQABAAwAAwADAA0AAwADAAMAAwADAAEAAQADAAMAAQABAAMAAwADAAEAAwADAAEAAwAIAAMAAwADAAMABgABAA4ACwAGAAEAAQADAAEAAQADAAEAAwABAAMAAwABAAEAAwABAAMAAwABAAEAAwADAAMAAwABAAMAAQABAAEAAQABAAMADwABAAMAAQADAAMAAwABAAEAAQAIAAEADAADAAMAAQABAAMAAwADAAEAAQABAAEAAQADAAEAAwADAAEA"
> > .base64
> > "AwABAAMAAQADAAMAAQABAAEAAwADAAMAAwADAAMAAQADAAMACAAQAA8AAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQA="
> > so it isn't all just totally unreadable stuff.
> >
> > Tested on top of the #embed patchset (just to get the above
> > data) and then with full bootstrap/regtest on top of vanilla trunk,
> > on x86_64-linux against trunk binutils prepended into $PATH (HAVE_GAS_BASE64
> > is true) and i686-linux (against older binutils, HAVE_GAS_BASE64 not
> > defined), ok for trunk?
>
> OK.
>
> Do we want to have a command-line switch controlling the use of
> .base64? It might be the 16+ char limit is too large for the
> readability purpose and/or with wide-strings, etc. .base64 is
> unwanted to some people at least when debugging? (does gas
> support sth like .wstring or UTF-8/16/32 input?) I don't have
> a good suggestion for the flag name, a quick scan for assembler
> related options doesn't reveal a nice "pattern"; -f[no-]base64-in-asm?
There is no directive to emit wide etc. strings, so UCS-2/UTF-16/UTF-32
are definitely barely readable even now.
const char8_t a[] = u8"Žluťoučký";
const char16_t b[] = u"Žluťoučký";
const char32_t c[] = U"Žluťoučký";
The first one is
.string "\305\275lu\305\245ou\304\215k\303\275"
so one can with a magnifying glass find the ASCII chars in there,
the second one
.string "}\001l"
.string "u"
.string "e\001o"
.string "u"
.string "\r\001k"
.string "\375"
.string ""
.string ""
on little endian and third
.string "}\001"
.string ""
.string "l"
.string ""
.string ""
.string "u"
.string ""
.string ""
.string "e\001"
.string ""
.string "o"
.string ""
.string ""
.string "u"
.string ""
.string ""
.string "\r\001"
.string ""
.string "k"
.string ""
.string ""
.string "\375"
.string ""
.string ""
.string ""
.string ""
.string ""
.string ""
I think that is simply binary rubbish.
The way the patch is written is that if it sees a limited string (i.e. zero
terminated) string and it is in itself shorter than base64 encoding, then it
emits it as such, so
.string "foobar"
should be emitted like that, or even
.string "barbaz\305\275foobarbar"
but once if is not cheaper, it chooses base64 in the ~ 256 char chunks
including the '\0's too. The 16 char limit is there just to avoid
.base64 "somethingveryloooooooooooooooooooooooooooooooong"
.string "I"
.base64 "somethingverylongagain"
which did happen but only with short readable strings that appear at the
~ 256 char boundaries, others in the middle of base64 were still emitted
unreadably. And it happens only after a previous .base64.
I think best way to read partially binary data is objdump or similar,
but if people insist otherwise, -fno-base64-data-asm is possible.
Jakub
