Dear GCC Community,
I have a bit of a question. I recently fixed up and deployed 2
separate implementations of a paper I am working on for C and C++
Standardization called #embed (found here -
https://thephd.github.io/vendor/future_cxx/papers/source/C%20-%20embed
).
I was trying to play with some of the semantics of implementing
this in GCC, beyond just doing plain expansion into an brace-init-list
of numbers. I noticed that that large payloads (firmware binary images
used in resets, lossless music data, generated config data, etc.)
takes quite a bit more memory and time to initialize and store in GCC;
Clang has much the same problem too, using ~2 GiB memory to handle a
20 MiB array.
My current working knowledge is that it takes a very long time to
parse, verify, and create a VECTOR_CST to use for array
initialization. I am wondering if it might be a good idea for
optimization purposes to introduce a new way of representing a blob of
binary data.
I was able to get extremely fast speeds in a test implementation
by making a special builtin called __builtin_array with keyword tag
RID_BUILTIN_ARRAY. The preprocessor directive would expand to
__builtin_array and one of its arguments was a string literal encoded
as base64 representing the original resource's binary data (so it
could survive re-preprocessing the file untouched). I would decode the
string literal and then build a static array of tree type STRING_CST
with the actual data.
It worked, but this approach required removing some type checks
in digest_init just to be able to fake-up a proper initialization from
a string literal. It also could not initialize data beyond `unsigned
char`, as that is what I had pinned the array representation to upon
creation of the STRING_CST.
I am new to this, and not really a compiler developer, so I was
wonder if anyone had any tips, tricks, or other places I should look
to learn something about better ways to do this or faster ways to
handle what is essentially large binary payloads! I appreciate any
feedback or thoughts tremendously.
Sincerely Yours,
JeanHeyd Meneide
