On Tue, 2018-07-17 at 14:49 +0200, Martin Liška wrote: > Hi. > > I've recently touched AWK option generate machinery and it's quite > unpleasant > to make any adjustments. My question is simple: can we starting using > a scripting > language like Python and replace usage of the AWK scripts? It's > probably question > for Steering committee, but I would like to see feedback from > community.
As you know, I'm a fan of Python. As I noted elsewhere in this thread, one issue is Python 2 vs Python 3 (and minimum versions). Within Python 2.*, Python 2.6 onwards is broadly compatible with Python 3.*, and there's a well-known common subset that works in both languages. To what extent would this complicate bootstrap? (I don't think so, in that it would appear to be just an external build-time dependency on the build machine). Would this make it harder for people to build GCC? It's one more dependency, but CPython is widely available and relatively easy to build. (I don't have experience of doing bring-up of a new architecture, though). > There are some bulletins why I would like to replace current AWK > scripts: > > 1) gcc/optc-save-gen.awk is full of copy&pasted code, due to lack of > flags type classes multiple > global variables are created (var_opt_char, var_opt_string, ...) > > 2) similar happens in gcc/opth-gen.awk > > 3) we do very many regex matches (mainly in gcc/opt-functions.awk), I > believe > we should come up with a structured option format that will make > parsing and > processing much simpler. > > 4) we can come up with new sanity checks of options: > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81397 Having some kind of .opt linting sounds useful. > 5) there are various targets that generate *.opt files, one example > is ARM: > gcc/config/arm/parsecpu.awk > > where transforms: > ./gcc/config/arm/arm-cpus.in > > I guess having a well-defined structured format for *.opt files will > make > it easier to write generated opt files? > > I'm attaching a prototype that can transform optionlist into options- > save.c > that can be compiled and works. > > I'm looking forward to a feedback. > Martin You named it "gcc-options.py", but I think we'll want something that can be imported from other scripts, and this isn't valid to "import" as a module, due to the "-". It should have a filename that either uses an underscore, or no separator. > # parse content of optionlist It's probably worth moving this into a class. Maybe: class OptionList: def __init__ (self, lines): # etc or similar. "optimization_flags" could be a member of that class. > # start printing This ought to be in a function, rather than having this at the top- level. Moving it into a function would allow for some unittest tests: (a) tests of parsing some lines provided as string literals, to unit- test the parser. (b) integration tests of parsing the actual optionlist, maybe. perhaps via a --unit-test command-line option to trigger unittest.main(). Maybe a way to ensure no semantic changes during the transition would be to diff the generated .c/.h files compared to the awk files, and verifying that there are no significant whitespace changes, for all supported configs? Hope this is constructive. Dave
