> On 16 Jan 2023, at 18:10, Anthony PERARD <anthony.per...@citrix.com> wrote:
>
> The get-fields.sh which generate all the include/compat/.xlat/*.h
> headers is quite slow. It takes for example nearly 3 seconds to
> generate platform.h on a recent machine, or 2.3 seconds for memory.h.
>
> Rewriting the mix of shell/sed/python into a single python script make
> the generation of those file a lot faster.
>
> No functional change, the headers generated are identical.
>
> Signed-off-by: Anthony PERARD <anthony.per...@citrix.com>
> ---
>
> Notes:
> To test the header generation, I've submit a branch to gitlab ci,
> where all the headers where generated, and for each one both the shell
> script and the python script where run and the result of both
> compared.
>
> v3:
> convert to python script instead of perl
> - this should allow more developper do be able to review/edit it.
> - it avoid adding a dependency on perl for the hypervisor build.
>
> It can be twice as slow than the perl version, but overall, when doing
> a build with make, there isn't much difference between the perl and
> python script.
> We might be able to speed the python script up by precompiling the
> many regex, but it's probably not worth it. (python already have a
> cache of compiled regex, but I think it's small, maybe 10 or so)
>
> v2:
> - Add .pl extension to the perl script
> - remove "-w" from the shebang as it is duplicate of "use warning;"
> - Add a note in the commit message that the "headers generated are
> identical".
>
> xen/include/Makefile | 6 +-
> xen/tools/compat-xlat-header.py | 468 ++++++++++++++++++++++++++++
> xen/tools/get-fields.sh | 528 --------------------------------
> 3 files changed, 470 insertions(+), 532 deletions(-)
> create mode 100644 xen/tools/compat-xlat-header.py
> delete mode 100644 xen/tools/get-fields.sh
>
> diff --git a/xen/include/Makefile b/xen/include/Makefile
> index 65be310eca..b950423efe 100644
> --- a/xen/include/Makefile
> +++ b/xen/include/Makefile
> @@ -60,9 +60,7 @@ cmd_compat_c = \
>
> quiet_cmd_xlat_headers = GEN $@
> cmd_xlat_headers = \
> - while read what name; do \
> - $(SHELL) $(srctree)/tools/get-fields.sh "$$what" compat_$$name $< ||
> exit $$?; \
> - done <$(patsubst $(obj)/compat/%,$(obj)/compat/.xlat/%,$(basename
> $<)).lst >$@.new; \
> + $(PYTHON) $(srctree)/tools/compat-xlat-header.py $< $(patsubst
> $(obj)/compat/%,$(obj)/compat/.xlat/%,$(basename $<)).lst > $@.new; \
> mv -f $@.new $@
>
> targets += $(headers-y)
> @@ -80,7 +78,7 @@ $(obj)/compat/%.c: $(src)/public/%.h $(srcdir)/xlat.lst
> $(srctree)/tools/compat-
> $(call if_changed,compat_c)
>
> targets += $(patsubst compat/%, compat/.xlat/%, $(headers-y))
> -$(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst
> $(srctree)/tools/get-fields.sh FORCE
> +$(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst
> $(srctree)/tools/compat-xlat-header.py FORCE
> $(call if_changed,xlat_headers)
>
> quiet_cmd_xlat_lst = GEN $@
> diff --git a/xen/tools/compat-xlat-header.py b/xen/tools/compat-xlat-header.py
> new file mode 100644
> index 0000000000..c1b361ac56
> --- /dev/null
> +++ b/xen/tools/compat-xlat-header.py
> @@ -0,0 +1,468 @@
> +#!/usr/bin/env python
Would it make sense to start with python3 since it is a new script?
