> From: Costas Argyris
> Date: Sat, 18 Mar 2023 16:37:20 +
>
> This is a proposed patch to enable UTF-8 support in GNU Make running on
> Windows host.
Thanks.
> Today, the make process on Windows is using the legacy system code page
> because of the "A" functions
> called in the source code.This means that any UTF-8 input to make on
> Windows will break.A few
> examples follow:
Yes, this misfeature of using the system codepage is known, together
with the consequences.
> The attached patch incorporates the UTF-8 manifest into the build process of
> GNU Make when hosted on
> Windows, and forces the built executable to use UTF-8 as its active code
> page, solving all problems shown
> above because this has a global effect in the process.All existing "A"
> calls use the UTF-8 code page now
> instead of the legacy one.This is the relevant Microsoft doc:
>
> https://learn.microsoft.com/en-us/windows/apps/design/globalizing/use-utf8-code-page
>
> With the patch, after building make, the above cases now work on Windows:
>
> ##
> C:\Users\cargyris\temp>cat utf8Makefile.mk
> hello :
> @echo ﹏
> @echo ❎
> C:\Users\cargyris\temp>make -f utf8Makefile.mk
> ﹏
> ❎
>
> C:\Users\cargyris\temp>make -f ❎\utf8Makefile.mk
> ﹏
> ❎
>
> C:\Users\cargyris\temp>cd ❎
>
> C:\Users\cargyris\temp\❎>make -f utf8Makefile.mk
> ﹏
> ❎
>
> C:\Users\cargyris\temp\❎>make -f ❎\utf8Makefile.mk
> ﹏
> ❎
> ##
>
> This change might also fix other existing issues on Windows having to do with
> filenames and paths, but I
> can't point at something particular right now.
>
> Would a patch like that be considered?
Yes, of course.
However, we need to understand better the conditions under which the
UTF-8 support in Make will be activated, and the consequences of
activating it. Here are some specific questions, based on initial
thinking about this:
. Does this support require Make to be linked against the UCRT
run-time library, or does it also work with the older MSVCRT? If
Make is built with MSVC, does it have to be built with some new
enough version of Studio to have the necessary run-time support
for this feature, or any version will do?
. Does using UTF-8 as the active page in Make mean that
locale-dependent C library functions will behave as expected? For
example, what happens with character classification functions such
as isalpha and isdigit, and what happens with functions related to
letter-case, such as tolower and stricmp -- will they perform
correctly with characters in the entire Unicode range? (This
might be related to the first question above.)
. Did you try running Make with this manifest on older Windows
systems, like Windows 8.1 or 7? It is important to make sure this
manifest doesn't preclude Make from running on those older
systems, even though the UTF-8 feature will then be unavailable.
. When Make invokes other programs (which it does quite a lot ;-),
and passes command-line arguments to it with non-ASCII characters,
what will happen to those non-ASCII characters? I'm guessing that
if the program also has such a manifest, it will get the UTF-8
encoded strings verbatim, but what if it doesn't have such a
manifest? (The vast majority of the programs Make invokes
nowadays don't have such manifests.) Will Windows convert the
UTF-8 encoded strings into the system codepage, or will the
program get UTF-8 regardless of whether it can or cannot handle
them? If the latter, it will become impossible to use non-ASCII
strings and file names with such programs even if those non-ASCII
characters can be represented using the current system ANSI
codepage, because most programs Make invokes on Windows don't
support UTF-8. Your examples invoked only the built-in commands
of cmd.exe, but what happens if you instead invoke, say, GCC, and
pass it a non-ASCII file name, including a file name which cannot
be represented in the current ANSI codepage?
. Even if the answer to the previous question is, as I expect, that
Windows will convert UTF-8 encoded strings to the current ANSI
codepage, it is important to understand that with the UTF-8 active
codepage enabled Make will still be unable to invoke programs with
UTF-8 encoded strings if those programs don't have the same UTF-8
active codepage enabled, except if the non-ASCII characters in
those strings can be represented by the current ANSI codepage. So
this feature will only be complete when the programs invoked by
Make are also UTF-8 capable.
A specific comment on your patch:
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -46,6 +46,8 @@ w32_SRCS = src/w32/pathstuff.c src/w32/w32os.c
> src/w32/compat/dirent.c \
> src/w32/subproc/misc.c src/w32/subproc/proc.h \
> src/w32/subproc/sub_proc.c src/w32/subproc/w32err.c
