Date: Mon, 28 Nov 2022 09:18:25 +0100 >> `texindex` is just one part of the problem.[*] We also need a UTF-8 >> locale for both `makeinfo` and `lilypond-book` – without it, the >> output for both PDF and HTML is simply wrong. My standard example >> is the incorrect display of the degree sign in the documentation of >> the '\rotate' markup command: >> >> https://lilypond.org/doc/v2.23/Documentation/notation/align >> >> In other words, such a test for `configure.ac` is inevitable. We >> might fix `lilypond-book` to emit a warning or even abort if it >> can't switch to a UTF-8 locale internally, but `makeinfo` (and soon >> `texi2any`, as a replacement for `texi2html`) is out of our >> control. > > I would like to understand why this happens. The French translation > has accents everywhere, and I never saw problems while reading > it. Judging from the image, the texi input itself output by > lilypond-book is buggy (since LilyPond does work on UTF-8).
Yes, it seems so. Maybe there is still a writing instruction in `lilypond-book` not marked as being Unicode? > Any reliance of lilypond-book on the locale encoding sounds > unexpected (except for encoding file paths for the > filesystem). lilypond-book should just use UTF-8 Everywhere, UTF-8 > Always, UTF-8 Only (TM, or rather ™). Yep. > At any rate, Python provides all the tools to work independently > from the locale encoding. (I look forward to dropping Python 3.6 > support. Then we’ll be able to use UTF-8 mode, which makes file > reads and the like default to UTF-8. Then we won’t need > encoding="utf-8" anymore. This mode will be the default in Python > 3.15.) I'm not sure when we can drop Python 3.6... >> [*] Even if rewritten in Perl, we eventually need language-specific >> locales to get correct collation in indices. This means that >> for building the German documentation, `de_DE.UTF-8` should be >> used, etc., etc. For this, however, there are ready-to-use >> autoconf macros available in 'gnulib'. If there are better >> suggestions to avoid such additional locale tests, I'm all >> ears. > > Doesn’t Perl implement the Unicode collation algorithm? It does, but AFAICS it does *not* have the locale data for sorting built in. In https://perldoc.perl.org/perllocale you can read Definitions for locales that you use must be installed. This means again that we have to test for language-specific UTF-8 locales... Werner
