On Mon, Jun 20, 2022 at 10:37:57AM +0200, Przemysław Sztoch wrote: > But ligature check is performed on combining_ids (result of translation), > not on base codepoint. > Without it, you will get assertions in get_plain_letters.
Hmm. I am wondering if we could make the whole logic a bit more intuitive here. The loop that builds the set of mappings gets now much more complicated with the addition of the categories beginning by N for the numbers, and that's mostly the same set of checks as the ones applied for T. > However, the current Latin-ASCII.xml suggests a conversion to x. > I found an open discussion on the internet about this and the suggestion > that the Latin-ASCII.xml file should be corrected for this letter. > But I wouldn't expect that Unicode makes the revised Latin-ASCII.xml quickly > into the official repo. Yeah, Latin-ASCII.xml is getting it wrong here, then. unaccent fetches the thing from this URL currently: https://raw.githubusercontent.com/unicode-org/cldr/release-41/common/transforms/Latin-ASCII.xml Could it be better to handle that as an exception in generate_unaccent_rules.py, documenting why we are doing it this way then? My concern is somebody re-running the script without noticing this exception, and the set of rules would be blindly, and incorrectly, updated. -- Michael
signature.asc
Description: PGP signature
