#36980: TranslationCatalog.__getitem__ gives wrong priority to non-plural
translations when plural forms mismatch
-------------------------------------+-------------------------------------
Reporter: UHHHHHHHHHHHHHH | Type: Bug
Status: new | Component:
| Internationalization
Version: 5.1 | Severity: Normal
Keywords: i18n translation | Triage Stage:
plural TranslationCatalog | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
= Description =
When a third-party package (e.g. django-allauth) has a slightly different
`Plural-Forms` header than the project's own translations (e.g. `plural=n
!= 1` vs `plural=(n != 1)`), `TranslationCatalog.update()` correctly
prepends a separate catalog to preserve plural lookup correctness.
However, `TranslationCatalog.__getitem__()` iterates catalogs in order and
returns the '''first match'''. This means the prepended catalog wins for
'''all''' lookups, including non-plural `gettext()` calls where plural
form separation is irrelevant. This silently overrides higher-priority
translations (from `LOCALE_PATHS` or later `INSTALLED_APPS`) with lower-
priority ones.
= Steps to reproduce =
1. Create a Django project with a translation in `LOCALE_PATHS`:
{{{
# project/locale/nl/LC_MESSAGES/django.po
# Plural-Forms: nplurals=2; plural=(n != 1);
msgid "Activate"
msgstr "Activeren"
}}}
2. Install a third-party app (e.g. django-allauth) that defines the same
msgid with a different `Plural-Forms` header:
{{{
# allauth/locale/nl/LC_MESSAGES/django.po
# Plural-Forms: nplurals=2; plural=n != 1;
msgid "Activate"
msgstr "Activeer"
}}}
Note: `n != 1` and `(n != 1)` are functionally identical and compile to
identical bytecode, but Python's `gettext.c2py()` produces functions with
different `__code__` objects. Django's `TranslationCatalog.update()`
compares `__code__` to decide whether to merge or prepend.
3. `gettext("Activate")` returns `"Activeer"` (allauth's version) instead
of `"Activeren"` (the project's version), even though `LOCALE_PATHS`
should have highest priority per Django's documentation.
= Root cause =
In `django/utils/translation/trans_real.py`:
{{{#!python
class TranslationCatalog:
def update(self, trans):
# Mismatched plural -> prepend to position 0
for cat, plural in zip(self._catalogs, self._plurals):
if trans.plural.__code__ == plural.__code__:
cat.update(trans._catalog)
break
else:
self._catalogs.insert(0, trans._catalog.copy())
self._plurals.insert(0, trans.plural)
def __getitem__(self, key):
# First catalog wins for ALL lookups
for cat in self._catalogs:
try:
return cat[key]
except KeyError:
pass
raise KeyError(key)
}}}
The `update()` prepend is correct for plural lookups (each catalog needs
its own plural function). But `__getitem__` shouldn't give the prepended
catalog priority for non-plural string lookups — these should follow the
normal priority order (LOCALE_PATHS > INSTALLED_APPS > Django built-in).
= Expected behavior =
Non-plural `gettext()` lookups should respect the documented translation
priority order regardless of plural form differences between catalogs.
= Actual behavior =
A third-party package with a cosmetically different `Plural-Forms` header
silently overrides all matching non-plural translations from higher-
priority sources.
= Impact =
In our project, this causes 28 translations to be silently wrong. The
project has no way to fix this without either:
* Monkey-patching `TranslationCatalog`
* Patching the third-party package's `.po` files in the Docker build
* Adding `pgettext` context to every conflicting string
= Environment =
* Django 5.1 (also affects 5.0, likely all versions since #34221 was
fixed)
* Python 3.12
* The issue was introduced/amplified by the fix for #34221 which changed
`update()` to only merge with the top catalog
--
Ticket URL: <https://code.djangoproject.com/ticket/36980>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/0107019cdc6276ec-c43be7a8-738c-4533-af80-13fab9a6fa8e-000000%40eu-central-1.amazonses.com.
