Hi,
PROBLEM: garbage po and mo files in packages
currently many of the scripts used for creating release tarballs of KDE
software accidentally also add those po files to the tarballs whose
translations are already fed back directly into the sources by scripty (with
data files like appdata, desktop, json, or mimetype.xml). So strings which
will never be loaded from an external mo or qm catalog.
Those po files then also end up in the binary packages as mo files given the
generic handling of po files during builds. Do this to see how much of those
unneeded mo files you might have yourself on your system:
du -ch `find /usr/share/locale \
-name "*appdata.mo" -o \
-name "*mimetypes.mo" -o \
-name "desktop_*.mo" -o \
-name "json_*.mo"`
which for me reports in total 43M.
find ... | wc -l
is 186 here.
Problem is that many people are not aware of all the different data file types
which scripty cares for, and i18n in general is more seen as "magic which just
works (mostly)" (well, shows the great work of the KDE i18n team :) ). And the
number of such file types is rising now and then, just all the different
release scripts ("official" and home-grown) are not keeping up.
PROPOSED SOLUTION
To keep things simple, what about separating pot/po files used with strings
fed back directly into the sources in a separate folder system? Similar to how
po/pot files with manual/doc strings are hold in a separate docmessages/
folder?
So the structure in svn would be
l10n-kf5/{templates,af,ar,...}/messages
l10n-kf5/{templates,af,ar,...}/docmessages
l10n-kf5/{templates,af,ar,...}/srcmessages < new one
This should help to make it more clear which po files should end up as mo (or
qm) files and which po files are just used for allowing translators to
translate strings directly in data files.
And it would allow release scripts to not be too clever about which po files
to get from svn and which not.
Unsure about this:
What better name could there be than "srcmessages"?
Can l10n-kde3/l10n-kde4/l10n-support also be handled the same way, without
breaking things?
Did I miss anything and those po/mo files should be part of source/binary
packages?
I assume moving those po/pot file into a separate folder "only" needs moving
of the files, adaption of scripty and translators making sure their files in
their checkout of svn are properly moved else well, given any uncommited
changes. And noone else should be affected, right?
Please comment also in general.
INTERMEDIATE FIXING OF RELEASE SCRIPTS
In the meantime authors of release scripts perhaps should be inspired by the
approach taken by the script preparing KF5 tarballs, which greps all
Messages.sh files in the sources for "*.pot" to guess about the names of po
files that should be added to the tarball. Because this will not include pot
files only created by scripty on the i18n server. Thus might be a good guess
(unless someone wrote Messages.sh with too much magic inside).
See git repo kde:sysadmin/release-tools, branch frameworks/5.0, file
make_rc_tag.sh or
https://quickgit.kde.org/?p=sysadmin%2Frelease-tools.git&a=blob&h=452db8ae4f18931771997d406d63687f9053eed3&hb=f2b88c76b687f55a4123c074d859d0f7b2d2e557&f=make_rc_tag.sh
(Using the term "guess" hints all this is still a fragile approach and perhaps
could see a better approach, but please let's spare that discussion for
another upcoming email of mine :) and here concentrate on this one proposal
for the given problem of garbage po/mo files).
Cheers
Friedrich
