Jason Harrop created FOP-3341:
---------------------------------
Summary: A font whose GSUB or GPOS has no DFLT script table
silently gets no substitution and no kerning under a default script, with no
warning
Key: FOP-3341
URL: https://issues.apache.org/jira/browse/FOP-3341
Project: FOP
Issue Type: Bug
Components: font/opentype
Affects Versions: 2.11
Reporter: Jason Harrop
GlyphTable.matchLookups falls back to the default script when a lookup match
comes up empty:
{noformat}
if (lm.isEmpty() && !OTFScript.isDefault(script) &&
!OTFScript.isWildCard(script)) {
return matchLookups(OTFScript.DEFAULT, OTFLanguage.DEFAULT, feature);
}
{noformat}
The fallback targets DFLT. So for a font that has no DFLT entry in its script
list, a script of DFLT finds nothing, and the fallback looks for DFLT again and
finds nothing again. Nothing is applied and nothing is reported. The font is
loaded, its tables are read, performsSubstitution and performsPositioning both
return true, and every lookup silently misses.
Fonts without a DFLT script table are not unusual. Carlito, the
metric-compatible Calibri substitute packaged by most Linux distributions,
declares cyrl, grek and latn and no DFLT. DejaVu declares DFLT alongside them,
which is why the two behave differently on the same code path and why this is
easy to mistake for a font-file or a loader problem.
Measured, on the same font objects with only the script argument changing:
{noformat}
script Carlito 1.104 DejaVuLGCSerif
latn 'fi' -> U+FB01 'fi' -> U+FB01
* 'fi' -> U+FB01 'fi' -> U+FB01
DFLT 'fi' -> 'f','i' 'fi' -> U+FB01
zyyy 'fi' -> 'f','i' 'fi' -> U+FB01
auto 'fi' -> 'f','i' 'fi' -> U+FB01
{noformat}
The language argument opens the same hole. A language with no langsys entry
under the script makes the (script, language) match come up empty, and the
fallback then goes to (DFLT, dflt), skipping the script's own default langsys.
Measured 2026-09-27 through the command line, one fo:block, only the language
attribute changing, read back with pdftotext:
{noformat}
attributes Carlito DejaVu Sans
script=latn, no language ffi ffl fi fl ft ti ligated ffi ffl fi fl
ligated
script=latn language="en" nothing ligated nothing ligated
{noformat}
And the language never matches in the first place. The FO language property
carries an ISO 639 code ("en", "tr"); the font's language systems are OpenType
tags ("ENG", "TRK"); FOP passes the code through untranslated. So a language
the font does distinguish is treated like one it does not: DejaVuLGCSerif's TRK
system omits liga, and performSubstitution("fi", "latn", "tr") ligates while
("fi", "latn", "TRK") does not.
Neither font has an ENG langsys under latn. DejaVu Sans does have DFLT, but its
DFLT langsys carries case, ccmp, dlig and kern and no liga, so the fallback
lands on a script table that lacks the feature. DejaVu Serif's DFLT langsys
does carry liga, and we measured that it ligates under the same attributes. So
whether a run ligates today is font-accidental: it depends on which features
the font's author happened to put in DFLT, not on what the font offers for the
script.
This is not a corner case for a consumer. docx4j writes language="en"
country="US" on every block and inline it produces, from w:lang and the
document defaults, and it cannot drop the attribute because FOP selects
hyphenation patterns by it. The first point of FOP-2092 (Thai shapes only under
language="dflt") is this language half. The fix belongs in the fallback order,
as OpenType layout engines do it: (script, language), then (script, dflt), then
(DFLT, dflt). GlyphTable.matchLookups goes straight from the first to the third.
It is not only substitution. The same script matching governs GPOS, so kerning
disappears too, and that is the wider harm because it reaches every letter pair
rather than a handful of ligature sequences:
{noformat}
Carlito legacy kern table: absent, 0 entries
GPOS 'AV' latn kerned * kerned DFLT null
DejaVu legacy kern table: present, 101 entries
GPOS 'AV' latn kerned * kerned DFLT kerned
{noformat}
Carlito has no legacy kern table at all, so under a default script it gets no
kerning from either mechanism. A consumer that sets kerning="true" on such a
font gets nothing and is told nothing.
Note that the wildcard script "*" works, because the wildcard matches the
font's real script tables. GlyphMapping maps "auto" and "zyyy" to "*", so the
common FO path is safe; it is a caller that passes DFLT, or "auto"/"zyyy"
unnormalised, that loses everything.
h3. Why it matters
The failure is silent and invisible in rendering: the text is drawn, just
unshaped and unkerned. Nobody notices until they compare advance widths against
another renderer. It also produces exactly the misleading symptom this was
first mistaken for: "FOP reads this font's tables but not that one's".
h3. Proposed fix
Done on the fork, branch CR-003-script-fallback (GlyphTable, OTFLanguage,
OTFScript; tests GlyphTableFallbackTestCase on synthetic tables shaped like
Carlito and DejaVu Sans, and LanguageSystemTestCase on the DejaVuLGCSerif in
the test tree, whose TRK system omits liga):
# Map the language to its OpenType tag first (a table of ISO 639 codes against
the tags OTFLanguage already names: "tr" to TRK, "en-US" to ENG, "zh-TW" to
ZHT; a tag is accepted as given; unknown codes match nothing and take the next
step), then fall back in the order OpenType layout engines use: (script,
language), then (script, dflt), then (DFLT, dflt). Each step is taken only when
the one before found nothing.
# OTFScript.isWildCard compares with the wildcard rather than DEFAULT (harmless
today, corrected alongside).
Measured after the change, same command line: Carlito under language="en"
ligates (ffi fi fl ft ti, 26 glyphs to 21) and kerns (AVATAR 43.19 pt, was
46.62); DejaVu Sans under "en" ligates; DejaVuLGCSerif
performSubstitution("fi", "latn", "tr") no longer ligates, which is what its
TRK system says; Noto Sans Arabic under "ar" unchanged.
This changes shaping and kerning for existing documents wherever a language was
set and the font had no system for it, so it wants the release note. A safer
first step, if preferred: an event when a font with GSUB or GPOS present yields
no lookups for the requested script, turning the silent miss into a diagnosable
one.
Note that the kerning half is also gated by the font configuration's kerning
flag, which did not reach GPOS at all (a separate report, a companion issue
filed alongside this one), and by a reader defect that drops a script's default
language system when a named language shares its table (a companion issue filed
alongside this one): DejaVu Sans needs all three before it kerns under "en".
h3. Provenance
Found while implementing the docx4j FO renderer's gsub-features hook, tracing
why one font shaped and another did not through the same pipeline. All the
numbers above are from loading each font directly with FontLoader and calling
performSubstitution and performPositioning, so no renderer or consumer pipeline
is involved.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
