On Tue, Dec 4, 2012 at 6:15 AM, Makoto Kato <m_k...@ga2.so-net.ne.jp> wrote: > A year ago, I investigated to replace uconv with ICU. > > Many (most is IBM code page) converters in ICU isn't compatible with uconv, > and ICU isn't have all character set converters that we already use in > uconv. > > If using ICU for uconv, we can remove a few table code into uconv, but not > all.
The main problem uconv has is that its API design is wrong: It doesn’t have a way to signal the end of the stream, so uconv fails to emit a REPLACEMENT CHARACTER as the last character of the output if the input ends with an incomplete multibyte sequence. (FWIW, iconv has the same API design flaw, AFAICT.) ICU gets the design of conversion function right, but otherwise ICU’s converter API surface is a complete overkill. But most importantly, ICU implements the de jure Unicode specs, IANA labels and a bunch of encodings that aren’t needed for the Web instead of implementing what’s Web-compatible and only what’s Web-compatible. Gecko needs converters and label alias resolution that conforms to the Encoding Standard (http://encoding.spec.whatwg.org/). We already have code for Encoding Standard-compliant label handling in the tree and uconv is being patched to be more and more Encoding Standard-compliant where it isn’t already. I think for encoders, decoders and encoding label handling, the end result should be compliance with the Encoding Standard and no dead weight in the release binaries arising from compiled-but-unused Encoding Standard-incompliant ICU converter or label handling code/data. (My understanding is that Chrome has a patched fork of ICU converters in order to have characteristics that are closer to the Encoding Standard than vanilla ICU. Hence, we shouldn’t expect to be able to use unpatched system ICU for converters and label handling and get right behavior. At least not without upstreaming Encoding Standard-compliant code into ICU and waiting for it to propagate to system libraries first.) -- Henri Sivonen hsivo...@iki.fi http://hsivonen.iki.fi/ _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
