On Fri, Nov 27, 2015 at 8:54 PM, Tatsuo Ishii <is...@postgresql.org> wrote: > I explain why the manual editing is necessary. > > One of the most famous problems with Unicode is "wave dash" > (U+301C). According the Unicode consortium's Unicode/SJIS map, it > corresponds to 0x8160 of Shift_JIS. Unfortunately this was a mistake > in Unicode (the glyph of Shift_JIS and Unicode is slightly different - > looks like to be rotated in 90 degrees of wave dash in vertical > scripting. Probably they did not understand the Japanese vertical > writing at that time). So later on the Unicode consortium decided to > add another "wave dash" as U+FF5E which has a correct glyph of "wave > dash". However since Unicode already decided that U+301C corresponds > to 0x8160 of Shift_JIS, there's no Shift_JIS code corresponding to > U+FF5E. Unlike Unicode's definition, Microsoft defines that 0x8160 > (wave dash) corresponds to U+FF5E. This is widely used in Japan. So I > decided to hire this for "wave dash". i.e. > > 0x8160 -> U+FF5E (sjis_to_utf8.map) > > U+301C -> 0x8160 (utf_to_sjis.map) > U+FF5E -> 0x8160 (utf_to_sjis.map) > > Another problem is vendor extension. > > There are several standards for SJIS and EUC_JP in Japan. There is a > standard "Shift_JIS" defined by Japanese Government (probably the > Unicode consortium's map can be based on this, but I need to > verify). However several major vendors include IBM, NEC added their > own additional characters to Shift_JIS and they are widely used in > Japan. Unfortunately they are not compatible. So as a compromise I and > other developers decided to "merge" NEC and IBM extension part and > added to Shift_JIS. Same thing can be said to EUC_JP. > > In short, there are number of reasons we cannot simply import the > consortium's mapping regarding SJIS (and EUC_JP).
I haven't seen a response to this point, but it seems important. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
