> the real problem isn't in viewing them however, but comes when you > start searching for them: it's easy to search for ë (e-umlaut) for > example, but what if it's described as e+"U+0308 COMBINING DIAERESIS"? > the answer is the UTS#18 Regular Expressions technical standard which > probably contributes at least half of the slowness of gnu grep > discussed in another thread. http://www.unicode.org/reports/tr18/
iirc, gnu grep calls malloc for each character of utf-8 input. awsome. at a minimum, it would be good to write to add support to tcs to translate to cannonical form utf. this would make the searching problem much easier. the unicode proposal says that matches depend on (re, locale, input). not just (re, input). i would think that is not acceptable. - erik
