Enrico Forestieri wrote:
> On Sun, Sep 03, 2006 at 09:47:50PM +0200, Georg Baum wrote:
>
>> I believe that the attached patch should make the unicode conversions
>> work on little and big endian machines, and removes the uncertainty
>> whether UCS-4 is LE or BE.
>
> Yep. With this patch I can load old LyX files when using the qt3
> frontend. Without it I get hollow squares on screen, instead.
> It makes no difference with qt4.
That is the byte order fix for ucs2 conversion.
>> Using UCS4-INTERNAL and memmove() the result as suggested by Angus would
>> be even better, but it seems that it is not available on windows.
>
> I am not sure I understand what you mean. This is what I get on cygwin:
>
> $ iconv -l | grep UCS
> ISO-10646-UCS-2 UCS-2 CSUNICODE
> UCS-2BE UNICODE-1-1 UNICODEBIG CSUNICODE11
> UCS-2LE UNICODELITTLE
> ISO-10646-UCS-4 UCS-4 CSUCS4
> UCS-4BE
> UCS-4LE
> UCS-2-INTERNAL
> UCS-2-SWAPPED
> UCS-4-INTERNAL
> UCS-4-SWAPPED
I can confirm Lars that UCS-4-INTERNAL is not present on my Linux box
either. I guess that UCS-4-INTERNAL is only available with GNU libiconv and
not with the builtin iconv in libc.
I am going to put the patch in, since it is clearly an improvement. I know
that this is not the final solution, and that the conversion functions need
to be optimized, but I am not going to do that now. Whether the
optimization will result in not using iconv anymore, or still using iconv
but without copying the data around has to be decided (and I am currently
unsure what I prefer). For now I simply want something that works.
Georg
Log:
Fix byte order problems
* src/support/unicode.C
(bytes_to_ucs4): make it work on big endian machines
(bytes_to_ucs2): make it work on little endian machines
(ucs2_to_ucs4): ditto
(utf8_to_ucs4): Invoke iconv with explicit BE suffix
(ucs2_to_ucs4): ditto
(ucs4_to_ucs2): ditto
(ucs4_to_utf8): ditto
* configure.ac: Check for byte order
* development/scons/SConstruct: Ditto
