More observations about wide strings and Guile.
First, here are the abridged call trees for low-level reading and
writing.
read <-+- scm_getc <-+- [the parser] <--- scm_read <--- scm_primitive_load
| |
| +- scm_read_char
|
|
+- scm_c_read
|
+- read_without_guile
write <-+- scm_lfwrite <-+- scm_display
| |
| +- scm_putc <-+- scm_write_char
| |
| +- scm_newline
|
+- scm_flush
1. To move to a Unicode-enabled guile, text information needs to be
converted to an internal representation when read and converted
back to the locale when written. Most reading and writing for
ports passes through scm_getc (input) and scm_lfwrite (output).
Conversion between locale strings and internal strings should
happen there.
2. If string conversion occurs in scm_getc, then the scm_read reader
will be receiving and parsing source code that has passed through
the conversion routines. This is initially not a problem since
scheme code is largely ASCII, and Guile will start up in the C
locale.
But, if a source code file is not ASCII, the reader needs to be
able to ascertain this before parsing the code from the file. The
encoding of a source code file is a property of the file and not
the locale in which Guile is being run.
This implies that a source code file should have syntax to
indicate its own encoding, if it is not ASCII. Something akin to
the <?xml encoding="utf-8"?> line in HTML files.
3. The text encoding of a port needs to be associated with the port.
R6RS has the idea of transcoders for ports that require
conversion. It is daunting, but, having played some ideas for a
few weeks, it seems that at least a subset of the transcoder
functionality needs to be implemented for this to make any sense.
I sent in my copyright assignment last week, so you should have it
now.
Thanks,
Mike Gran
