Andy Wingo <wi...@pobox.com> writes:
> On Tue 29 Jan 2013 20:22, Neil Jerram <n...@ossau.homelinux.net> writes:
>
>> (define (read-csv file-name)
>> (let ((s (utf16->string (get-bytevector-all (open-input-file file-name))
>> 'little)))
>>
>> ;; Discard possible byte order mark.
>> (if (and (>= (string-length s) 1)
>> (char=? (string-ref s 0) #\xfeff))
>> (set! s (substring s 1)))
>>
>> ...))
>
> FWIW the procedure I had was:
>
> (define (consume-byte-order-mark port)
> (let ((enc (or (port-encoding port) "ISO-8859-1")))
> (set-port-encoding! port "ISO-8859-1")
> (case (peek-char port)
> ((#\xEF)
> (read-char port)
> (case (peek-char port)
> ((#\xBB)
> (read-char port)
> (case (peek-char port)
> ((#\xBF)
> (read-char port)
> (set-port-encoding! port "UTF-8"))
> (else
> (unread-char #\xBB port)
> (unread-char #\xEF port)
> (set-port-encoding! port enc))))
> (else
> (unread-char #\xEF port)
> (set-port-encoding! port enc))))
> ((#\xFE)
> (read-char port)
> (case (peek-char port)
> ((#\xFF)
> (read-char port)
> (set-port-encoding! port "UTF-16BE"))
> (else
> (unread-char #\xFE port)
> (set-port-encoding! port enc))))
> ((#\xFF)
> (read-char port)
> (case (peek-char port)
> ((#\xFE)
> (read-char port)
> (set-port-encoding! port "UTF-16LE"))
> (else
> (unread-char #\xFF port)
> (set-port-encoding! port enc))))
> (else
> (set-port-encoding! port enc)))))
>
> The encoding dance is because there is no unread-u8 from Scheme, only
> unread-char.
I can see why you'd want to do something about that!
Regards,
Neil
