On Sun, 05 Oct 2014 10:36:31 -0400, Sven Van Caekenberghe <s...@stfx.eu>
wrote:
How come you got WideStrings ?
What does the input look like, can you give a partial example ?
I'm guessing I got WideStrings because the file is indeed in UTF-8, with
lots of characters outside the lower 128 code points. A sample couple of
lines might look like
光田康典,The National,Sarah McLachlan,周杰倫,Indochine,Rise Against,City
and Colour,Cæcilie Norby,El Cumbanchero,Death Letter
The Beatles,The Who,Barenaked Ladies,The Doors,Bob Dylan
These are two play lists, one per line; each comma-delimited element is a
band on that play list.
The full line for reading and tokenization is just "pathToFile
asFileReference contents lines collect: [ :line | (',' split: line)
collect: [ :ea | ea trimmed ]]." Based on the profile indicating that a
lot of time is lost on things like WideString>>copyFrom:to:, I wasn't
optimistic about trying to stream the contents instead of just calling
"contents lines", but I admit I didn't try.
--Benjamin
