Re: [fpc-pascal] Read lines into UnicodeString variable from UCS2 (UTF-16) encoded text file

2019-09-05 Thread LacaK
From user POV we have this situation: - on one side there is input text file encoded UTF-16 (either LE or BE) - on other side there is FPC, where RTL procedures like AssignFile, SetTextCodePage, Reset, Read(Ln), Write(Ln) are available. My original intention was simply use call to existing proc

Re: [fpc-pascal] Read lines into UnicodeString variable from UCS2 (UTF-16) encoded text file

2019-09-05 Thread Tomas Hajny
On 2019-09-05 13:04, Joost van der Sluis wrote: Op 05-09-19 om 12:06 schreef Tomas Hajny: On 2019-09-05 09:00, LacaK wrote: Is there consensus/demand on such solution and any patch in this direction will be accepted? I'm not aware of potential discussion about this so far, thus I cannot talk

Re: [fpc-pascal] Read lines into UnicodeString variable from UCS2 (UTF-16) encoded text file

2019-09-05 Thread Joost van der Sluis
Op 05-09-19 om 12:06 schreef Tomas Hajny: On 2019-09-05 09:00, LacaK wrote: Is there consensus/demand on such solution and any patch in this direction will be accepted? I'm not aware of potential discussion about this so far, thus I cannot talk about any existing consensus (let's hear others)

Re: [fpc-pascal] Read lines into UnicodeString variable from UCS2 (UTF-16) encoded text file

2019-09-05 Thread Tony Whyman
Text files and the console should look the same from the programmer's point of view. The general principle is that you should be able to redirect stdin to a file and for there to be no difference as far the program is concerned when reading a file as opposed to reading console input. If there

Re: [fpc-pascal] Read lines into UnicodeString variable from UCS2 (UTF-16) encoded text file

2019-09-05 Thread Tomas Hajny
On 2019-09-05 09:00, LacaK wrote: . . Is there consensus/demand on such solution and any patch in this direction will be accepted? I'm not aware of potential discussion about this so far, thus I cannot talk about any existing consensus (let's hear others), but I believe that such a consensu

Re: [fpc-pascal] Read lines into UnicodeString variable from UCS2 (UTF-16) encoded text file

2019-09-05 Thread Tomas Hajny
On 2019-09-05 10:24, Tony Whyman wrote: A few points: 1. IMHO: This is currently a Windows problem where the console buffer is UCS2. Linux (and probably all other cases its UTF8 - to be verified). . . No, the subject refers to text files, not to console. Obviously, console output has its ca

Re: [fpc-pascal] Read lines into UnicodeString variable from UCS2 (UTF-16) encoded text file

2019-09-05 Thread Tony Whyman
Apologies: when I typed "FTP" below I meant "FPC" :( I'm currently drowning in acronym soup. On 05/09/2019 09:24, Tony Whyman wrote: A few points: 1. IMHO: This is currently a Windows problem where the console buffer is UCS2. Linux (and probably all other cases its UTF8 - to be verified).

Re: [fpc-pascal] Read lines into UnicodeString variable from UCS2 (UTF-16) encoded text file

2019-09-05 Thread Tony Whyman
A few points: 1. IMHO: This is currently a Windows problem where the console buffer is UCS2. Linux (and probably all other cases its UTF8 - to be verified). 2. The following Microsoft blog post is interesting background on where MS are going with this: https://devblogs.microsoft.com/command

Re: [fpc-pascal] Read lines into UnicodeString variable from UCS2 (UTF-16) encoded text file

2019-09-05 Thread LacaK
You may be able to improve on this using system.BlockRead. Probably yes, but then I must read in local buffer and examine buffer for CR/LF. And return from my function UCS2ReadLn() only portion of string up to CR/LF and rest of string return on next call to my function. (so I must keep unp