So open file for binary read, then put textDecode(it, “UTF-8”) into tText.
Bob S > On Oct 29, 2024, at 8:31 AM, Bob Sneidar <bobsnei...@iotecdigital.com> wrote: > > I suppose you could also use textDecode(<string>, “UTF-8”) to convert the > text to that format. > > Bob S > > >> On Oct 29, 2024, at 8:17 AM, Bob Sneidar via use-livecode >> <use-livecode@lists.runrev.com> wrote: >> >> There is a Wikipedia article on this. Turns out it is not straightforward. >> There can be a Byte Order Mark that the file begins with but not all vendors >> use it. And I do not think you can make the determination simply by >> examining the contents of the file. >> >> Byte-order mark[edit] >> If the Unicode byte-order mark U+FEFF is at the start of a UTF-8 file, the >> first three bytes will be 0xEF, 0xBB, 0xBF. >> The Unicode Standard neither requires nor recommends the use of the BOM for >> UTF-8, but warns that it may be encountered at the start of a file >> trans-coded from another encoding.[23] While ASCII text encoded using UTF-8 >> is backward compatible with ASCII, this is not true when Unicode Standard >> recommendations are ignored and a BOM is added. A BOM can confuse software >> that isn't prepared for it but can otherwise accept UTF-8, e.g. programming >> languages that permit non-ASCII bytes in string literals but not at the >> start of the file. Nevertheless, there was and still is software that always >> inserts a BOM when writing UTF-8, and refuses to correctly interpret UTF-8 >> unless the first character is a BOM (or the file only contains ASCII).[24] >> >> https://en.wikipedia.org/wiki/UTF-8# >> >> Bob S >> >> >>> On Oct 29, 2024, at 1:53 AM, jbv via use-livecode >>> <use-livecode@lists.runrev.com> wrote: >>> >>> Hi list, >>> >>> How to determine if a text file is UTF8 or just plain ASCII ? >>> In other words, how to know if one should use >>> open file myfile.txt for UTF8 read >>> or >>> open file myfile.txt for read >>> >>> Thank you. >>> jbv >>> >>> _______________________________________________ >>> use-livecode mailing list >>> use-livecode@lists.runrev.com >>> Please visit this url to subscribe, unsubscribe and manage your >>> subscription preferences: >>> http://lists.runrev.com/mailman/listinfo/use-livecode >> >> >> _______________________________________________ >> use-livecode mailing list >> use-livecode@lists.runrev.com >> Please visit this url to subscribe, unsubscribe and manage your subscription >> preferences: >> http://lists.runrev.com/mailman/listinfo/use-livecode > _______________________________________________ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
