[fpc-pascal] FPC Unicode / CodePage aware strings
Hi, "official" Unicode support in FPC has been discussed for so long (years), that I came up with my own solution for fpGUI since 2007 (LCL did the similar) and it still works 100% and well tested. But now I want to learn what the FPC team finally came up with in FPC 3.0. Is there some white paper, blog post, awesome MvC article that explains it all. And how it compares to Delphi's Unicode support (if this is important - though I don't use Delphi at all any more). For example, I looked at a few recent messages here and in the Lazarus mailing list. I also looked at a length discussion in the Lazarus Forum which had some example code. I tried out a few things with various String CodePage valuse and writeln() statements using a UTF-8 encoded .pas unit, and I got horrible results on my FreeBSD 10.1 system. The Unicode (UTF-8) output to console that I expected to work simply didn't. I need to figure out why. There seems to be a million and one tweaks (options) one can now apply to the FPC compiler, the RTL, different modes can be set in each unit etc. It is all very confusing. So basically what I'm asking, is there a Unicode (or is that the wrong term) chapter or something in the FPC documentation that describes all this in easy to understand English. The do's and don't, what works and what doesn't in FPC 3.0, is the RTL fully Unicode enabled now etc. Regards, - Graeme - -- fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://fpgui.sourceforge.net/ My public PGP key: http://tinyurl.com/graeme-pgp ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] FPC Unicode / CodePage aware strings
Graeme Geldenhuys wrote on Thu, 10 Dec 2015: Is there some white paper, blog post, awesome MvC article that explains it all. And how it compares to Delphi's Unicode support (if this is important - though I don't use Delphi at all any more). http://wiki.freepascal.org/FPC_New_Features_3.0#Support_for_codepage-aware_strings Jonas ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
[fpc-pascal] changes in TThread.OnTerminate in fcp-3.0?
Hello, in a couple of programs I'm using this idiom: FMyThread:=TMyTread.Create; FMyThread.FreeOnTerminate:=false; FMyThread.OnTerminate:=@MyThreadTerminate procedure TMyForm.MyThreadTerminate(Sender:TObject); begin //do something with FMyThread then FreeAndNil(FMyThread); end; With fpc 2.6.4 it worked flawlessly, with fpc 3.0 sometimes it causes a sigsev somewhere inside CheckSynchronize. I could find nothing here http://wiki.freepascal.org/User_Changes_3.0 related to this. I realize that's an implementation detail so probably what I did was wrong (was it?). What are the possible solutions (apart from omitting the call to FreeAndNil and setting FreeOnTerminate to true)? Bye -- Luca ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] changes in TThread.OnTerminate in fcp-3.0?
Am 10.12.2015 11:05 schrieb "Luca Olivetti" : > > Hello, > > in a couple of programs I'm using this idiom: > > > FMyThread:=TMyTread.Create; > FMyThread.FreeOnTerminate:=false; > FMyThread.OnTerminate:=@MyThreadTerminate > > procedure TMyForm.MyThreadTerminate(Sender:TObject); > begin > //do something with FMyThread then > FreeAndNil(FMyThread); > end; > > > With fpc 2.6.4 it worked flawlessly, with fpc 3.0 sometimes it causes a sigsev somewhere inside CheckSynchronize. > > I could find nothing here > http://wiki.freepascal.org/User_Changes_3.0 > related to this. > I realize that's an implementation detail so probably what I did was wrong (was it?). I'd need to check what the RTL does after OnTerminate is called, but it could be that this worked by chance... > What are the possible solutions (apart from omitting the call to FreeAndNil and setting FreeOnTerminate to true)? That would be the safest solution (you can set FMyThread to Nil however inside that handler). Regards, Sven ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] FPC Unicode / CodePage aware strings
On 2015-12-10 09:50, Jonas Maebe wrote:
>
> Graeme Geldenhuys wrote on Thu, 10 Dec 2015:
>
>> Is there some white paper, blog post, awesome MvC article that explains
>> it all. And how it compares to Delphi's Unicode support (if this is
>> important - though I don't use Delphi at all any more).
>
> http://wiki.freepascal.org/FPC_New_Features_3.0#Support_for_codepage-aware_strings
Thanks, but here's the problem. The wiki says FPC 3.0 is now Delphi 2009
string compatible. So I starting reading the "Delphi_and_Unicode.pdf"
white paper. I tried the first example shown on page 7 (code below):
var
ch1: Char;
ch2: AnsiChar;
begin
ch1 := 'ù'; // error
Memo1.Lines.Add ('WideChar');
Memo1.Lines.Add ('UpCase ù: ' + UpCase(ch1));
Memo1.Lines.Add ('ToUpper ù: ' + ToUpper (ch1)); // warning
ch2 := 'ù'; // error
Memo1.Lines.Add ('AnsiChar');
Memo1.Lines.Add ('UpCase ù: ' + UpCase(ch2));
Memo1.Lines.Add ('ToUpper ù: ' + ToUpper(ch2)); // warning
end;
Tried to compile that with FPC (yes, I included the new "Character" unit
in uses clause) and I get a couple of compiler errors. I tried first
with mode OBJFPC and then with mode DELPHI. Same compiler errors.
Compile Project, Target: project1: Exit code 1, Errors: 3, Warnings: 2
project1.lpr(35,10) Error: Incompatible types: got "Constant String"
expected "Char"
project1.lpr(38,50) Warning: Implicit string type conversion with
potential data loss from "WideString" to "AnsiString"
project1.lpr(39,10) Error: Incompatible types: got "Constant String"
expected "Char"
project1.lpr(42,50) Warning: Implicit string type conversion with
potential data loss from "WideString" to "AnsiString"
project1.lpr(54,13) Error: Variable identifier expected
So already it seems information is missing for FPC usage, or FPC clearly
isn't as "delphi compatible" as the wiki makes out to be. Or something
else went astray. This is exactly what I'm talking about. It clearly
doesn't "just work". I'll read the FPC_Unicode_support wiki from top to
bottom. Maybe that sheds some light.
My test project was created using Lazarus (Project -> New Application).
I tried the same using a fpGUI toolkit project too.
Regards,
- Graeme -
--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/
My public PGP key: http://tinyurl.com/graeme-pgp
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] changes in TThread.OnTerminate in fcp-3.0?
On Thu, Dec 10, 2015 at 10:26:29AM +0100, Luca Olivetti wrote: > Hello, > > in a couple of programs I'm using this idiom: > > > FMyThread:=TMyTread.Create; > FMyThread.FreeOnTerminate:=false; > FMyThread.OnTerminate:=@MyThreadTerminate > > procedure TMyForm.MyThreadTerminate(Sender:TObject); > begin > //do something with FMyThread then > FreeAndNil(FMyThread); > end; This is why FreeOnTerminate exists. Set that to True and set FMyThread to nil here, and the thread will free itself after this function returns. Henry ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] FPC Unicode / CodePage aware strings
2015-12-10 11:18 GMT+01:00 Graeme Geldenhuys : > On 2015-12-10 09:50, Jonas Maebe wrote: > > > > Graeme Geldenhuys wrote on Thu, 10 Dec 2015: > > Tried to compile that with FPC (yes, I included the new "Character" unit > in uses clause) and I get a couple of compiler errors. I tried first > with mode OBJFPC and then with mode DELPHI. Same compiler errors. > And mode DelphiUnicode? Vincent ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] FPC Unicode / CodePage aware strings
Graeme Geldenhuys wrote on Thu, 10 Dec 2015:
On 2015-12-10 09:50, Jonas Maebe wrote:
http://wiki.freepascal.org/FPC_New_Features_3.0#Support_for_codepage-aware_strings
Thanks, but here's the problem. The wiki says FPC 3.0 is now Delphi 2009
string compatible. So I starting reading the "Delphi_and_Unicode.pdf"
white paper. I tried the first example shown on page 7 (code below):
The language feature is implemented in a Delphi 2009-compatible way
(although some Delphi-specific behaviour is only activated in {$mode
delphiunicode}). Not all RTL (let alone Lazarus, like for TMemo)
support is ready yet. You even explicitly added an example at
http://wiki.freepascal.org/FPC_Unicode_support#RTL_todos
So already it seems information is missing for FPC usage,
It's all documented on the wiki page you mention below.
or FPC clearly
isn't as "delphi compatible" as the wiki makes out to be. Or something
else went astray. This is exactly what I'm talking about. It clearly
doesn't "just work". I'll read the FPC_Unicode_support wiki from top to
bottom. Maybe that sheds some light.
It is the "white paper [or] blog post" you were asking about.
Jonas
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] FPC Unicode / CodePage aware strings
On 2015-12-10 10:22, Vincent Snijders wrote: > And mode DelphiUnicode? Oh, yet another compiler mode. Okay, just tried, and got these warnings, but the project at least compiled and generated a binary. Compile Project, Target: project1: Success, Warnings: 5 project1.lpr(37,47) Warning: Implicit string type conversion with potential data loss from "UnicodeString" to "AnsiString" project1.lpr(38,50) Warning: Implicit string type conversion with potential data loss from "UnicodeString" to "AnsiString" project1.lpr(41,36) Warning: Implicit string type conversion from "Char" to "UnicodeString" project1.lpr(41,47) Warning: Implicit string type conversion with potential data loss from "UnicodeString" to "AnsiString" project1.lpr(42,50) Warning: Implicit string type conversion with potential data loss from "UnicodeString" to "AnsiString" Now I try and run the binary and get the following error. [t2]$ ./project1 This binary has no unicodestrings support compiled in. Recompile the application with a unicodestrings-manager in the program uses clause. An unhandled exception occurred at $0044354D: ENoWideStringSupport: SIGQUIT signal received. $0044354D Okay, so I add the "unicodestrings-manager" unit to the program unit's uses clause, as the above message suggests. Now I can't compile the project... project1.lpr(10,43) Fatal: Cannot find unicodestrings used by project1 of the Project Inspector. Yeah, my experience with FPC Unicode is going really well. ;-) Regards, - Graeme - -- fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://fpgui.sourceforge.net/ My public PGP key: http://tinyurl.com/graeme-pgp ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] FPC Unicode / CodePage aware strings
On 2015-12-10 10:29, Jonas Maebe wrote:
> The language feature is implemented in a Delphi 2009-compatible way
> (although some Delphi-specific behaviour is only activated in {$mode
> delphiunicode}).
Okay, so forget Delphi 2009 compatibility for the moment (I don't use
Delphi). So does this mean to use "official" FPC Unicode, I can't use
{$mode objfpc} any more?
> Not all RTL (let alone Lazarus, like for TMemo)
> support is ready yet.
No problem, I changed the example to a console program and used
WriteLn() instead.
Does WriteLn() work correctly with Unicode text? Will it detect my
console encoding used in mate-terminal or xterm under FreeBSD? If not,
what else must I do to test this Unicode support?
>> So already it seems information is missing for FPC usage,
>
> It's all documented on the wiki page you mention below.
As past experience has shown me, the wiki is a very unreliable source -
often things are way out of date, or "concepts" was documented but never
removed when something else was decided etc.
But I'll spend my lunch time reading the wiki page in question and see
if anything helps. But like I said, the experience thus far is anything
but ideal or straight forward.
>
> It is the "white paper [or] blog post" you were asking about.
Both. I can't follow the examples shown in the Delphi_and_Unicode.pdf,
because they simply don't compiler or don't run.
The wiki pages doesn't have a single compilable "test program" example
that can get you going.
Here is the Lazarus Forum discussion on FPC's new String support in 3.0.
I tried that example on my FreeBSD system too, and the console output
was total rubbish no matter what I tried.
http://forum.lazarus.freepascal.org/index.php/topic,30200.msg191799.html#msg191799
Regards,
- Graeme -
--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/
My public PGP key: http://tinyurl.com/graeme-pgp
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] FPC Unicode / CodePage aware strings
On 2015-12-10 10:35, Graeme Geldenhuys wrote:
> Now I try and run the binary and get the following error.
>
> [t2]$ ./project1
> This binary has no unicodestrings support compiled in.
> Recompile the application with a unicodestrings-manager in the program
> uses clause.
> An unhandled exception occurred at $0044354D:
> ENoWideStringSupport: SIGQUIT signal received.
> $0044354D
>
>
> Okay, so I add the "unicodestrings-manager" unit to the program unit's
> uses clause, as the above message suggests.
>
> Now I can't compile the project...
Okay, I changed my program to a console program. After much searching I
found that "unicodestrings-manager" is not actually a unit. I tried
"cwstrings" instead - the wiki page didn't mention this - and could
compile and run my program.
[t2]$ ./project1
WideChar
UpCase ù: Ù
ToUpper ù: Ù
AnsiChar
UpCase ù: ?
ToUpper ù: ?
The output is not the same as the output from the
"Delphi_and_Unicode.pdf" example.
Source code:
The .pas unit is stored as a UTF-8 encoded file, and the program was run
in mate-terminal and xterm, showing the same output as above.
===
program project1;
{$mode delphiunicode}{$H+}
uses
cwstring,
Classes, SysUtils, Character;
var
ch1: Char;
ch2: AnsiChar;
begin
ch1 := 'ù';
Writeln('WideChar');
Writeln('UpCase ù: ' + UpCase(ch1));
Writeln('ToUpper ù: ' + ToUpper (ch1));
ch2 := 'ù';
Writeln('AnsiChar');
Writeln('UpCase ù: ' + UpCase(ch2));
Writeln('ToUpper ù: ' + ToUpper(ch2));
end.
===
Regards,
- Graeme -
--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/
My public PGP key: http://tinyurl.com/graeme-pgp
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] FPC Unicode / CodePage aware strings
On Thursday 10 December 2015 12:01:49 Graeme Geldenhuys wrote:
>
> ===
> program project1;
>
> {$mode delphiunicode}{$H+}
>
Try {$codepage utf8}.
Martin
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] changes in TThread.OnTerminate in fcp-3.0?
El 10/12/15 a les 11:13, Sven Barth ha escrit: Am 10.12.2015 11:05 schrieb "Luca Olivetti" mailto:l...@ventoso.org>>: > > Hello, > > in a couple of programs I'm using this idiom: > > > FMyThread:=TMyTread.Create; > FMyThread.FreeOnTerminate:=false; > FMyThread.OnTerminate:=@MyThreadTerminate > > procedure TMyForm.MyThreadTerminate(Sender:TObject); > begin > //do something with FMyThread then > FreeAndNil(FMyThread); > end; > > > With fpc 2.6.4 it worked flawlessly, with fpc 3.0 sometimes it causes a sigsev somewhere inside CheckSynchronize. > > I could find nothing here > http://wiki.freepascal.org/User_Changes_3.0 > related to this. > I realize that's an implementation detail so probably what I did was wrong (was it?). I'd need to check what the RTL does after OnTerminate is called, but it could be that this worked by chance... Yes and no: OnTerminate is called via Synchronize. In 2.6.4 there could be only one outstanding synchronized method and CheckSynchronize wouldn't access the thread after calling it, so freeing the thread inside OnTerminate was no problem. Now it's been changed to use a queue of waiting methods, and CheckSynchronize, after calling the method, calls RtlEnventSetEvent(tmpentry^.SyncEvent) where SyncEvent is a member of the thread, and if the thread has already been freed that's going to bomb. It's obvious that you cannot free the thread from inside a "normal" Synchronize call, it's no so obvious you cannot do it in the OnTerminate method, so maybe a note could be added in http://wiki.freepascal.org/User_Changes_3.0 The new implementation is clearly betteronce you know *not* to free the thread inside OnTerminate. Bye -- Luca ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] FPC Unicode / CodePage aware strings
Graeme Geldenhuys wrote on Thu, 10 Dec 2015:
On 2015-12-10 10:29, Jonas Maebe wrote:
The language feature is implemented in a Delphi 2009-compatible way
(although some Delphi-specific behaviour is only activated in {$mode
delphiunicode}).
Okay, so forget Delphi 2009 compatibility for the moment (I don't use
Delphi). So does this mean to use "official" FPC Unicode, I can't use
{$mode objfpc} any more?
There is no such thing as "official FPC Unicode". We added support for
codepage-aware ansistrings. We added unicodestring overloads for a
number of RTL routines, and made sure that those same routines also
correctly handle the ansistring codepage information. This
functionality is always available everywhere, and behaves the same
everywhere.
We also added the {$mode delphiunicode} syntax mode, which changes
some type definitions ((p)char = (p)widechar, string in {$h+} =
unicodestring), and the default source file code page in a Delphi 2009
and later compatible way. It does not modify the above functionlaty,
nor is it required.
No problem, I changed the example to a console program and used
WriteLn() instead.
The test program you posted later does not work completely because it
uses routines that have not yet been adapted (UpCase and ToUpper are
not mentioned in
http://wiki.freepascal.org/FPC_Unicode_support#RTL_changes ).
Does WriteLn() work correctly with Unicode text?
Yes.
Will it detect my
console encoding used in mate-terminal or xterm under FreeBSD?
It will use whatever code page has been (correctly) configured for
your terminal, yes.
As past experience has shown me, the wiki is a very unreliable source -
Wiki articles that are linked from official release notes are by
definition supported and guaranteed to be reliable by us.
But I'll spend my lunch time reading the wiki page in question and see
if anything helps. But like I said, the experience thus far is anything
but ideal or straight forward.
You seem to be looking for a tutorial. We have written no such
document. There is only a technical description of what has been added
and how it behaves.
Jonas
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] FPC Unicode / CodePage aware strings
Martin Schreiber wrote on Thu, 10 Dec 2015:
On Thursday 10 December 2015 12:01:49 Graeme Geldenhuys wrote:
===
program project1;
{$mode delphiunicode}{$H+}
Try {$codepage utf8}.
That won't change anything, because {$mode delphiunicode} implies "use
the default system code page to interpret the source code", and that's
UTF-8 on his system.
Jonas
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] FPC Unicode / CodePage aware strings
On Thu, 10 Dec 2015 12:17:28 +0100
Jonas Maebe wrote:
>[...]
> > Try {$codepage utf8}.
>
> That won't change anything, because {$mode delphiunicode} implies "use
> the default system code page to interpret the source code", and that's
> UTF-8 on his system.
Does that mean a {$mode delphiunicode} resets a -Fc parameter?
Mattias
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] FPC Unicode / CodePage aware strings
On 2015-12-10 11:10, Martin Schreiber wrote:
> Try {$codepage utf8}.
It made no difference to the console output.
Regards,
- Graeme -
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] FPC Unicode / CodePage aware strings
On 2015-12-10 11:16, Jonas Maebe wrote: > There is no such thing as "official FPC Unicode". We added support for > codepage-aware ansistrings. We added unicodestring overloads for a Does the type UnicodeString always mean UTF-16 encoded text (like it's Delphi counterpart)? If so, that unfortunate. Can the encoding of UnicodeString be changed to say UTF-8? And yes, I know there is a UTF8String type too, but I'm specifically referring to UnicodeString (as in the Unicode standard supporting multiple encodings), as many RTL functions seem to use that specific type. > The test program you posted later does not work completely because it > uses routines that have not yet been adapted (UpCase and ToUpper are OK, so FPC unicode is still far from usable, as some RTL function will work, and others won't. :-/ With that conclusion, I'll then keep fpGUI's Unicode support exactly as it currently is. > It will use whatever code page has been (correctly) configured for > your terminal, yes. Excellent, thanks. >> As past experience has shown me, the wiki is a very unreliable source - > > Wiki articles that are linked from official release notes are by > definition supported and guaranteed to be reliable by us. Very good to know. > You seem to be looking for a tutorial. We have written no such > document. Umm, with all the hype around FPC + CodePage-aware AnsiStrings, I would have thought such a tutorial would be a given with the FPC 3.0 release. It sure would reduce a lot of questions and confusion. But as I mentioned above, maybe it's because the RTL is not really ready for the new FPC feature, so hence the sparse information out there. [just a thought] Regards, - Graeme - My public PGP key: http://tinyurl.com/graeme-pgp ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] FPC Unicode / CodePage aware strings
Mattias Gaertner wrote on Thu, 10 Dec 2015:
On Thu, 10 Dec 2015 12:17:28 +0100
Jonas Maebe wrote:
That won't change anything, because {$mode delphiunicode} implies "use
the default system code page to interpret the source code", and that's
UTF-8 on his system.
Does that mean a {$mode delphiunicode} resets a -Fc parameter?
Yes, similar to how a {$codepage XXX} directive does.
Jonas
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] FPC Unicode / CodePage aware strings
On Thu, 10 Dec 2015 12:02:31 + Graeme Geldenhuys wrote: > On 2015-12-10 11:16, Jonas Maebe wrote: > > There is no such thing as "official FPC Unicode". We added support for > > codepage-aware ansistrings. We added unicodestring overloads for a > > Does the type UnicodeString always mean UTF-16 encoded text (like it's > Delphi counterpart)? If so, that unfortunate. Yes. > Can the encoding of UnicodeString be changed to say UTF-8? No. > And yes, I > know there is a UTF8String type too, but I'm specifically referring to > UnicodeString (as in the Unicode standard supporting multiple > encodings), as many RTL functions seem to use that specific type. Many RTL functions support UTF8String, AnsiString too, as Jonas already pointed out. > > The test program you posted later does not work completely because it > > uses routines that have not yet been adapted (UpCase and ToUpper are > > OK, so FPC unicode is still far from usable, as some RTL function will > work, and others won't. :-/ With that conclusion, I'll then keep > fpGUI's Unicode support exactly as it currently is. If you have a working Unicode solution there is no reason to abandon it. The FPC team did a great job keeping compatibility (contrary to Delphi). Mattias ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] FPC Unicode / CodePage aware strings
Mattias Gaertner wrote on Thu, 10 Dec 2015: If you have a working Unicode solution there is no reason to abandon it. Exactly. There is absolutely no point in changing your code to unicodestring in reaction to these changes (nor in reaction to future changes an sich, if we can help it). There's not even any point to changing your code at all in many cases. And all functionality that has been added is indeed also available when using single byte strings, as explained on the FPC_Unicode_support page. The main reason so few RTL-level features are supported yet is exactly because it takes a lot of planning and energy to keep functionality as backward compatible as possible. If we had just abandoned backward compatibility and made everything UTF-16 only, we would probably have finished a long time ago. Jonas ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] FPC Unicode / CodePage aware strings
On 2015-12-10 12:18, Mattias Gaertner wrote: > If you have a working Unicode solution there is no reason to abandon it. > The FPC team did a great job keeping compatibility (contrary to Delphi). Indeed - from what I have tested and seen using FPC 3.0 with fpGUI. So kudos to the FPC team for that! Regards, - Graeme - My public PGP key: http://tinyurl.com/graeme-pgp ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] changes in TThread.OnTerminate in fcp-3.0?
I use a four method pattern with threads, while reusing the same thread class everywhere. 1) define a status method to update your user interface based on feedback from the thread 2) define a terminate method to do clean up when a thread finishes 3) define a work method where the thread does its execution 4) define a method which starts the thread feeding it all three methods above procedure TReplaceForm.ThreadStatus(Sender: TObject); begin Caption := FThread.Status; end; procedure TReplaceForm.ThreadTerminate(Sender: TObject); begin FThread := nil; end; procedure TReplaceForm.ThreadExecute(Thread: TSimpleThread); begin Thread.Status := 'Step 1'; Sleep(1000); Thread.Status := 'Step 2'; Sleep(1000); end; procedure TReplaceForm.ReplaceButtonClick(Sender: TObject); begin if FThread = nil then FThread := TSimpleThread.Create(ThreadExecute, ThreadStatus, ThreadTerminate); end; And done. You never have to define your own TThread derived class again. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] changes in TThread.OnTerminate in fcp-3.0?
Am 10.12.2015 12:15 schrieb "Luca Olivetti" : > It's obvious that you cannot free the thread from inside a "normal" Synchronize call, it's no so obvious you cannot do it in the OnTerminate method, so maybe a note could be added in > > http://wiki.freepascal.org/User_Changes_3.0 > > The new implementation is clearly betteronce you know *not* to free the thread inside OnTerminate. In all honesty it was never documented either that you *can* free the object from within OnTerminate and in other occurrences you wouldn't free the object that invoked the handler either, right? So why for TThread then? => no need to mention as a change Regards, Sven ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] changes in TThread.OnTerminate in fcp-3.0?
On Thu, Dec 10, 2015 at 02:49:50PM +0100, Sven Barth wrote: > Am 10.12.2015 12:15 schrieb "Luca Olivetti" : > > It's obvious that you cannot free the thread from inside a "normal" > Synchronize call, it's no so obvious you cannot do it in the > OnTerminate method, so maybe a note could be added in > > > > http://wiki.freepascal.org/User_Changes_3.0 > > > > The new implementation is clearly betteronce you know *not* to > > free > the thread inside OnTerminate. > > In all honesty it was never documented either that you *can* free the > object from within OnTerminate and in other occurrences you wouldn't > free the object that invoked the handler either, right? So why for > TThread then? => no need to mention as a change Is it worthwhile mentioning that OnTerminate gets called with Synchronize()? I had to dig into the source to verify this. Henry ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] changes in TThread.OnTerminate in fcp-3.0?
Am 10.12.2015 15:06 schrieb "Henry Vermaak" : > > On Thu, Dec 10, 2015 at 02:49:50PM +0100, Sven Barth wrote: > > Am 10.12.2015 12:15 schrieb "Luca Olivetti" : > > > It's obvious that you cannot free the thread from inside a "normal" > > Synchronize call, it's no so obvious you cannot do it in the > > OnTerminate method, so maybe a note could be added in > > > > > > http://wiki.freepascal.org/User_Changes_3.0 > > > > > > The new implementation is clearly betteronce you know *not* to > > > free > > the thread inside OnTerminate. > > > > In all honesty it was never documented either that you *can* free the > > object from within OnTerminate and in other occurrences you wouldn't > > free the object that invoked the handler either, right? So why for > > TThread then? => no need to mention as a change > > Is it worthwhile mentioning that OnTerminate gets called with > Synchronize()? I had to dig into the source to verify this. That would indeed be worth mentioning, but that would belong in the documentation then. Would you file a bug report, please? Regards, Sven ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] changes in TThread.OnTerminate in fcp-3.0?
On Thu, Dec 10, 2015 at 05:23:41PM +0100, Sven Barth wrote: > Am 10.12.2015 15:06 schrieb "Henry Vermaak" : > > Is it worthwhile mentioning that OnTerminate gets called with > > Synchronize()? I had to dig into the source to verify this. > > That would indeed be worth mentioning, but that would belong in the > documentation then. Would you file a bug report, please? Sure: http://bugs.freepascal.org/view.php?id=29166 Henry ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
[fpc-pascal] extreme memory growth using cthreads and default heap manager
I am experiencing a weird issue regarding threading and shared libraries
under Linux. I've boiled it down to a very simple test case in hopes you
can help me understand what I'm doing wrong.
I'm running this test with FPC 3.0 (built from source) on x86_64 Debian
Linux.
First, I have this shared library:
===
library fpcso1;
{$mode objfpc}{$H+}
uses
cthreads,
// cmem,
SysUtils;
procedure Blow; cdecl;
var
p : pbyte;
begin
p := GetMem(1024);
FreeMem(p);
p := nil;
end;
exports
Blow name 'fpcso1_blow';
end.
===
It is used by this program:
===
program fpcprog;
{$mode objfpc}{$H+}
uses
cthreads,
cmem,
Classes,
SysUtils,
DateUtils;
const
lib1Name = 'libfpcso1.so';
procedure fpcso1_blow; cdecl; external lib1Name;
var
finishedThreads : longint = 0;
runningThreads : longint = 0;
startedThreads : longint = 0;
type
TTestThread = class(TThread)
protected
procedure Execute; override;
public
constructor Create;
end;
constructor TTestThread.Create;
begin
inherited Create(true, 1024*1024);
FreeOnTerminate := true;
end;
procedure TTestThread.Execute;
begin
InterLockedIncrement(runningThreads);
try
fpcso1_blow();
finally
InterLockedIncrement(finishedThreads);
InterLockedDecrement(runningThreads);
end;
end;
procedure BeginTestThread();
var
tmpThread : TTestThread;
begin
tmpThread := TTestThread.Create();
tmpThread.Start;
end;
const
threadCount = 10;
threadMaxConcurrent = 30;
var
lastTime : TDateTime;
lastCount : longint = 0;
begin
lastCount := 0;
lastTime := now;
while (startedThreads < threadCount) do begin
if (runningThreads < threadMaxConcurrent) then begin
InterLockedIncrement(startedThreads);
BeginTestThread();
end else begin
sleep(1);
end;
if MilliSecondsBetween(lastTime, now) >= 1000 then begin
writeln(runningThreads, ' threads runningThreads, ',
finishedThreads, ' threads finished (',
finishedThreads-lastCount, '/sec)');
lastCount := finishedThreads;
lastTime := now;
end;
end;
while (runningThreads > 0) do begin
sleep(1000);
writeln(runningThreads, ' threads runningThreads, ', finishedThreads,
' threads finished (', finishedThreads-lastCount, '/sec)');
lastCount := finishedThreads;
end;
sleep(1000);
writeln(runningThreads, ' threads runningThreads, ',
finishedThreads, ' threads finished');
end.
===
When I execute the program, over the course of the ten seconds it takes
to run, I watch the process' memory usage grow to over a gigabyte:
1s 235.81 MiB
2s 351.34 MiB
3s 463.64 MiB
4s 575.35 MiB
5s 688.54 MiB
6s 799.47 MiB
7s 916.26 MiB
8s1.01 GiB
9s1.12 GiB
10s1.15 GiB
However, if I uncomment the "// cmem," line in the library, the memory does
not grow.
Why does using the default memory manager seem to leak memory like crazy if
I'm doing heavy threading? I'm setting FreeOnTerminate to true in
conjunction
with creating the thread suspended, then calling Start, which the
examples in the wiki and documentation seems to indicate is valid.
Is cmem *required* if I'm using threading on Linux? I hope not, because
while this example is boiled down to as simple as I could make it, I'm
having the same issue in my production application, in which I'm using my
own power-of-two memory manager, and I would really like to figure out
what I can do to continue to use my own memory manager, plus threading,
without it leaking so much memory for every thread that starts/finishes.
Thanks,
-Seth Grover
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] extreme memory growth using cthreads and default heap manager
Interesting: playing around with it some more, if I put an empty try/except block inside my library's function from the example I posted, it leaks the memory whether or not I'm using cmem. -SG -- Seth Grover Be kind to all of your neighbors Because they're just like you. And you're nothing special Unless they are too. On Thu, Dec 10, 2015 at 2:26 PM, Seth Grover wrote: > I am experiencing a weird issue regarding threading and shared libraries > under Linux. > ... > ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
