[fpc-pascal] question about FPC heap and threading

[Seth Grover]

These are very simple questions, but I just want to make sure a few of
my assumptions are correct:

1. The default FPC heap manager uses one heap per thread, right? I am
basing this assumption off of things I have read in various threads on
this and other mailing lists since around 2.2.x-ish. Looking at
heap.inc I see that the "freelists" variable is a threadvar, which
indicates to me that each thread has its own heap.

2. When a thread is started via, for example, BeginThread
(http://www.freepascal.org/docs-html/rtl/system/beginthread.html) this
heap, as well as all other threadvars in my code and the rtl code, is
setup and initialized automatically, correct?

3. When this thread exits (in the example of using BeginThread, when
the TThreadFunc function passed in to BeginThread terminates) the heap
for that thread, in addition to the other threadvars, are
automatically disposed, correct?

The reason I ask is that I have a program (which executes a lot of
worker threads via BeginThread) which is experiencing increasing
memory usage. heaptrc does not indicate that there are any leaks, but
it's a fairly complex program (we have some units where we're getting
memory from libc directly, don't ask) so I'm not even supposing at
this point that the increasing memory belongs to the FPC heaps at all,
but I'm just trying to make sure I understand everything that's going
on before I go making assumptions about where the memory allocation is
coming from.

Thanks,

-SG

--
This email is fiction. Any resemblance to actual events
or persons living or dead is purely coincidental.

Seth Grover
sethdgrover[at]gmail[dot]com
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] question about FPC heap and threading

[Jonas Maebe]

On 11 Aug 2010, at 23:35, Seth Grover wrote:

> 1. The default FPC heap manager uses one heap per thread, right?

Yes.

> 2. When a thread is started via, for example, BeginThread
> (http://www.freepascal.org/docs-html/rtl/system/beginthread.html) this
> heap, as well as all other threadvars in my code and the rtl code, is
> setup and initialized automatically, correct?

Yes.

> 3. When this thread exits (in the example of using BeginThread, when
> the TThreadFunc function passed in to BeginThread terminates) the heap
> for that thread, in addition to the other threadvars, are
> automatically disposed, correct?

Only the already freed blocks that are still around are automatically disposed, 
because allocated blocks may still be referenced by other threads. The memory 
reserved for holding threadvars is indeed freed.

> The reason I ask is that I have a program (which executes a lot of
> worker threads via BeginThread) which is experiencing increasing
> memory usage. heaptrc does not indicate that there are any leaks, but
> it's a fairly complex program (we have some units where we're getting
> memory from libc directly, don't ask) so I'm not even supposing at
> this point that the increasing memory belongs to the FPC heaps at all,
> but I'm just trying to make sure I understand everything that's going
> on before I go making assumptions about where the memory allocation is
> coming from.

You can add the "cmem" unit to your uses clause and see whether the memory 
ballooning still happens. If that solves the problem, then the problem is 
related to the FPC heap manager, otherwise it isn't. If it does not solve the 
problem, compile everything with -gv and run the program under Valgrind (in 
case you are under Linux or Mac OS X).


Jonas___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal

[fpc-pascal] Re: question about FPC heap and threading

[sethdgrover]

Jonas, thank you for your response.

I actually think I have discovered my problem: I use "BeginThread" but I do  
not use "EndThread". I was under the impression that was optional, but from  
reading some other threads and issues in mantis I can see that failing to  
call EndThread (ie., just letting the function exit) can orphan the thread  
handle, at least on linux.


To illustrate:

--
program Project1;

{$mode objfpc}{$H+}

uses
cthreads,
Classes,
SysUtils,
DateUtils;

{$R *.res}


var
finished : longint;

function threadFunc (p : pointer) : ptrint;
var
someMem : pointer;
begin
Writeln('thread ', longint(p), ' started');
someMem := GetMem(1024*1024);
sleep(100);
FreeMem(someMem);
someMem := nil;
Writeln('thread ', longint(p), ' finished');
InterLockedIncrement(finished);
result := 0;
// EndThread;
end;

var
i : longint;
secondsToRun : integer;
startTime : TDateTime;
begin
finished:=0;
secondsToRun := StrToIntDef(ParamStr(1), 20);
startTime := now;
i := 0;
while(SecondsBetween(now, startTime) < secondsToRun) do begin
BeginThread(@threadFunc, pointer(i));
inc(i);
sleep(50);
end;
while (finished < i) do begin
// do nothing
end;
Writeln(finished);
end.
--

Running that program causes the memory reported by "top" to skyrocket.  
However, if I uncomment the "EndThread" line, memory doesn't go up at all.  
So basically I just need to go around to any of the functions I pass in to  
be run by BeginThread and make sure they all explicitly call EndThread.


Thanks!

-SG
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal