Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Marco van de Voort
In our previous episode, Vinzent H?fler said: > >> So for a quick brainstorming approach: > >> > >> - Suspend/Resume should be gone (you can't guarantee it to work, > >>so there's no point in doing it at all) > > > > Your wish is my command :-) > > That was quick. I still have two wishes left,

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Vinzent Höfler
On Sat, 16 Oct 2010 00:17:33 +0200, Marco van de Voort wrote: In our previous episode, Vinzent H?fler said: Hmm. Actually, I did not think too hard about it yet. So for a quick brainstorming approach: - Suspend/Resume should be gone (you can't guarantee it to work, so there's no point i

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Marco van de Voort
In our previous episode, Vinzent H?fler said: > Hmm. Actually, I did not think too hard about it yet. > > So for a quick brainstorming approach: > > - Suspend/Resume should be gone (you can't guarantee it to work, >so there's no point in doing it at all) Your wish is my command :-) As of now

Re: [fpc-pascal] FPC for High Performance Computing (HPC) Components with Application API

2010-10-15 Thread Henry Vermaak
On 15 October 2010 22:41, Andrew Brunner wrote: > Do you have a link to the latest version of pthreads? I've told you before: NPTL is part of the glibc sources. Download the latest glibc sources and look in the nptl directory. The latest glibc release is 2.11.2. > It was my understanding that

Re: [fpc-pascal] FPC for High Performance Computing (HPC) Components with Application API

2010-10-15 Thread Vinzent Höfler
On Fri, 15 Oct 2010 23:41:36 +0200, Andrew Brunner wrote: Do you have a link to the latest version of pthreads? You probably mean NPTL (Native Posix Threading Library). It was my understanding that libc took over the pthreads project. I just took up this issue a few days ago so I recogni

Re: [fpc-pascal] FPC for High Performance Computing (HPC) Components with Application API

2010-10-15 Thread Marco van de Voort
In our previous episode, Andrew Brunner said: > Do you have a link to the latest version of pthreads? > > It was my understanding that libc took over the pthreads project. I > just took up this issue a few days ago so I recognize my info may be > stale or even wrong. pthreads means "posix thread

Re: [fpc-pascal] FPC for High Performance Computing (HPC) Components with Application API

2010-10-15 Thread Andrew Brunner
Do you have a link to the latest version of pthreads? It was my understanding that libc took over the pthreads project. I just took up this issue a few days ago so I recognize my info may be stale or even wrong. On Fri, Oct 15, 2010 at 4:19 PM, Henry Vermaak wrote: > On 15 October 2010 21:24,

Re: [fpc-pascal] FPC for High Performance Computing (HPC) Components with Application API

2010-10-15 Thread Henry Vermaak
On 15 October 2010 21:24, Andrew Brunner wrote: > Suggested Reading: >   http://ftp.gnu.org/gnu/glibc/glibc-linuxthreads-2.3.6.tar.gz Why would you want to look at this? It's not used any more (except for really old kernels). Henry ___ fpc-pascal mail

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Vinzent Höfler
On Fri, 15 Oct 2010 22:50:29 +0200, Jonas Maebe wrote: On 15 Oct 2010, at 22:31, Vinzent Höfler wrote: - Suspend/Resume should be gone (you can't guarantee it to work, so there's no point in doing it at all) FWIW, they're deprecated since Delphi 2009, so they will disappear over time. To

Re: [fpc-pascal] FPC for High Performance Computing (HPC) Components with Application API

2010-10-15 Thread Marco van de Voort
In our previous episode, Andrew Brunner said: > > IMO - it may be smarter to develop of a new threading paradigm. A HPC > threading system that addresses issues with waitfor/suspend/resume and > resolve them for *nix/windows/mac but do so knowing that there are > most likely tens of thousands of

Re: [fpc-pascal] FPC for High Performance Computing (HPC) Components with Application API

2010-10-15 Thread Felipe Monteiro de Carvalho
Just one suggestion: If the current TThread uses too many security facilities for your purpose, then I think that it may not be possible to make a single thread manager which fits all purposes. So you could create your own high-performance thread manager, and for most users which don't need this

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Jonas Maebe
On 15 Oct 2010, at 22:31, Vinzent Höfler wrote: > - Suspend/Resume should be gone (you can't guarantee it to work, > so there's no point in doing it at all) FWIW, they're deprecated since Delphi 2009, so they will disappear over time. To resume a tthread created with fcreatesuspended=true, the

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Vinzent Höfler
On Fri, 15 Oct 2010 22:13:36 +0200, Sven Barth wrote: I'm curious. How should this FPThread class look like? You saw flaws in Delphi's API so what would you do better? Hmm. Actually, I did not think too hard about it yet. So for a quick brainstorming approach: - Suspend/Resume should be

[fpc-pascal] FPC for High Performance Computing (HPC) Components with Application API

2010-10-15 Thread Andrew Brunner
I wanted to start a discussion based on current state of FPC and High Performance Computing (HPC). The premise of the thread is to discuss high scale threads. Currently FPC has matured to the point where it might be beneficial to start targeting a particular market for users. The first thing tha

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Sven Barth
On 15.10.2010 21:26, Vinzent Höfler wrote: IMO, cThreads has a lot of RFI. (Room for improvement). No. TThread has. Last time I looked into CThreads it was a more or less thin binding to the pthreads API. TThread has RFI. But are you then suggesting you want to improve w/o using libc !?! I

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Vinzent Höfler
On Fri, 15 Oct 2010 18:26:59 +0200, Andrew Brunner wrote: Externally suspending a thread *IS* absolutely unsafe. At least we're on the same page here. :) But the notion of suspending an ENGINE at runtime externally is totally expected. The thread itself MUST put itself in a suspended state.

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Vinzent Höfler
On Fri, 15 Oct 2010 20:15:51 +0200, Andrew Brunner wrote: On Fri, Oct 15, 2010 at 12:01 PM, Vinzent Höfler wrote: Exactly. It was poor implementation. You should have had a global barrier onExecute. That would unlock the thread after everything you needed was readable. If it's global, i

Re: [fpc-pascal] declaration of functions

2010-10-15 Thread Jonas Maebe
On 15 Oct 2010, at 17:18, Jürgen Hestermann wrote: > Sven Barth schrieb: >>> FPC requires the full declaration to be repeated. Can this be avoided >>> somehow? I have many functions/procedures without full declaration in >>> the implementation part. >> Add >> {$mode delphi} > > I would not sugge

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Andrew Brunner
On Fri, Oct 15, 2010 at 12:01 PM, Vinzent Höfler wrote: >> Exactly.  It was poor implementation.  You should have had a global >> barrier onExecute.  That would unlock the thread after everything you >> needed was readable. > > If it's global, it would unlock *any* thread at that time. Yes, belo

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Vinzent Höfler
On Fri, 15 Oct 2010 18:19:14 +0200, Andrew Brunner wrote: On Fri, Oct 15, 2010 at 10:57 AM, Vinzent Höfler wrote: If you access it inside the execute method, you more or less crash (or at least leak memory). You obviously had a problem with access ThreadID before it was assigned. Accessi

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Andrew Brunner
On Fri, Oct 15, 2010 at 11:03 AM, wrote: > On Fri, 15 Oct 2010 17:24:03 +0200, Sven Barth > wrote: > >> The problem isn't the pthreads library. The problem is the POSIX threading >> specification of which pthreads is an implementation (look at the comment in >> CSuspendThread again). > > Actuall

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Andrew Brunner
On Fri, Oct 15, 2010 at 10:57 AM, Vinzent Höfler wrote: > > Huh? Why should the pthreads library have a semaphore or similar on each > thread? Good question. And according to pThreads source comments they themselves did unwillingly. ___ fpc-pascal mail

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Andrew Brunner
On Fri, Oct 15, 2010 at 10:57 AM, Vinzent Höfler wrote: > If you access it inside the execute method, you more or > less crash (or at least leak memory). You obviously had a problem with access ThreadID before it was assigned. Accessing it should not arrive at a RAV. I'm not sure this is on top

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Vinzent Höfler
On Fri, 15 Oct 2010 17:39:59 +0200, Andrew Brunner wrote: BTW: I did notice that the semaphore was being used to suspend the thread. And I also know that that was it's only function... Suspend and Resume. The problem is not pthreads or anything, the problem is how the semantics of the Classe

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread JeLlyFish . software
On Fri, 15 Oct 2010 17:24:03 +0200, Sven Barth wrote: The problem isn't the pthreads library. The problem is the POSIX threading specification of which pthreads is an implementation (look at the comment in CSuspendThread again). Actually, it's not even a problem. Externally suspending a th

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Vinzent Höfler
On Fri, 15 Oct 2010 16:45:04 +0200, Andrew Brunner wrote: It was also suggested that PThread's Suspend and Resume may not do anything. What I would do (if that is the case), is try to find where PThread has access to the Semaphore or CriticalSection it has on that thread... Huh? Why should

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Andrew Brunner
On Fri, Oct 15, 2010 at 10:24 AM, Sven Barth wrote: > The problem isn't the pthreads library. The problem is the POSIX threading > specification of which pthreads is an implementation (look at the comment in > CSuspendThread again). POSIX is just like you say a specification. Suspend and Resume

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Andrew Brunner
On Fri, Oct 15, 2010 at 10:06 AM, Jonas Maebe wrote: > First of all, if you use tthread.create(true) (i.e., create a suspended > thread), then the "execute" method will never be called: BTW: I did notice that the semaphore was being used to suspend the thread. And I also know that that was it's

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Sven Barth
Am 15.10.2010 17:11, schrieb Andrew Brunner: On Fri, Oct 15, 2010 at 9:55 AM, Sven Barth wrote: Am 15.10.2010 16:31, schrieb Andrew Brunner: On Fri, Oct 15, 2010 at 9:24 AM, Jonas Maebe wrote: You replaced a bunch of semaphore create/lock/unlock/destroy operations with calls to a function

Re: [fpc-pascal] declaration of functions

2010-10-15 Thread Jürgen Hestermann
Sven Barth schrieb: FPC requires the full declaration to be repeated. Can this be avoided somehow? I have many functions/procedures without full declaration in the implementation part. Add {$mode delphi} I would not suggest to use {$mode delphi} but {$ModeSwitch REPEATFORWARD+} instead. This o

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Andrew Brunner
On Fri, Oct 15, 2010 at 9:55 AM, Sven Barth wrote: > Am 15.10.2010 16:31, schrieb Andrew Brunner: >> >> On Fri, Oct 15, 2010 at 9:24 AM, Jonas Maebe >>  wrote: >>> >>> You replaced a bunch of semaphore create/lock/unlock/destroy operations >>> with >>> calls to a function that does not do anything

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Jonas Maebe
On 15 Oct 2010, at 16:31, Andrew Brunner wrote: On Fri, Oct 15, 2010 at 9:24 AM, Jonas Maebe > wrote: You replaced a bunch of semaphore create/lock/unlock/destroy operations with calls to a function that does not do anything. I did not. I REMOVED create/destroy/lock/unlock. I ADDED calls

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Sven Barth
Am 15.10.2010 16:31, schrieb Andrew Brunner: On Fri, Oct 15, 2010 at 9:24 AM, Jonas Maebe wrote: You replaced a bunch of semaphore create/lock/unlock/destroy operations with calls to a function that does not do anything. I did not. I REMOVED create/destroy/lock/unlock. I ADDED calls to the

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Andrew Brunner
On Fri, Oct 15, 2010 at 9:24 AM, Michael Van Canneyt wrote: > > Thank you, > > I'll study it to see if/how we can do something to increase speed of > creating threads > in FPC. > > But if I understand you correctly, the WRITE_MEMORY_BARRIER() call is > beyond our reach, so there's little we can do

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Andrew Brunner
On Fri, Oct 15, 2010 at 9:24 AM, Jonas Maebe wrote: > You replaced a bunch of semaphore create/lock/unlock/destroy operations with > calls to a function that does not do anything. I did not. I REMOVED create/destroy/lock/unlock. I ADDED calls to the ThreadManager's own functions for thread Resu

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Michael Van Canneyt
Thank you, I'll study it to see if/how we can do something to increase speed of creating threads in FPC. But if I understand you correctly, the WRITE_MEMORY_BARRIER() call is beyond our reach, so there's little we can do about that. Michael. On Fri, 15 Oct 2010, Andrew Brunner wrote: rtl c

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Jonas Maebe
On 15 Oct 2010, at 15:54, Andrew Brunner wrote: rtl changes alone added 20-30% speed increase over the test case. And in the process they break all tthread suspending under Unix. CurrentTM.SuspendThread() will always fail there (it calls through to CSuspentThread): function CSuspendT

Re: [fpc-pascal] What are the issues involved in threads sharing variables?

2010-10-15 Thread Andrew Brunner
On Fri, Oct 15, 2010 at 5:20 AM, Frank Church wrote: > > I am writing an app involving 2 threads, and a timer loop in the UI. > > The first thread monitors network events, logs them into a database, appends > them to a list. > Any time I see append to a list I always think "RTLCriticalSection" >

[fpc-pascal] What are the issues involved in threads sharing variables?

2010-10-15 Thread Frank Church
I am writing an app involving 2 threads, and a timer loop in the UI. The first thread monitors network events, logs them into a database, appends them to a list. The second thread monitors the list, checks the events and updates a data structure, which is used by a timer loop in the main applicat

Re: [fpc-pascal] Re: declaration of functions

2010-10-15 Thread Felipe Monteiro de Carvalho
On Fri, Oct 15, 2010 at 11:41 AM, Uffe Kousgaard wrote: > Since I'm compiling for WinCE at the moment, would anybody know if there is > something wrong with the emulator from MS? > http://www.microsoft.com/downloads/en/details.aspx?FamilyID=A120E012-CA31-4BE9-A3BF-B9BF4F64CE72 > > It keeps stoppin

[fpc-pascal] Re: declaration of functions

2010-10-15 Thread Uffe Kousgaard
Thanks to all. I just found the link on http://www.freepascal.org/maillist.var to GMANE, so I can follow the list in my newsreader. But the correct syntax for the link to work is news:// , not nntp://. At least with outlook express. Since I'm compiling for WinCE at the moment, would anybody know

Re: [fpc-pascal] declaration of functions

2010-10-15 Thread Sven Barth
Am 15.10.2010 11:27, schrieb Michael Van Canneyt: On Fri, 15 Oct 2010, Jonas Maebe wrote: On 15 Oct 2010, at 11:07, Sven Barth wrote: Am 14.10.2010 14:50, schrieb Uffe Kousgaard: interface function somefunction(a: integer): integer; implementation function somefunction; begin result:= a*

Re: [fpc-pascal] declaration of functions

2010-10-15 Thread Michael Van Canneyt
On Fri, 15 Oct 2010, Jonas Maebe wrote: On 15 Oct 2010, at 11:07, Sven Barth wrote: Am 14.10.2010 14:50, schrieb Uffe Kousgaard: interface function somefunction(a: integer): integer; implementation function somefunction; begin result:= a*2; end; Add {$mode delphi} at the top of your u

Re: [fpc-pascal] declaration of functions

2010-10-15 Thread Sven Barth
Am 15.10.2010 11:13, schrieb Jonas Maebe: On 15 Oct 2010, at 11:07, Sven Barth wrote: Am 14.10.2010 14:50, schrieb Uffe Kousgaard: interface function somefunction(a: integer): integer; implementation function somefunction; begin result:= a*2; end; Add {$mode delphi} at the top of your un

Re: [fpc-pascal] TR: Strange Behaviour of TBits

2010-10-15 Thread Jonas Maebe
On 14 Oct 2010, at 17:24, Julien Devillers wrote: The above code returns 1, 2, 3, 3, 4 while it should return 1, 2, 3, 4, 5. I Built a TMyBits class using strictly the TBits.inc file from fpc and... the bug does not appear... ! I'm using lazarus 0.9.28.2 beta and fpc 2.2.4 with linux 64

Re: [fpc-pascal] declaration of functions

2010-10-15 Thread Jonas Maebe
On 15 Oct 2010, at 11:07, Sven Barth wrote: Am 14.10.2010 14:50, schrieb Uffe Kousgaard: interface function somefunction(a: integer): integer; implementation function somefunction; begin result:= a*2; end; Add {$mode delphi} at the top of your unit, then this "Delhpi compatible" syntax wi

Re: [fpc-pascal] declaration of functions

2010-10-15 Thread Michael Van Canneyt
On Thu, 14 Oct 2010, Uffe Kousgaard wrote: While trying to make delphi code compatible with FPC, I have hit this issue: Delphi allows: interface function somefunction(a: integer): integer; implementation function somefunction; begin result:= a*2; end; FPC requires the full declaration to b

Re: [fpc-pascal] declaration of functions

2010-10-15 Thread Sven Barth
Am 14.10.2010 14:50, schrieb Uffe Kousgaard: While trying to make delphi code compatible with FPC, I have hit this issue: Delphi allows: interface function somefunction(a: integer): integer; implementation function somefunction; begin result:= a*2; end; FPC requires the full declaration to be

Re: [fpc-pascal] declaration of functions

2010-10-15 Thread Jonas Maebe
On 14 Oct 2010, at 14:50, Uffe Kousgaard wrote: While trying to make delphi code compatible with FPC, I have hit this issue: Delphi allows: interface function somefunction(a: integer): integer; implementation function somefunction; begin result:= a*2; end; FPC requires the full declaratio

[fpc-pascal] declaration of functions

2010-10-15 Thread Uffe Kousgaard
While trying to make delphi code compatible with FPC, I have hit this issue: Delphi allows: interface function somefunction(a: integer): integer; implementation function somefunction; begin result:= a*2; end; FPC requires the full declaration to be repeated. Can this be avoided somehow? I ha

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Michael Van Canneyt
On Thu, 14 Oct 2010, Andrew Brunner wrote: What are the exact differences from this test to your last one? Would be nice to know that... :) Every barrier causes a significant increase in time. In high performance parallel computing time is something we minimize. Big reductions in linear ex

Re: [fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

2010-10-15 Thread Sven Barth
Am 14.10.2010 20:09, schrieb Andrew Brunner: What are the exact differences from this test to your last one? Would be nice to know that... :) Every barrier causes a significant increase in time. In high performance parallel computing time is something we minimize. Big reductions in linear exe