Re: [fpc-pascal] How to find where my app consumes CPU?
On Wed, 19 May 2021 00:29:17 +0200, Bo Berglund via fpc-pascal wrote: >Could this tiny loop consume 6-7% CPU cycles? Thanks for all of the responses, I will test tomorrow. I have noted that there are no other service processes that are waiting for some action but show up with CPU cycles above 0 in the top display... So there is obviously something that can be done, possibly not by free pascal though... Most other services are written in other languages, I assume. Example for the two openvpn service processes I have on another RPi: top - 01:57:13 up 91 days, 11:04, 2 users, load average: 0.04, 0.04, 0.00 Tasks: 2 total, 0 running, 2 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.1 us, 0.0 sy, 0.0 ni, 99.9 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem : 948304 total,45656 free, 506952 used, 395696 buff/cache KiB Swap: 102396 total,86780 free,15616 used. 357904 avail Mem PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND 458 openvpn 20 08588 5180 4508 S 0.0 0.5 5:47.97 openvpn 438 openvpn 20 08588 5328 4568 S 0.0 0.6 5:45.86 openvpn -- Bo Berglund Developer in Sweden ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] FTP support gone - switch to HTTP ?
On 18/05/2021 11:49 am, Sven Barth via fpc-pascal wrote: > I don't think we need to mention any specific clients (and why did your > list not include Filezilla? :P ) +1 on both counts. :-D Regards, Graeme ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] How to find where my app consumes CPU?
Op 2021-05-18 om 23:29 schreef Travis Siegel via fpc-pascal: I'm not positive, but I'm pretty sure the sleep command in linux does not behave the same way it does in windows. As you know, in windows, a sleep command (even if delivered with a parameter of 0) gives up time slices to other programs on the system. This does not appear to be the case on linux. On linux, the sleep command simply suspends the process for the specified amount of time, but so far as I can tell, does nothing for unused cpu cycles. This is what "suspend" means, that process doesn't compete for cycles, so they go to the other processes. Since the sleep is a kernel call, I can't really imagine why this would not be the case. So my guess something else is spinning (e.g. in a thread). ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] How to find where my app consumes CPU?
On Tue, 18 May 2021 18:37:38 -0400 Travis Siegel via fpc-pascal wrote: > Apparently, you can release cpu cycles, but it's with the sched_yield > (section 2 in the man pages), not the sleep command on linux. What sleep command are you referring to? What do you mean with cpu cycles? Sleep works pretty well under Linux: uses sysutils; var i: integer; begin for i:=1 to 1 do sleep(1); end. time ./test1 real0m10,791s user0m0,021s sys 0m0,018s Mattias ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] How to find where my app consumes CPU?
On Wed, 19 May 2021 00:44:18 +0200, Martin Frb via fpc-pascal wrote: >Also CheckSyncronize afaik takes a timeout. >So if you do not need to check the variables every millisecond, then do > > While not (bSTerm or bSInt or bsHup) do > begin > //Here is where the server runs as defined elsewhere > //Eternal loop to wait for system messages > CheckSynchronize(50); //To get thread comm working > end; > So I tested by changing the loop to: While not (bSTerm or bSInt or bsHup) do begin //Here is where the server runs as defined elsewhere //Eternal loop to wait for system msg bSTerm, bSInt or bsHup CheckSynchronize(5); //To get thread comm working end; It still worked as before so I can get rid of the sleep() but it still runs at CPU = 6% while idling. I guess this "proves" that the main loop is not what is causing this CPU usage. Next I will search all of the code for instances of sleep() and see if there are other CPU "thieves" around. -- Bo Berglund Developer in Sweden ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] How to find where my app consumes CPU?
No doubt your sleep code works just fine. I'm not saying the sleep command doesn't work. I'm saying the sleep command doesn't release unused cpu cycles for other threads/programs to use. Apparently, if you want that behavior, you need to yield the cpu time your process would otherwise take, that's done with a different kernel function than sleep. Alexander hit the nail on the head though with his solution, so I'm satisfied that the original poster got what he needed, and I learned something new about linux processes as well, which makes for a good all around solution. On 5/19/2021 6:02 AM, Mattias Gaertner wrote: On Tue, 18 May 2021 18:37:38 -0400 Travis Siegel via fpc-pascal wrote: Apparently, you can release cpu cycles, but it's with the sched_yield (section 2 in the man pages), not the sleep command on linux. What sleep command are you referring to? What do you mean with cpu cycles? Sleep works pretty well under Linux: uses sysutils; var i: integer; begin for i:=1 to 1 do sleep(1); end. time ./test1 real0m10,791s user0m0,021s sys 0m0,018s Mattias ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] How to find where my app consumes CPU?
On Wed, 19 May 2021 at 14:36, Travis Siegel via fpc-pascal wrote: > > No doubt your sleep code works just fine. > > I'm not saying the sleep command doesn't work. > > I'm saying the sleep command doesn't release unused cpu cycles for other > threads/programs to use. No, fpc uses nanosleep() inside sysutils.sleep() which is documented to suspend execution (i.e. no busy waiting) so the kernel will switch to another thread/process. Henry ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] How to find where my app consumes CPU?
On Wed, 19 May 2021 11:45:34 +0200, Marco van de Voort via fpc-pascal wrote: >So my guess something else is spinning (e.g. in a thread). Seems likely based on my test with giving ChecSynchronize an argument 5 and not noticing any difference... So the original conundrum still applies: ** How to find where my app consumes CPU? ** The "other" items might be hidden inside other used classes such as the Indy10 components I use to implement the TCP/IP communications. But when searching for sleep through the complete project sources I came up empty-handed. Something else must be going on. I have a thread to handle the measuring sequence server and this uses timers (TFpTimer) in order to check if it is time to run a task, but when idling no task is running so no task execution thread spins off either... -- Bo Berglund Developer in Sweden ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] How to find where my app consumes CPU?
On 5/19/2021 9:53 AM, Henry Vermaak wrote: On Wed, 19 May 2021 at 14:36, Travis Siegel via fpc-pascal wrote: No doubt your sleep code works just fine. I'm not saying the sleep command doesn't work. I'm saying the sleep command doesn't release unused cpu cycles for other threads/programs to use. No, fpc uses nanosleep() inside sysutils.sleep() which is documented to suspend execution (i.e. no busy waiting) so the kernel will switch to another thread/process. That's useful information. I did run across nanosleep in my digging, but I wasn't aware the sleep command called it. I'm still a (little) puzzled though, why there are so many different suspend/sleep modes in linux kernel implementations. They appear to do similar things, so perhaps it's just incremental differences, depending on what's trying to be accomplished, or maybe, under the hood, they all call the same routines, I can't really say, but it sure has been educational digging around to find answers. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] How to find where my app consumes CPU?
On Wed, 19 May 2021 16:02:00 +0200, Bo Berglund via fpc-pascal wrote: >The "other" items might be hidden inside other used classes such as the Indy10 >components I use to implement the TCP/IP communications. >But when searching for sleep through the complete project sources I came up >empty-handed. >Something else must be going on. > >I have a thread to handle the measuring sequence server and this uses timers >(TFpTimer) in order to check if it is time to run a task, but when idling no >task is running so no task execution thread spins off either... So now I am down to the timers... I am using TFPTimer timers in the scheduler to handle various things, some of them are just one-shots to delay an action for some predetarmined time. These are only executing as one-shots. But the schedule timer restarts itself with an interval of 60 s. Another timer is used to handle a message queue between parts of the system. It runs at a shorter time, like a second or so. Restarts itself too. But it runs only if scheduling is active. I have no idea how TFpTimers work internally, maybe these cause extra CPU cycles while waiting for the time they should fire? I could switch off the scheduling timer, but it made no difference. The other timers running for other purposes I canot disable. An these fire more often. Right now the server is running at 10% when doing nothing, not even checking schedules... After I did a service restart it drops to about 5% -- Bo Berglund Developer in Sweden ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] How to find where my app consumes CPU?
> Am 19.05.2021 um 19:00 schrieb Bo Berglund via fpc-pascal > : > > On Wed, 19 May 2021 16:02:00 +0200, Bo Berglund via fpc-pascal > wrote: > >> The "other" items might be hidden inside other used classes such as the >> Indy10 >> components I use to implement the TCP/IP communications. >> But when searching for sleep through the complete project sources I came up >> empty-handed. >> Something else must be going on. >> >> I have a thread to handle the measuring sequence server and this uses timers >> (TFpTimer) in order to check if it is time to run a task, but when idling no >> task is running so no task execution thread spins off either... > > So now I am down to the timers... > I am using TFPTimer timers in the scheduler to handle various things, some of > them are just one-shots to delay an action for some predetarmined time. > These are only executing as one-shots. > But the schedule timer restarts itself with an interval of 60 s. > Another timer is used to handle a message queue between parts of the system. > It > runs at a shorter time, like a second or so. Restarts itself too. > But it runs only if scheduling is active. > > I have no idea how TFpTimers work internally, maybe these cause extra CPU > cycles > while waiting for the time they should fire? > I could switch off the scheduling timer, but it made no difference. > The other timers running for other purposes I canot disable. > An these fire more often. > > Right now the server is running at 10% when doing nothing, not even checking > schedules... > After I did a service restart it drops to about 5% Even it’s a service, running it with gprof enabled should be possible, no? ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] How to find where my app consumes CPU?
On Wed, 19 May 2021 19:42:27 +0200, Florian Klämpfl via fpc-pascal wrote: >Even its a service, running it with gprof enabled should be possible, no? I do not know what you mean, but the program is installed as a service but it is essentially a normal console application with no GUI parts. The reason it is run as a service is that it must run 24/7 on the target machine so this is how I am testing it too. But initially I always tested it by starting it in a terminal (actually a PuTTY SSH session from my Windows PC). Googling gave me this: https://www.freepascal.org/docs-html/user/userse56.html and this: https://wiki.freepascal.org/Profiling I have checked that gprof is found on path on the development machine, but I have never ever heard of it let alone use it. But if it is able to find out where the cycles are spent, then very good! Seems like there is a bit of read-up to do... Thanks for the suggestion, btw! -- Bo Berglund Developer in Sweden ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] How to find where my app consumes CPU?
On 18/05/2021 20:59, Bo Berglund via fpc-pascal wrote: I have a pretty sizable console app written with Delphi 15 years ago but ported to Linux using FreePascal (3.2.0) with Lazarus (2.0.12) as IDE. I have the same problem with high CPU usage for a 20-year-old Delphi App compiled in Lazarus(2.0) When run in debug mode within the Lazarus IDE it runs fine. When run stand-alone as a Windows exe the CPU usage shoots up to 30% and stays there. I found that the problem is in the Lazarus Application.OnIdle event, which fails to honour the Done parameter and runs continuously: http://docwiki.embarcadero.com/Libraries/Sydney/en/Vcl.Forms.TApplication.OnIdle Martin. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] How to find where my app consumes CPU?
On Wed, 19 May 2021 19:46:43 +0100, Martin Wynne via fpc-pascal wrote: >On 18/05/2021 20:59, Bo Berglund via fpc-pascal wrote: >> I have a pretty sizable console app written with Delphi 15 years ago but >> ported >> to Linux using FreePascal (3.2.0) with Lazarus (2.0.12) as IDE. > >I have the same problem with high CPU usage for a 20-year-old Delphi App >compiled in Lazarus(2.0) > >When run in debug mode within the Lazarus IDE it runs fine. > >When run stand-alone as a Windows exe the CPU usage shoots up to 30% and >stays there. > >I found that the problem is in the Lazarus Application.OnIdle event, >which fails to honour the Done parameter and runs continuously: > > >http://docwiki.embarcadero.com/Libraries/Sydney/en/Vcl.Forms.TApplication.OnIdle > >Martin. The link you posted shows a page where Forms are used. My app does not use Forms. Is your application a GUI type app? Mine is specifically ported to be a non-GUI console app so there are no GUI-realted items in there like Form dependencies. For example Application.Processmessages is totally non-existent. I had a lot of trouble removing all of the GUI-related stuff from the Delphi code when porting. Had to rewrite parts of the underlying system to get it done. -- Bo Berglund Developer in Sweden ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] How to find where my app consumes CPU?
On Wed, 19 May 2021 19:46:43 +0100 Martin Wynne via fpc-pascal wrote: >[...] > I found that the problem is in the Lazarus Application.OnIdle event, > which fails to honour the Done parameter and runs continuously: How do you know, it is a bug in OnIdle, and not something continuously emitting events? Have you tested in a small test application? Mattias ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] How to find where my app consumes CPU?
> So now I am down to the timers... > I am using TFPTimer timers in the scheduler to handle various things, > some of them are just one-shots to delay an action for some predetarmined > time. I don't know how TFpTimers work either, but when I want a timer in one of my console apps, I just do it by comparing a variable to "Now" like this: Var MyTimerVariable, MyTimerDelay : TDateTime; //Start Timer MyTimerVariable := Now + MyTimerDelay; Then in my main loop where I also have a Sleep (), I just do: If Now >= MyTimerVariable Then Do_Whatever; I can't imagine why you would need anything more complicated than that for a general timer. As I recall "Now" has a quite small resolution, much less than milliseconds. This is pretty much guaranteed to not use any system resources other than the compare at whatever frequency the main loop runs at.. in fact I regulate the frequency of the main loop in a similar way.. At the end of the main loop I have a tight loop with Sleep and the comparison to Now and when I'm past the minimum time I want for the main loop, I stop doing the loop: Repeat Sleep(1); Until Now >= MyTimerVariable; James ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
