On Sun, Jun 16, 2024 at 5:59 AM Frank Steinmetzger <war...@gmx.de> wrote: > > Am Sat, Jun 15, 2024 at 04:07:28PM -0700 schrieb Mark Knecht: > > > Now, the fun part. I wrote you a little Python program which on > > my system is called Dales_Loop.py. This program has 3 > > parameters - a value to count to, the number of cores to be used, > > and a timeout value to stop the program. Using a program like > > this can give you repeatable results. > > FYI, there is a problem with your approach: python is not capable of true > multiprocessing. While you can have multiple threads in your programm, in > the end they are executed by a single thread in a time-sharing manner. > > This problem is known as the GIL—the Global Interpreter Lock. Unless you use > an external program to do the actual CPU work, i.e. let the linux kernel do > the actual parallelism and not python, your program is not faster than doing > everything in a single loop. > > See this page for a nice example which does basically the same as your > program (heading “The Impact on Multi-Threaded Python Programs”), including > some comparative benchmarking between single loop and threaded loops: > https://realpython.com/python-gil/ >
I'm sorry Frank but apparently you didn't read the code. Indeed the GIL i an issue but this program uses the multiprocessing library and starts processes, not threads. Running the program for 30 seconds with different values of num_processes I see 1 2212 2 4444 3 6107 4 8199 5 10174 I additionally I see roughly the number of num_process python jobs running in btop at all times the program is running clearly demonstrating each has its own process ID and are being managed by the kernel. As each process is completed a new one is started with a new process number. I use this library successfully in a home built AI program I'm writing and I clearly get roughly through the number of processes * data sets during training, validation and testing. If someone wants to use a packaged stress test program I see nothing wrong with that. If you don't want to read the code and understand it for yourself, then I see nothing wrong with that either, but please, if you're gonna talk to me about a little code snippet then at least read it , run it and understand it first. Dale - sorry to bother you. Mark
