I use multiproc / proc to speed things up in python instead of a for loop, each step needs to be independent of the other steps for this to work. On an i7 with 4 cores I can process 8 layers at a time and hit 100% on all of them. I haven't used this inside qgis yet but for the relatively simple work I'm doing it works well.
Matt On Sat, Apr 17, 2021 at 2:12 AM Wolfgang Meinolf <[email protected]> wrote: > > Thanks for the excellent feedback and hints. First of all, my QGIS qork is > mainly with vector layers. Therefore unfortunately the OpenCL capabilities > don’t help. To my understanding this is mainly for accelerating heavy raster > calculations. Secondly and as already mentioned, my most urgent need is to > accelerate the execution of GRASS v.net.steiner and v.clean as part of my > toolbox processing scripts for a number of given route networks in separate > areas. > > > > However, on you feedback I searched a bit and found, that obviously since > QGIS3 some parallelization seems to be supported. In the developers coockbook > here: > https://docs.qgis.org/testing/en/docs/pyqgis_developer_cookbook/tasks.html#task-from-a-processing-algorithm > I found some promising info including the link to here: > https://www.opengis.ch/2018/06/22/threads-in-pyqgis3/. I will spend some time > to modify my loop through my separate network areas to try, to start python > jobs from inside the loop, which are then collected at the end. > > > > Thanks again for your good ideas and have a good weekend > > Br > > Wolfgang > > ------------- > > > > Hi, > > > > You can also activate the OpenCL option. Some algorithms have been optimized > (I believe it’s still experimental). If you have a compatible OpenCL video > card, it will run these on the card. If not, it can probably run on > compatible CPUs. > > > > https://www.itopen.it/opencl-acceleration-now-available-in-qgis/ > > > > I find it’s worth testing equivalent algorithms from different sources in the > Processing plugin is worth the trouble. Some Saga algorithm are faster. > Merging vector lines, for example, could take a long time using the QGIS > algorithm (talking hours) and was much faster (a few minutes) using the Saga > version of the tool. I also found that some algorithms are much faster to > handle when the files are not open beforehand in the layer manager. For > example, the Processing plugin can sometimes grab the needed files directly > on the hard drive rather than picking one that is already open in the layer > manager. Looking at the processor, memory and disk usage can also help > identify bottle necks in the hardware. > > Nicolas Cadieux > > https://gitlab.com/njacadieux > > > > Le 16 avr. 2021 à 03:34, Francesco Pelullo <[email protected]> a écrit : > > > > > > Il ven 16 apr 2021, 02:56 Stewart Holt <[email protected]> ha scritto: > > Parallelizing script execution is complex and I doubt that it is currently > done in QGIS. > > > > That's true. > > > > In Settings / Options / Rendering / there's a checkbox for render layers in > parallel using as many CPU cores you want, but It Is enabled by default so I > suppose that there are no others settings that could make QGIS more > aggressive in CPU usage. > > > > Regards > > > > > > > > _______________________________________________ > Qgis-user mailing list > [email protected] > List info: https://lists.osgeo.org/mailman/listinfo/qgis-user > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user > > _______________________________________________ > Qgis-user mailing list > [email protected] > List info: https://lists.osgeo.org/mailman/listinfo/qgis-user > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user _______________________________________________ Qgis-user mailing list [email protected] List info: https://lists.osgeo.org/mailman/listinfo/qgis-user Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
