I have a possible idea for this problem. Since QGIS relies heavily on Python, it would be beneficial to integrate pip (and conda for conda builds) into QGIS. Maybe add a pip/conda section in the Plugin Manager so that people can easily install extra python packages? Now with this system, a plugin can be written which depends on a python package. And with the plugin would add metadata like the qgis-plugin-dev-tools toml file to specify library dependencies/requirements. In the plugins repo, the people reviewing the plugin would vet the list of required python packages to make sure it’s not requiring anything malicious (this replaces the need to maintain a list of “acceptable” packages). When a user goes to install a plugin that has python dependencies, they will be notified (aside from that information being presented in the plugin info) about additional dependencies which QGIS will automatically install for the user (if the user accepts).
-Ethan From: Joona Laine <joona.p.la...@gmail.com> Sent: Wednesday, October 23, 2024 8:10 AM To: Matthias Kuhn <matth...@opengis.ch> Cc: John Stevenson - BGS <jos...@bgs.ac.uk>; i...@opengis.it; qgis-developer <qgis-developer@lists.osgeo.org> Subject: Re: [QGIS-Developer] How to deal with QGIS plugins which install additional packages Qgis-plugin-dev-tools approach solves this problem by vendoring the packages and rewriting <https://github.com/nlsfi/qgis-plugin-dev-tools/blob/2df5c099c9c86700e0d323c67243902f1df46fce/src/qgis_plugin_dev_tools/build/rewrite_imports.py#L10> the imports so that "import module.x.y" imports are rewritten in a vendored format: "import something._vendor.module.x.y". Thus multiple plugins can have different versions of packages since they all import their own vendored versions. Joona ke 23. lokak. 2024 klo 14.58 Matthias Kuhn <matth...@opengis.ch<mailto:matth...@opengis.ch>> kirjoitti: Hi, This approach will work fine within limitations, as soon as multiple plugins ship the same library things become risky as there is no isolation between libraries. For python libraries, this may be caused by singletons being used and for native libraries (as in this example), it's easy to cause crashes by multiple versions of the same library exporting the same symbols being loaded in parallel. That being said: it will work fine in many cases, but I wouldn't promote this as "best practice". After all, python invented virtualenvs for good reasons -- each process will always run one environment (potentially composed of multiple cascading virtual envs, but never multiple "parallel" envs). Cheers Matthias On Wed, Oct 23, 2024 at 1:31 PM John Stevenson - BGS via QGIS-Developer <qgis-developer@lists.osgeo.org<mailto:qgis-developer@lists.osgeo.org>> wrote: Hi, Mergin Maps plugin also packages the dependencies (including the geodiff binary) into the plugin itself. I’m not sure how it handles cross-platform differences, though. Plugin: https://plugins.qgis.org/plugins/Mergin/#plugin-details GitHub Actions code: https://github.com/MerginMaps/qgis-plugin/blob/ef0b2502ddb4bcbc1670b0d82832e93b658c18b2/.github/workflows/packages.yml#L116 Cheers, John From: QGIS-Developer <qgis-developer-boun...@lists.osgeo.org<mailto:qgis-developer-boun...@lists.osgeo.org>> On Behalf Of Joona Laine via QGIS-Developer Sent: 23 October 2024 10:58 To: i...@opengis.it<mailto:i...@opengis.it> Cc: qgis-developer <qgis-developer@lists.osgeo.org<mailto:qgis-developer@lists.osgeo.org>> Subject: Re: [QGIS-Developer] How to deal with QGIS plugins which install additional packages One alternative way of managing the dependencies is to package the non-binary runtime dependencies (including licenses) with the plugin. This also tackles the problem with different versions of the same requirements between multiple plugins. There is a tool for that https://github.com/nlsfi/qgis-plugin-dev-tools which also has many more useful features for developing QGIS plugins. One example of plugins using this tool is pickLayer (https://plugins.qgis.org/plugins/pickLayer/) which bundles https://github.com/GispoCoding/qgis_plugin_tools with it. What do you think about this approach? Regards, Joona ke 23. lokak. 2024 klo 12.01 Info O.GIS via QGIS-Developer <qgis-developer@lists.osgeo.org<mailto:qgis-developer@lists.osgeo.org>> kirjoitti: I also did a similar thing in qgis2web plugin. I explained to the user that he can install qtwebengine to get the latest features and to do so he will have to click on a button that indicates that an installation will start. Here is the screen: Could it be okay? The code: try: if system == 'Windows': pip_exec = os.path.join(sysconfig.get_path("scripts"), "pip3") env = os.environ.copy() if full_proxy_url: env['http_proxy'] = full_proxy_url env['https_proxy'] = full_proxy_url subprocess.check_call([pip_exec, "install", "--upgrade", "PyQtWebEngine==5.15.6"], env=env) elif system == 'Linux': subprocess.check_call(["sudo", "apt-get", "install", "python3-pyqt5.qtwebengine"]) elif system == 'Darwin': # macOS subprocess.check_call(["brew", "install", "pyqt5"]) Andrea Ordonselli O.GIS - opengis.it<http://opengis.it> Da "QGIS-Developer" qgis-developer-boun...@lists.osgeo.org<mailto:qgis-developer-boun...@lists.osgeo.org> A "Matthias Kuhn" matth...@opengis.ch<mailto:matth...@opengis.ch> Cc "Thomas B via QGIS-Developer" qgis-developer@lists.osgeo.org<mailto:qgis-developer@lists.osgeo.org> Data Wed, 23 Oct 2024 16:16:43 +1000 Oggetto Re: [QGIS-Developer] How to deal with QGIS plugins which install additional packages On Wed, 23 Oct 2024, 4:07 pm Matthias Kuhn, <matth...@opengis.ch<mailto:matth...@opengis.ch>> wrote: On Wed, Oct 23, 2024 at 2:49 AM Nyall Dawson via QGIS-Developer <qgis-developer@lists.osgeo.org<mailto:qgis-developer@lists.osgeo.org>> wrote: On Wed, 23 Oct 2024, 9:20 am Greg Troxel via QGIS-Developer, <qgis-developer@lists.osgeo.org<mailto:qgis-developer@lists.osgeo.org>> wrote: Thomas B via QGIS-Developer <qgis-developer@lists.osgeo.org<mailto:qgis-developer@lists.osgeo.org>> writes: > Dear QGIS-Developers, > > Are there any guidelines from the QGIS project regarding whether a QGIS > plugin is allowed to autonomously install required packages using PIP or > similar tools without manual installation by the user? > > While this might seem convenient, I see it as a potential security risk, > especially if the user is not explicitly informed about what is happening > in the background. Agreed this is not ok. I think a plugin downloading anything to be executed or interpreted should be entirely prohibited. +1 . This practice should lead to a plugin being removed from the repositories. (Possibly we could do something on the code side too, eg by monkey patching over subprocess/etc and explicitly blocking execution of sip, with a developer-friendly exception stating this policy. It'd be easy for someone motivated to circumvent, but could at least be used to advise plugin developers that this is not acceptable practice...) We've tried to come up with a more transparent approach with support for requirements.txt (see https://github.com/opengisch/qpip). It is using pip but with a frontend which informs the user and lets him confirm an eventual installation. Is this approach generally acceptable? Well, I definitely trust yourself/OpenGIS significantly more then other random plugin developers 👍 I would personally feel safest if this was something officially endorsed, with an explicit allow list of acceptable packages. Nyall Matthias Nyall _______________________________________________ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org<mailto:QGIS-Developer@lists.osgeo.org> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer _______________________________________________ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org<mailto:QGIS-Developer@lists.osgeo.org> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer _______________________________________________ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org<mailto:QGIS-Developer@lists.osgeo.org> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses. _______________________________________________ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org<mailto:QGIS-Developer@lists.osgeo.org> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer DISCLAIMER: This message and any documents attached may contain confidential information and are intended only for the individual(s) named. If you are not the intended recipient, or the employee or agent authorized to received for the intended recipient, you should not disseminate, distribute or copy this e-mail and any attached documents. If you have received this e-mail in error, please immediately notify the sender at Remington & Vernick Engineers by replying to this e-mail and delete the original e-mail and any reply e-mail messages from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. Thank you.
_______________________________________________ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
