Hi Ethan, The qpip tool does more or less what you are describing https://github.com/opengisch/qpip We are looking for more testing at the moment, if we get good feedback for it, a core adoption can be discussed.
Bests Matthias On Mon, Oct 28, 2024 at 2:11 PM Ethan Snyder via QGIS-Developer < qgis-developer@lists.osgeo.org> wrote: > I have a possible idea for this problem. Since QGIS relies heavily on > Python, it would be beneficial to integrate pip (and conda for conda > builds) into QGIS. Maybe add a pip/conda section in the Plugin Manager so > that people can easily install extra python packages? Now with this system, > a plugin can be written which depends on a python package. And with the > plugin would add metadata like the qgis-plugin-dev-tools toml file to > specify library dependencies/requirements. In the plugins repo, the people > reviewing the plugin would vet the list of required python packages to make > sure it’s not requiring anything malicious (this replaces the need to > maintain a list of “acceptable” packages). When a user goes to install a > plugin that has python dependencies, they will be notified (aside from that > information being presented in the plugin info) about additional > dependencies which QGIS will automatically install for the user (if the > user accepts). > > > > -Ethan > > > > *From:* Joona Laine <joona.p.la...@gmail.com> > *Sent:* Wednesday, October 23, 2024 8:10 AM > *To:* Matthias Kuhn <matth...@opengis.ch> > *Cc:* John Stevenson - BGS <jos...@bgs.ac.uk>; i...@opengis.it; > qgis-developer <qgis-developer@lists.osgeo.org> > *Subject:* Re: [QGIS-Developer] How to deal with QGIS plugins which > install additional packages > > > > Qgis-plugin-dev-tools approach solves this problem by vendoring the > packages and rewriting > <https://github.com/nlsfi/qgis-plugin-dev-tools/blob/2df5c099c9c86700e0d323c67243902f1df46fce/src/qgis_plugin_dev_tools/build/rewrite_imports.py#L10>the > imports so that "import module.x.y" imports are rewritten in a vendored > format: "import something._vendor.module.x.y". Thus multiple plugins can > have different versions of packages since they all import their own > vendored versions. > > > > Joona > > > > ke 23. lokak. 2024 klo 14.58 Matthias Kuhn <matth...@opengis.ch> > kirjoitti: > > Hi, > > > > This approach will work fine within limitations, as soon as multiple > plugins ship the same library things become risky as there is no isolation > between libraries. > > For python libraries, this may be caused by singletons being used and for > native libraries (as in this example), it's easy to cause crashes by > multiple versions of the same library exporting the same symbols being > loaded in parallel. > > That being said: it will work fine in many cases, but I wouldn't promote > this as "best practice". After all, python invented virtualenvs for good > reasons -- each process will always run one environment (potentially > composed of multiple cascading virtual envs, but never multiple "parallel" > envs). > > > > Cheers > > Matthias > > > > On Wed, Oct 23, 2024 at 1:31 PM John Stevenson - BGS via QGIS-Developer < > qgis-developer@lists.osgeo.org> wrote: > > Hi, > > Mergin Maps plugin also packages the dependencies (including the geodiff > binary) into the plugin itself. I’m not sure how it handles cross-platform > differences, though. > > > Plugin: > > https://plugins.qgis.org/plugins/Mergin/#plugin-details > > > > GitHub Actions code: > > > > > https://github.com/MerginMaps/qgis-plugin/blob/ef0b2502ddb4bcbc1670b0d82832e93b658c18b2/.github/workflows/packages.yml#L116 > > > Cheers, > John > > *From:* QGIS-Developer <qgis-developer-boun...@lists.osgeo.org> *On > Behalf Of *Joona Laine via QGIS-Developer > *Sent:* 23 October 2024 10:58 > *To:* i...@opengis.it > *Cc:* qgis-developer <qgis-developer@lists.osgeo.org> > *Subject:* Re: [QGIS-Developer] How to deal with QGIS plugins which > install additional packages > > > > One alternative way of managing the dependencies is to package the > non-binary runtime dependencies (including licenses) with the plugin. This > also tackles the problem with different versions of the same requirements > between multiple plugins. There is a tool for that > https://github.com/nlsfi/qgis-plugin-dev-tools which also has many more > useful features for developing QGIS plugins. > > > > One example of plugins using this tool is pickLayer ( > https://plugins.qgis.org/plugins/pickLayer/) which bundles > https://github.com/GispoCoding/qgis_plugin_tools with it. > > > > What do you think about this approach? > > > > Regards, > > Joona > > > > ke 23. lokak. 2024 klo 12.01 Info O.GIS via QGIS-Developer < > qgis-developer@lists.osgeo.org> kirjoitti: > > I also did a similar thing in qgis2web plugin. > > I explained to the user that he can install qtwebengine to get the latest > features and to do so he will have to click on a button that indicates that > an installation will start. > > Here is the screen: > > > > > > Could it be okay? > > > > The code: > > > > *try:* > > * if system == 'Windows':* > > * pip_exec = os.path.join(sysconfig.get_path("scripts"), > "pip3")* > > * env = os.environ.copy()* > > * if full_proxy_url:* > > * env['http_proxy'] = full_proxy_url* > > * env['https_proxy'] = full_proxy_url* > > * subprocess.check_call([pip_exec, "install", "--upgrade", > "PyQtWebEngine==5.15.6"], env=env)* > > * elif system == 'Linux':* > > * subprocess.check_call(["sudo", "apt-get", "install", > "python3-pyqt5.qtwebengine"])* > > * elif system == 'Darwin': # macOS* > > * subprocess.check_call(["brew", "install", "pyqt5"])* > > > > > > *Andrea Ordonselli* > > *O.GIS - opengis.it <http://opengis.it>* > > > > Da "QGIS-Developer" qgis-developer-boun...@lists.osgeo.org > > A "Matthias Kuhn" matth...@opengis.ch > > Cc "Thomas B via QGIS-Developer" qgis-developer@lists.osgeo.org > > Data Wed, 23 Oct 2024 16:16:43 +1000 > > Oggetto Re: [QGIS-Developer] How to deal with QGIS plugins which install > additional packages > > > > > > On Wed, 23 Oct 2024, 4:07 pm Matthias Kuhn, <matth...@opengis.ch> wrote: > > On Wed, Oct 23, 2024 at 2:49 AM Nyall Dawson via QGIS-Developer < > qgis-developer@lists.osgeo.org> wrote: > > > > On Wed, 23 Oct 2024, 9:20 am Greg Troxel via QGIS-Developer, < > qgis-developer@lists.osgeo.org> wrote: > > Thomas B via QGIS-Developer <qgis-developer@lists.osgeo.org> writes: > > > Dear QGIS-Developers, > > > > Are there any guidelines from the QGIS project regarding whether a QGIS > > plugin is allowed to autonomously install required packages using PIP or > > similar tools without manual installation by the user? > > > > While this might seem convenient, I see it as a potential security risk, > > especially if the user is not explicitly informed about what is happening > > in the background. > > Agreed this is not ok. I think a plugin downloading anything to be > executed or interpreted should be entirely prohibited. > > > > +1 . This practice should lead to a plugin being removed from the > repositories. > > > > (Possibly we could do something on the code side too, eg by monkey > patching over subprocess/etc and explicitly blocking execution of sip, with > a developer-friendly exception stating this policy. It'd be easy for > someone motivated to circumvent, but could at least be used to advise > plugin developers that this is not acceptable practice...) > > > > We've tried to come up with a more transparent approach with support for > requirements.txt (see https://github.com/opengisch/qpip). It is using pip > but with a frontend which informs the user and lets him confirm an eventual > installation. > > Is this approach generally acceptable? > > > > Well, I definitely trust yourself/OpenGIS significantly more then other > random plugin developers 👍 > > > > I would personally feel safest if this was something officially endorsed, > with an explicit allow list of acceptable packages. > > > > > > > > Nyall > > > > > > > > Matthias > > > > > > Nyall > > > > _______________________________________________ > QGIS-Developer mailing list > QGIS-Developer@lists.osgeo.org > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer > > _______________________________________________ > QGIS-Developer mailing list > QGIS-Developer@lists.osgeo.org > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer > > _______________________________________________ > QGIS-Developer mailing list > QGIS-Developer@lists.osgeo.org > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer > > > > This email and any attachments are intended solely for the use of the > named recipients. If you are not the intended recipient you must not use, > disclose, copy or distribute this email or any of its attachments and > should notify the sender immediately and delete this email from your > system. UK Research and Innovation (UKRI) has taken every reasonable > precaution to minimise risk of this email or any attachments containing > viruses or malware but the recipient should carry out its own virus and > malware checks before opening the attachments. UKRI does not accept any > liability for any losses or damages which the recipient may sustain due to > presence of any viruses. > > _______________________________________________ > QGIS-Developer mailing list > QGIS-Developer@lists.osgeo.org > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer > > DISCLAIMER: This message and any documents attached may contain > confidential information and are intended only for the individual(s) named. > If you are not the intended recipient, or the employee or agent authorized > to received for the intended recipient, you should not disseminate, > distribute or copy this e-mail and any attached documents. If you have > received this e-mail in error, please immediately notify the sender at > Remington & Vernick Engineers by replying to this e-mail and delete the > original e-mail and any reply e-mail messages from your system. E-mail > transmission cannot be guaranteed to be secure or error-free as information > could be intercepted, corrupted, lost, destroyed, arrive late or > incomplete, or contain viruses. The sender therefore does not accept > liability for any errors or omissions in the contents of this message, > which arise as a result of e-mail transmission. If verification is required > please request a hard-copy version. Thank you. > _______________________________________________ > QGIS-Developer mailing list > QGIS-Developer@lists.osgeo.org > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer >
_______________________________________________ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
