Re: Terminal Emulator (Posting On Python-List Prohibited)
On 2024-05-20 00:26:03 +0200, Roel Schroeven via Python-list wrote: > Skip Montanaro via Python-list schreef op 20/05/2024 om 0:08: > > > Modern debian (ubuntu) and fedora block users installing using pip. > > > > Even if you're telling it to install in ~/.local? I could see not allowing > > to run it as root. > > I assumed pip install --user would work, but no. I tried it (on Debian 12 > (bookworm)): > > > $ pip install --user docopt > > error: externally-managed-environment > > > > × This environment is externally managed > > ╰─> To install Python packages system-wide, try apt install > > python3-xyz, where xyz is the package you are trying to > > install. > > > > If you wish to install a non-Debian-packaged Python package, > > create a virtual environment using python3 -m venv path/to/venv. > > Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make > > sure you have python3-full installed. > > > > If you wish to install a non-Debian packaged Python application, > > it may be easiest to use pipx install xyz, which will manage a > > virtual environment for you. Make sure you have pipx installed. > > > > See /usr/share/doc/python3.11/README.venv for more information. > > > > note: If you believe this is a mistake, please contact your Python > > installation or OS distribution provider. You can override this, at the > > risk of breaking your Python installation or OS, by passing > > --break-system-packages. > > hint: See PEP 668 for the detailed specification. > > Exactly the same output for sudo pip install. This message (quoted in all its glory) is too long to be useful. The important bit is at the end: > > You can override this, at the risk of breaking your Python > > installation or OS, by passing --break-system-packages. (I admit I didn't see this the first time I got this message) python3 -m pip install --user --break-system-packages does indeed install into ~/.local/lib/python3.XX/site-packages. This inconvenient, but otoh I have accidentally installed packages into ~/.local in the past, so maybe it's good to make that more explicit. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature -- https://mail.python.org/mailman/listinfo/python-list
Re: venvs vs. package management
There are several independent problems here: 1. Very short release cycle. This is independent of the Python venv module but is indirectly influenced by Python's own release cycle. Package maintainers don't have time for proper testing, they are encouraged to release a bunch of new (and poorly tested) versions, and they never get a break. So, when you install the latest, there will be something else broken. There's never a window to properly test anything. 2. Python made a very short-sighted decision about how imports work. Python doesn't have a concept of "application", and therefore there's no way to specify dependencies per application (and imports import anything that's available, not versioned). That's why every Python application ends up carrying its own Python, with the version of its own dependencies around. Python's venv module is just an acknowledgement of this design flaw. I.e. the proper solution would've been a concept of application and per-application dependency specification, but instead we got this thing that doesn't really work (esp. when native modules and shared libraries are considered), but it "works" often enough to be useful. 3. The Python community grew to be very similar to what PHP 4 was, where there were several "poisonous" examples, which were very popular on the Web, which popularized a way of working with MySQL databases that was very conducive to SQL injections. Python has spread very bad ideas about project management. Similar to how PHP came up with mysql_real_escape() and mysql_this_time_promise_for_real_escape() and so on functions, Python came up with bad solutions to the problems that had to be fixed by removing bad functionality (or, perhaps, education). So, for example, it's very common to use requirements.txt, which is generated by running pip freeze (both practices are bad ideas). Then PyPA came up with a bunch of bad ideas in response to problems like this, eg. pyproject.toml. In an absurd way very much mirroring the situation between makefiles and makefiles generated by autotools, today Python developers are very afraid of doing simple things when it comes to project infrastructure (it absolutely has to be a lot of configuration fed into another configuration, processed by a bunch of programs to generate even more configuration...) And most Python programmers don't really know how the essential components of all of this infrastructure work. They rely on a popular / established pattern of insane multi-step configuration generation to do simple things. And the tradition thus developed is so strong, that it became really cultish. This, of course, negatively contributes to the overall quality of Python packages and tools to work with them. Unfortunately, the landscape of Python today is very diverse. There's no universally good solution to package management because it's broken in the place where nobody is allowed to fix it. Commercial and non-commercial bodies alike rely on people with a lot of experience and knowledge of particular Python gotchas to get things done. (Hey, that's me!) And in different cases, the answer to the problem will be different. Sometimes venv is good enough. Other times you may want a container or a vm image. Yet in a different situation you may want a PyPA or conda package... and there's more. On Sun, May 19, 2024 at 4:05 PM Piergiorgio Sartor via Python-list wrote: > > On 19/05/2024 08.49, Peter J. Holzer wrote: > [...] > > That's what package management on Linux is for. Sure, it means that you > > won't have the newest version of anything and some packages not at all, > > but you don't have to care about dependencies. Or updates. > > Well, that doesn't work as well. > Distributions do not pack everything, this > also depending on licenses. > > Sometimes, or often, you need to use the > *latest* version of something, due to some > bugfix or similar. > > The distribution does not always keep up > to date everything, so you're stuck. > > The only solution is a venv, with all > needed packages for the given task. > > Typical problem with PyTorch / TensorFlow. > > In case of trouble, the first answer is: > "Check with the latest (nightly) release". > > Which means installing something *outside* > the Linux distribution support. > And this impossible, because this will pull > in dependencies like crazy, which are not > (yet) in the Linux distribution path. > > Saying it differently, the latest greatest > update is not a wish, it's a must... > > So, long story short, the only solution I > know are venvs... > > Of course, other solutions are welcome! > > bye, > > -- > > piergiorgio > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Terminal Emulator (Posting On Python-List Prohibited)
I'm on Manjaro Of course, I'm not here to tell you how to use your computer, and it's great that you're using Linux, but I'd suggest that you look into installing Arch Linux proper. Arch Linux isn't as difficult as people make it out to be (I'd argue that anyone who's had to deal with the Calamares installer has seen worse), and it's hugely rewarding as it gives you a fundamental overview of how your system operates and what makes it tick, since you need to install it yourself manually. Plus, Manjaro holds back packages by about a month or so, sometimes more, which breaks AUR packages, which are designed around Arch Linux's packages, which are newer. On top of this, the Manjaro team just can't be trusted with basic things like not having their SSL certs expire on their website (this has happened half a dozen times in the past, which is embarassing, given that things like certbot make installing a certificate the easiest thing in the world). Again, I'm not some power hungry elitist Arch Linux shill or whatever, it's your computer, use it how you want, these are just my suggestions. Don't say I didn't warn you though :) -- https://mail.python.org/mailman/listinfo/python-list
Re: Terminal Emulator (Posting On Python-List Prohibited)
On 20/05/2024 10:58, Peter J. Holzer wrote: On 2024-05-20 00:26:03 +0200, Roel Schroeven via Python-list wrote: Skip Montanaro via Python-list schreef op 20/05/2024 om 0:08: Modern debian (ubuntu) and fedora block users installing using pip. Even if you're telling it to install in ~/.local? I could see not allowing to run it as root. I assumed pip install --user would work, but no. I tried it (on Debian 12 (bookworm)): $ pip install --user docopt error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install. If you wish to install a non-Debian-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make sure you have python3-full installed. If you wish to install a non-Debian packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed. See /usr/share/doc/python3.11/README.venv for more information. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. Exactly the same output for sudo pip install. This message (quoted in all its glory) is too long to be useful. The important bit is at the end: You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. (I admit I didn't see this the first time I got this message) python3 -m pip install --user --break-system-packages does indeed install into ~/.local/lib/python3.XX/site-packages. This inconvenient, but otoh I have accidentally installed packages into ~/.local in the past, so maybe it's good to make that more explicit. hp Perhaps an alias like so: $ alias 'pip install'='pip install --user --break-system-packages' Would work here? Someone please advise if that is a good idea. -- https://mail.python.org/mailman/listinfo/python-list
pip and venvs on Debian (was: Terminal Emulator)
Alan Gauld via Python-list writes: > On 18/05/2024 19:12, Piergiorgio Sartor via Python-list wrote: > > >> So venvs make managing all that pretty convenient. Dunno why everybody's > >> so down on venvs... > > Not so much down on them, they are just one extra step that's > mostly not needed(in my use case) Years ago, I used to have trouble with pip install --user on Debian -- sometimes things would end up under .local, sometimes in other places that I've forgotten. So I reluctantly started using venvs. And you know, they work fine. I have one master venv that I created with python3 -m venv --system-site-packages ~/pythonenv/envname and I activate that automatically when I log in, so when I need to install anything that Debian doesn't package, I just pip install it (no --user or --break-system-packages needed) and it installs to that venv. Every so often I need to regenerate it (like when Debian updates the system Python version) but that's easy to do: I don't try to duplicate what's installed there, I just delete the old venv, create a new one and then pip install packages as needed. I have a few special venvs (without --system-site-packages) for specific purposes, but most of the time I'm just using my default venv and it's all pretty transparent and automatic. I know this isn't the usual pythonista model of "you should have a zillion different venvs, one for each program you use, and never use system Python packages", but it works well for me: my pip installed packages are all in a predictable place, and I get security updates for all the software Debian *does* package. That's my biggest beef with pip, the lack of an easy way to update everything at once, and it's the reason I prefer Debian packages when available. ...Akkana -- https://mail.python.org/mailman/listinfo/python-list
