Re: on writing a number as 2^s * q, where q is odd
On Sun, 3 Dec 2023 at 10:25, Julieta Shem via Python-list wrote: > > Alan Bawden writes: > > > > def powers_of_2_in(n): > > bc = (n ^ (n - 1)).bit_count() - 1 > > return bc, n >> bc > > That's pretty fancy and likely the fastest. It might be the fastest but it depends how big you expect n to be and how many trailing zeros you expect. If n is a very large integer then this does three large integer operations in (n^(n-1)).bit_count(). They are relatively fast operations but all linear in bit size. By contrast a check like n & 1 is O(1) and half the time proves that no further steps are necessary. The mpmath library needs this exact operation and is generally intended for the large n case so I checked how it is implemented there: https://github.com/mpmath/mpmath/blob/f13ea4dc925d522062ac734bd19a0a3cc23f9c04/mpmath/libmp/libmpf.py#L160-L177 That code is: # Strip trailing bits if not man & 1: t = trailtable[man & 255] if not t: while not man & 255: man >>= 8 exp += 8 bc -= 8 t = trailtable[man & 255] man >>= t exp += t bc -= t The trailtable variable is a pre-initialised list of shifts needed to remove zeros from an 8-bit integer. The bc variable here is just bc=man.bit_length() which is redundant but this code predates the addition of the int.bit_length() method. In principle this could use a large number of man>>=8 shifts which would potentially be quadratic in the bit size of man. In practice the probability of hitting the worst case is very low so the code is instead optimised for the expected common case of large man with few trailing zeros. -- Oscar -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
On Fri, 29 Dec 2023 at 13:04, Left Right via Python-list wrote: > > Wow. That place turned out to be the toxic pit I didn't expect. > > It's a shame that a public discussion of public goods was entrusted to > a bunch of gatekeepers with no sense of responsibility for the thing > they keep the keys to. Here is the discussion referred to: https://discuss.python.org/t/what-is-install-paths-to-in-wheel-file/42005 I don't see anything "toxic" in that discussion. You asked questions and people took the time to give clear answers. The basic answer to your question is that PEP 491 was never completed and so there is no accepted specification of the Install-Paths-To feature that it had been intended to introduce. The PEP text itself is reasonably clear about this and also links to the up to date specifications: https://peps.python.org/pep-0491/#pep-deferral Instead for understanding the wheel format the appropriate document is: https://packaging.python.org/en/latest/specifications/binary-distribution-format/ That document does not mention Install-Paths-To because it documents the standards as defined and accepted via the PEP process but PEP 491 was never accepted. -- Oscar -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
On Fri, 29 Dec 2023 at 22:38, Left Right via Python-list wrote: > > > Then your understanding is flat-out wrong. Encouraging participation > > by everyone DOES mean deleting what is unproductive, offensive, and > > likely to discourage participation. > > I haven't written anything unproductive or offensive. I offered > constructive criticism with a detailed plan on how to fix the problem. > The forum owners chose to ban me because they don't like hearing that > the code they've written is bad. And that's the long and the short of > it. This has been a pattern in behavior of PyPA members I've > interacted with so far. You are conflating several different groups of people. The PyPA are the people who currently maintain the code for various libraries/tools. That is very often not the same as the people who originally wrote the code for the same libraries/tools or for preceding ones. Neither group is the same as the forum moderators (I suspect that there is no intersection between the moderators and the PyPA etc.). > And whenever they had a chance, they'd use it > to pretend that the problems I'm talking about don't exist by deleting > every mention of the problem. That is an example of unproductive and > offensive behavior because it produces nothing and wastes my time I've > dedicated to locating, reporting and solving their problem. Actually you are wasting the time of others by putting across inaccurate and unhelpful information in a rude way and at the same time criticising others without really understanding who you are criticising and for what. Your contribution is unhelpful mostly (but not exclusively) because of the way that you choose to communicate. I did not realise earlier what you were referring to but I see now that I have email notifications with the content of your posts that were deleted. I am not surprised that they were deleted and that you were banned because if I was a moderator looking at those then I would not expect a promising future for your interactions with others in the forum. There is some significant irony in you describing the forum as a "toxic pit" for deleting your posts. I don't always agree with the moderators and I am not sure that I would have reacted the way that they did but these threads remind me precisely why moderation (including deleting posts such as yours) is needed to *prevent* a forum from turning into a toxic pit. -- Oscar -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
On Sun, 31 Dec 2023 at 00:35, Left Right wrote: > > It's not for you to choose the way I communicate. There are accepted > boundaries, and I'm well within those boundaries. Anything beyond that > is not something I'm even interested in hearing your opinion on. You might not be interested in my opinion but you might want to reflect on the fact that although you consider your behavior to be within "accepted boundaries" the evidence here (and in the forum) suggests that others do not and so your notion of what is "accepted" is not universally shared. I am not going to reply to your other points except to say that if you want to influence anything then I expect that you would have more success with a different approach. To anyone else considering replying in this thread: please don't. I very much doubt that anything good will happen here. -- Oscar -- https://mail.python.org/mailman/listinfo/python-list
Re: Couldn't install numpy on Python 2.7
On Wed, 12 Jun 2024 at 22:38, AVI GROSS via Python-list wrote: > > The discussion though was about a specific OP asking if they can fix their > problem. One solution being suggested is to fix a deeper problem and simply > make their code work with a recent version of python 3. The OP has not replied with any explanation as to why they are using Python 2.7 and has not said whether they have any existing code that only works with Python 2.7. It is unclear at this point whether there is any reason that they shouldn't just install a newer version of Python. They are seeing a warning that explicitly says "You can upgrade to a newer version of Python to solve this". I don't know whether that SSL warning is directly connected to pip not finding any versions of numpy but with the available information so far that seems like the first thing to consider. It is entirely reasonable to start by suggesting to use a newer version of Python until some reason is given for not doing that. -- Oscar -- https://mail.python.org/mailman/listinfo/python-list
Re: Couldn't install numpy on Python 2.7
On Wed, 12 Jun 2024 at 23:11, Chris Angelico via Python-list wrote: > > On Thu, 13 Jun 2024 at 07:57, Oscar Benjamin via Python-list > wrote: > > They are seeing a warning that explicitly says "You can upgrade to a > > newer version of Python to solve this". I don't know whether that SSL > > warning is directly connected to pip not finding any versions of numpy > > but with the available information so far that seems like the first > > thing to consider. > > I think it is; AIUI, with an ancient SSL library, pip is unable to > download packages safely from the current pypi server. So if anyone > actually does need to use pip with Python 2.7, they probably need to > set up a local server, using older encryption protocols (which should > therefore NOT be made accessible to the internet). Since pip can't > contact the upstream pypi, there's no available numpy for it to > install. I don't know much about SSL and related networking things especially on Windows. I would be surprised if pip on old Python can't install from current PyPI though. I imagine that something strange has happened like a new version of pip running on an old version of Python or old Python on new OS (or old PyCharm...). There is no problem using Python 2.7 with pip and PyPI on this Linux machine but I guess it has a newer SSL library provided by the OS: $ pip install numpy DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality. Collecting numpy Downloading numpy-1.16.6-cp27-cp27mu-manylinux1_x86_64.whl (17.0 MB) || 17.0 MB 14.3 MB/s Installing collected packages: numpy Successfully installed numpy-1.16.6 If it is actually the case that pip on Python 2.7 (on Windows) cannot download from PyPI then an easier option rather than creating a local server would just be to download the numpy wheels from PyPI using a browser: https://pypi.org/project/numpy/1.15.4/#files Then you can do pip install .\numpy-1.15.4-cp27-none-win_amd64.whl Using a newer version of Python is still my primary suggestion though. -- Oscar -- https://mail.python.org/mailman/listinfo/python-list
Re: Couldn't install numpy on Python 2.7
On Wed, 12 Jun 2024 at 23:52, Greg Ewing via Python-list wrote: > On 13/06/24 10:09 am, Chris Angelico wrote: > > So if anyone > > actually does need to use pip with Python 2.7, they probably need to > > set up a local server > > You should also be able to download a .tar.gz from PyPI and use pip > to install that. Although you'll have to track down the dependencies > yourself in that case. It is almost certainly better to download the wheel (.whl) file rather than the sdist (.tar.gz) file. Building NumPy from source needs not just compilers etc but also you first need to build/install a BLAS library. -- Oscar -- https://mail.python.org/mailman/listinfo/python-list
Re: Best use of "open" context manager
On Sat, 6 Jul 2024 at 11:55, Rob Cliffe via Python-list
wrote:
>
> Consider this scenario (which I ran into in real life):
> I want to open a text file and do a lot of processing on the lines
> of that file.
> If the file does not exist I want to take appropriate action, e.g.
> print an error message and abort the program.
> I might write it like this:
>
> try:
> with open(FileName) as f:
> for ln in f:
> print("I do a lot of processing here")
> # Many lines of code here .
> except FileNotFoundError:
> print(f"File {FileName} not found")
> sys.exit()
>
> but this violates the principle that a "try" suite should be kept small,
> so that only targeted exceptions are trapped,
> not to mention that having "try" and "except" far apart decreases
> readability.
This is catching a targeted exception (FileNotFoundError) so I think
it is fine. If the intention is just to call sys.exit() on error then
I wouldn't worry too much about having too much code in the try. Just
make sure that you do this in any other place where you open a file as
well.
One possible improvement is that you could catch the exception and use
its filename attribute:
except FileNotFoundError as e
print(f"File {e.filename} not found")
That way if you did catch the wrong FileNotFoundError then at least
you print the correct filename.
Alternatively:
except FileNotFoundError as e
if e.filename != FileName:
raise # re-raise if not the intended exception
print(f"File {e.filename} not found")
For readability I would just move the many lines of code into a
separate function.
The reason to avoid having too much code in the try mainly applies to
situations where you are going to do something other than call
sys.exit() and the exception is overly generic like ValueError or
TypeError. If the exception can easily be raised by a bug or something
other than the intended cause then it is bad to catch exceptions
around a larger block of code.
If it is expected that the caller of a function might have good reason
to catch the exception and handle it somehow then it is better to make
a dedicated exception class and raise that instead. When there is only
one place in the code that raises a particular exception type and only
one place that catches it then it is usually going to be clear that
you are catching the expected exception.
--
Oscar
--
https://mail.python.org/mailman/listinfo/python-list
Re: Relatively prime integers in NumPy
(posting on-list this time) On Thu, 11 Jul 2024 at 15:18, Popov, Dmitry Yu via Python-list wrote: > > Dear Sirs. > > Does NumPy provide a simple mechanism to identify relatively prime integers, > i.e. integers which don't have a common factor other than +1 or -1? For > example, in case of this array: > [[1,5,8], > [2,4,8], > [3,3,9]] > I can imagine a function which would return array of common factors along > axis 0: [1,2,3]. Those triples of numbers along axis 1 with the factor of1 or > -1 would be relatively prime integers. It sounds like you want the gcd (greatest common divisor) of each row. The math module can do this: In [1]: a = [[1,5,8], ...: [2,4,8], ...: [3,3,9]] In [2]: import math In [3]: [math.gcd(*row) for row in a] Out[3]: [1, 2, 3] NumPy can also do it apparently: In [10]: np.gcd.reduce(np.transpose(a)) Out[10]: array([1, 2, 3]) https://en.wikipedia.org/wiki/Greatest_common_divisor -- Oscar -- https://mail.python.org/mailman/listinfo/python-list
Re: Division-Bug in decimal and mpmath
On Sat, 14 Dec 2024 at 19:02, Mark Bourne via Python-list
wrote:
>
> Martin Ruppert wrote:
> > Hi,
> >
> > the division 0.4/7 provides a wrong result. It should give a periodic
> > decimal fraction with at most six digits, but it doesn't.
> >
> > Below is the comparison of the result of decimal, mpmath, dc and calc.
...
>
> I looks like you might be running into limitations in floating-point
> numbers. At least with decimal, calculating 4/70 instead of 0.4/7
> appears to give the correct result. As does:
> ```
> from decimal import Decimal as dec
> z2 = dec(4) / dec(10)
> print(z2 / dec(nen))
> ```
> You can also pass a string, and `dec("0.4")/dec(10)` gives the correct
> result as well.
For completeness this is how to do it with mpmath:
>>> from mpmath import mp
>>> mp.dps = 60
>>> mp.mpf('0.4') / 7
mpf('0.0571428571428571428571428571428571428571428571428571428571428549')
You can also use SymPy:
>>> from sympy import Rational
>>> a = Rational('0.4') / 7
>>> a
2/35
>>> a.evalf(60)
0.0571428571428571428571428571428571428571428571428571428571429
SymPy uses mpmath for evalf but it allows doing exact calculations
first and then evaluating the final exact expression to however many
digits are desired at the end which means that you don't need to
accumulate rounding errors before calling evalf.
--
Oscar
--
https://mail.python.org/mailman/listinfo/python-list
Re: Lengthy numbers
On Sun, 22 Dec 2024 at 19:17, Gilmeh Serda via Python-list wrote: > > Was just playing with numbers and stumbled on something I've never seen > before. ... > > >>> 9**9**4 > Traceback (most recent call last): > File "", line 1, in > ValueError: Exceeds the limit (4300 digits) for integer string conversion; > use sys.set_int_max_str_digits() to increase the limit > > Explanation: > https://discuss.python.org/t/int-str-conversions-broken-in-latest-python-bugfix-releases/18889 I think that the original security concern was mainly motivated by the string to int direction i.e. calling int(s) for a possibly large string s (possibly from an untrusted source) might be slow with CPython. To solve that problem conversions from string->int and int->string were disallowed. Now that more time has passed it becomes clearer that disabling int->string conversion is more likely to be the thing that people bump into as a result of this limitation (as you just did). I find it harder to see what the security problem is in that direction but I don't think this will be changed. CPython has an implementation of arbitrarily large integers but an important part of it is hobbled. If you do want to work with such large integers then I recommend using either gmpy2's gmpy2.mpz type or python-flint's flint.fmpz type. At the same time it is not hard to run into slowness with integers e.g. 10**10**10 but that won't come up in string parsing if not using eval. Not a likely security issue but I am suddenly reminded of this dangerous snippet: x = [0]; x.extend(iter(x)) If you want to test it then make sure to save your work etc and be prepared to hard reset the computer. On this machine Ctrl-C doesn't work for this but Ctrl-\ does if you do it quick enough. -- Oscar -- https://mail.python.org/mailman/listinfo/python-list
Re: Pip installs to unexpected place
On Fri, 18 Apr 2025 at 16:50, Peter J. Holzer via Python-list wrote: > > On 2025-04-18 13:24:28 +1200, Greg Ewing via Python-list wrote: > > On 18/04/25 9:41 am, Mats Wichmann wrote: > > > There's just not a really great answer to this. > > > > Seems to me a system-installed application shouldn't be looking in the > > user's .local packages in the first place. That should only be for things > > the user has installed "for this user". > > It's not the application that looks into .local, it's Python. If you say > that a system-installed Python shouldn't look into ~/.local, then you've > just disabled that mechanism completely. If not then Python would > somehow have to distinguish between system-installed and user-installed > scripts. This isn't as easy as checking whether the path starts with > /usr/bin or whether it belongs to root. Tying into the system's package > manager doesn't look appealing to me (not to mention that it might be > unacceptably slow). Couldn't the system-installed scripts have a shebang like: #!/usr/bin/python3 -s -- Oscar -- https://mail.python.org/mailman/listinfo/python-list
