Re: import question
On 11/18/21 21:00, Dan Stromberg wrote: On Thu, Nov 18, 2021 at 6:19 PM Chris Angelico wrote: On Fri, Nov 19, 2021 at 11:24 AM Dan Stromberg wrote: On Thu, Nov 18, 2021 at 12:21 PM Chris Angelico wrote: If you're trying to make a Python-in-Python sandbox, I recommend not. Instead, use an OS-level sandbox (a chroot, probably some sort of CPU usage limiting, etc), and use that to guard the entire Python process. Python-in-Python will basically *never* be secure. Good advice to not try to sandbox python. But chroot can sometimes be broken out of. It isn't a cure-all. That's true, but it's way better than attempting Python-in-Python sandboxing. In any case, all the options worth investigating will be at the OS level. (Or maybe higher, but I can't imagine it being practical to create individual VMs for each client who comes to the web site.) Actually, there are ports of CPython and Micropython that run inside a web browser over WASM. Going with one of these might be safer. indeed... see pyodide https://github.com/pyodide/pyodide -- https://mail.python.org/mailman/listinfo/python-list
Re: get_axes not present?
>And what is the result of plot()? Is it a valid object, or is it None?
Well the error happens on the plot() line. I tried to print some information
like this:
print("axes=", axes)
print("axes[0]=", axes[0])
print("cnt=", cnt)
print("row=", row)
ax1 = row.plot( fontsize=font_size, linewidth=line_width,
markersize=marker_size, marker='o', title='Raw values', label=cnt, ax=axes[0] )
The output looks like
axes= [ ]
axes[0]= AxesSubplot(0.125,0.53;0.775x0.35)
cnt= 1
row= 1 278528
2 278528
3 278528
4 278528
5 278528
...
5604 278528
5605 278528
5606 278528
5607 278528
5608 278528
Name: 4, Length: 5608, dtype: int64
Traceback (most recent call last):
File "process_csv.py", line 178, in
plot_kernels( my_dict2 )
File "process_csv.py", line 66, in plot_kernels
should_plot = plot_dataframe(df, cnt, axes)
File "process_csv.py", line 38, in plot_dataframe
ax1 = row.plot( fontsize=font_size, linewidth=line_width,
markersize=marker_size, marker='o', title='Raw values', label=cnt, ax=axes[0] )
File
"/home/mahmood/.local/lib/python3.8/site-packages/pandas/plotting/_core.py",
line 955, in __call__
return plot_backend.plot(data, kind=kind, **kwargs)
File
"/home/mahmood/.local/lib/python3.8/site-packages/pandas/plotting/_matplotlib/__init__.py",
line 61, in plot
plot_obj.generate()
File
"/home/mahmood/.local/lib/python3.8/site-packages/pandas/plotting/_matplotlib/core.py",
line 283, in generate
self._adorn_subplots()
File
"/home/mahmood/.local/lib/python3.8/site-packages/pandas/plotting/_matplotlib/core.py",
line 483, in _adorn_subplots
all_axes = self._get_subplots()
File
"/home/mahmood/.local/lib/python3.8/site-packages/pandas/plotting/_matplotlib/core.py",
line 903, in _get_subplots
ax for ax in self.axes[0].get_figure().get_axes() if isinstance(ax, Subplot)
AttributeError: 'NoneType' object has no attribute 'get_axes'
The error is weird. I have stick at this error...
Any thoughts on that?
Regards,
Mahmood
--
https://mail.python.org/mailman/listinfo/python-list
pytest segfault, not with -v
I have a battery of tests done with pytest. My tests break with a segfault if I run them normally. If I run them using pytest -v, the segfault does not happen. What could cause this quantical phenomenon? -- https://mail.python.org/mailman/listinfo/python-list
getting source code line of error?
I am trying to get the source code line of the last error.
I know traceback.format_exc() but this contains much more information, e.g.:
Traceback (most recent call last):
File "./error.py", line 18, in main
x=1/0
ZeroDivisionError: division by zero
I could extract the source code line with re.search(), but is there an
easier way?
I have:
exc_type,exc_str,exc_tb = sys.exc_info()
fname = exc_tb.tb_frame.f_code.co_filename
line = exc_tb.tb_lineno
print('%s in %s line %d' % (exc_str,fname,line))
But I also want to output the line itself, not only its number.
--
Ullrich Horlacher Server und Virtualisierung
Rechenzentrum TIK
Universitaet Stuttgart E-Mail: horlac...@tik.uni-stuttgart.de
Allmandring 30aTel:++49-711-68565868
70569 Stuttgart (Germany) WWW:http://www.tik.uni-stuttgart.de/
--
https://mail.python.org/mailman/listinfo/python-list
Re: Unexpected behaviour of math.floor, round and int functions (rounding)
Le 19/11/2021 à 03:51, MRAB a écrit : On 2021-11-19 02:40, 2qdxy4rzwzuui...@potatochowder.com wrote: On 2021-11-18 at 23:16:32 -0300, René Silva Valdés wrote: Hello, I would like to report the following issue: Working with floats i noticed that: int(23.99/12) returns 1, and int(23.999/12) returns 2 This implies that int() function is rounding ... It's not int() that's doing the rounding; that second numerator is being rounded before being divided by 12: Python 3.9.7 (default, Oct 10 2021, 15:13:22) [GCC 11.1.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> 23.999 24.0 >>> (23.999).hex() '0x1.8p+4' I think this is a bit clearer because it shows that it's not just being rounded for display: Python 3.10.0 (tags/v3.10.0:b494f59, Oct 4 2021, 19:00:18) [MSC v.1929 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> 23.99 == 24 False >>> 23.999 == 24 True >>> 0.3 + 0.3 + 0.3 == 0.9 False -- https://mail.python.org/mailman/listinfo/python-list
Re: Unexpected behaviour of math.floor, round and int functions (rounding)
Le 19/11/2021 à 12:43, ast a écrit : Le 19/11/2021 à 03:51, MRAB a écrit : On 2021-11-19 02:40, 2qdxy4rzwzuui...@potatochowder.com wrote: On 2021-11-18 at 23:16:32 -0300, René Silva Valdés wrote: Hello, I would like to report the following issue: Working with floats i noticed that: int(23.99/12) returns 1, and int(23.999/12) returns 2 This implies that int() function is rounding ... It's not int() that's doing the rounding; that second numerator is being rounded before being divided by 12: Python 3.9.7 (default, Oct 10 2021, 15:13:22) [GCC 11.1.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> 23.999 24.0 >>> (23.999).hex() '0x1.8p+4' I think this is a bit clearer because it shows that it's not just being rounded for display: Python 3.10.0 (tags/v3.10.0:b494f59, Oct 4 2021, 19:00:18) [MSC v.1929 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> 23.99 == 24 False >>> 23.999 == 24 True >>> 0.3 + 0.3 + 0.3 == 0.9 False Better use math.isclose to test equality between 2 floats >>> import math >>> math.isclose(0.3 + 0.3 + 0.3, 0.9) True -- https://mail.python.org/mailman/listinfo/python-list
Re: import question
ok. all good advice. thank you for that. and with all that I've decided what to do. I'm going to close off any server-side python access so that I don't expose my server or the file system to vulnerabilities and/or wonton attacks. I am building a site for education and what I will configure is allow students to setup and save their projects on the server but only direct them to program in client-side Brython, which is a javascript translation of python for browsers, hence "Brython" or "browser python". my server will provide the javascript files for Brython and its standard libraries and any processing of the student's projects will be directly on the client-side. this way there is no access to the server or cpu or memory management problems. the server will simply server html and Brython-based text, i.e., static pages, to the client browser and the browser will process and interact with the Brython directly. overall, the server will stay secure and the students can learn python through Brython. sound, right? Lucas -- https://mail.python.org/mailman/listinfo/python-list
Re: getting source code line of error?
Have you tried the logger module and the format options?
On Fri, 19 Nov 2021 at 19:09, Ulli Horlacher
wrote:
>
> I am trying to get the source code line of the last error.
> I know traceback.format_exc() but this contains much more information, e.g.:
>
> Traceback (most recent call last):
> File "./error.py", line 18, in main
> x=1/0
> ZeroDivisionError: division by zero
>
> I could extract the source code line with re.search(), but is there an
> easier way?
>
>
> I have:
>
> exc_type,exc_str,exc_tb = sys.exc_info()
> fname = exc_tb.tb_frame.f_code.co_filename
> line = exc_tb.tb_lineno
> print('%s in %s line %d' % (exc_str,fname,line))
>
> But I also want to output the line itself, not only its number.
>
> --
> Ullrich Horlacher Server und Virtualisierung
> Rechenzentrum TIK
> Universitaet Stuttgart E-Mail: horlac...@tik.uni-stuttgart.de
> Allmandring 30aTel:++49-711-68565868
> 70569 Stuttgart (Germany) WWW:http://www.tik.uni-stuttgart.de/
> --
> https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: pytest segfault, not with -v
On 2021-11-19 17:48, Marco Sulla wrote: I have a battery of tests done with pytest. My tests break with a segfault if I run them normally. If I run them using pytest -v, the segfault does not happen. What could cause this quantical phenomenon? Are you testing an extension that you're compiling? That kind of problem can occur if there's an uninitialised variable or incorrect reference counting (Py_INCREF/Py_DECREF). -- https://mail.python.org/mailman/listinfo/python-list
Re: import question
On 20/11/2021 03.38, lucas wrote: > ok. all good advice. thank you for that. and with all that I've decided > what to do. > > I'm going to close off any server-side python access so that I don't expose > my server or the file system to vulnerabilities and/or wonton attacks. I am > building a site for education and what I will configure is allow students to > setup and save their projects on the server but only direct them to program > in client-side Brython, which is a javascript translation of python for > browsers, hence "Brython" or "browser python". my server will provide the > javascript files for Brython and its standard libraries and any processing of > the student's projects will be directly on the client-side. this way there > is no access to the server or cpu or memory management problems. the server > will simply server html and Brython-based text, i.e., static pages, to the > client browser and the browser will process and interact with the Brython > directly. > > overall, the server will stay secure and the students can learn python > through Brython. sound, right? Lucas Alternately, 'stand on the shoulders of giants' and consider https://pythontutor.com/visualize.html#mode=edit This has the additional value for your trainees of showing a visual execution of their code. They can see step-by-step how Python/the computer interprets their (?perfect) instruction and exactly where things fall-over - with-out the added complication/cognitive-load of having to master a debugger! If you're still determined to invest a lot of time, it looks as if Phil has invested a lot of time, more recently, in widening the range of languages, (which is perhaps why(?)) the system has fallen behind in Python release. -- Regards, =dn -- https://mail.python.org/mailman/listinfo/python-list
Re: Unexpected behaviour of math.floor, round and int functions (rounding)
On 11/18/21 19:40, 2qdxy4rzwzuui...@potatochowder.com wrote: On 2021-11-18 at 23:16:32 -0300, René Silva Valdés wrote: Hello, I would like to report the following issue: Working with floats i noticed that: int(23.99/12) returns 1, and int(23.999/12) returns 2 This implies that int() function is rounding ... It's not int() that's doing the rounding; that second numerator is being rounded before being divided by 12: Python 3.9.7 (default, Oct 10 2021, 15:13:22) [GCC 11.1.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> 23.999 24.0 >>> (23.999).hex() '0x1.8p+4' The documentation has a fair bit to say on the subject of floating point. I never remember the precise link, but fortunately it's pretty easy to search for: https://docs.python.org/3/tutorial/floatingpoint.html -- https://mail.python.org/mailman/listinfo/python-list
Re: Unexpected behaviour of math.floor, round and int functions (rounding)
On Sat, Nov 20, 2021 at 5:08 AM ast wrote: > > Le 19/11/2021 à 03:51, MRAB a écrit : > > On 2021-11-19 02:40, 2qdxy4rzwzuui...@potatochowder.com wrote: > >> On 2021-11-18 at 23:16:32 -0300, > >> René Silva Valdés wrote: > >> > >>> Hello, I would like to report the following issue: > >>> > >>> Working with floats i noticed that: > >>> > >>> int(23.99/12) returns 1, and > >>> int(23.999/12) returns 2 > >>> > >>> This implies that int() function is rounding ... > >> > >> It's not int() that's doing the rounding; that second numerator is being > >> rounded before being divided by 12: > >> > >> Python 3.9.7 (default, Oct 10 2021, 15:13:22) > >> [GCC 11.1.0] on linux > >> Type "help", "copyright", "credits" or "license" for more > >> information. > >> >>> 23.999 > >> 24.0 > >> >>> (23.999).hex() > >> '0x1.8p+4' > >> > > I think this is a bit clearer because it shows that it's not just being > > rounded for display: > > > > Python 3.10.0 (tags/v3.10.0:b494f59, Oct 4 2021, 19:00:18) [MSC v.1929 > > 64 bit (AMD64)] on win32 > > Type "help", "copyright", "credits" or "license" for more information. > > >>> 23.99 == 24 > > False > > >>> 23.999 == 24 > > True > > >>> 0.3 + 0.3 + 0.3 == 0.9 > False That's because 0.3 is not 3/10. It's not because floats are "unreliable" or "inaccurate". It's because the ones you're entering are not what you think they are. When will people understand this? (Probably never. Sigh.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Unexpected behaviour of math.floor, round and int functions (rounding)
On 20/11/2021 09.17, Chris Angelico wrote: > On Sat, Nov 20, 2021 at 5:08 AM ast wrote: >> Le 19/11/2021 à 03:51, MRAB a écrit : >>> On 2021-11-19 02:40, 2qdxy4rzwzuui...@potatochowder.com wrote: On 2021-11-18 at 23:16:32 -0300, René Silva Valdés wrote: > Working with floats i noticed that: > int(23.99/12) returns 1, and > int(23.999/12) returns 2 Has the OP (now) realised that the observation is a "feature" not a "bug"? It is one of the difficulties of representing small numbers or numerical-components in binary - there are many decimal values which cannot be accurately-expressed in binary - exactly as noted. ... >> >>> 0.3 + 0.3 + 0.3 == 0.9 >> False > > That's because 0.3 is not 3/10. It's not because floats are > "unreliable" or "inaccurate". It's because the ones you're entering > are not what you think they are. > > When will people understand this? > (Probably never. Sigh.) Am not aware of any institution which teaches the inner-workings of a CPU/ALU/FPU/GPU in a general programming class, ie "Programming" and particularly "Coding", have diverged from "Computer Science" - in at least this respect. As well as the approximations involved in trying to maintain decimal-numbers (floats/floating-point numbers), and particularly values to the 'right' of a decimal-point; we had to study?suffer classes in "Numerical Analysis" and be able to gauge the declining accuracy and precision of sundry calculations. A skill disappearing as fast as slide-rules!? This 'pool of ignorance' is particularly noticeable in folk who have come 'up' through the 'CodeCamp'/'BootCamp' approach to training. On the other hand, if one is not intending to 'get into' a scientific or highly mathematical branch of computing/Python, eg commercial applications using (only) Decimal (or int), the average web-app, and similar; why bother? -- Regards, =dn -- https://mail.python.org/mailman/listinfo/python-list
frozenset can be altered by |=
(venv_3_10) marco@buzz:~$ python
Python 3.10.0 (heads/3.10-dirty:f6e8b80d20, Nov 18 2021, 19:16:18)
[GCC 10.1.1 20200718] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> a = frozenset((3, 4))
>>> a
frozenset({3, 4})
>>> a |= {5,}
>>> a
frozenset({3, 4, 5})
--
https://mail.python.org/mailman/listinfo/python-list
Re: frozenset can be altered by |=
On Sat, Nov 20, 2021 at 8:13 AM Marco Sulla
wrote:
>
> (venv_3_10) marco@buzz:~$ python
> Python 3.10.0 (heads/3.10-dirty:f6e8b80d20, Nov 18 2021, 19:16:18)
> [GCC 10.1.1 20200718] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> a = frozenset((3, 4))
> >>> a
> frozenset({3, 4})
> >>> a |= {5,}
> >>> a
> frozenset({3, 4, 5})
That's the same as how "x = 4; x += 1" can "alter" four into five.
>>> a = frozenset((3, 4))
>>> id(a), a
(140545764976096, frozenset({3, 4}))
>>> a |= {5,}
>>> id(a), a
(140545763014944, frozenset({3, 4, 5}))
It's a different frozenset.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: frozenset can be altered by |=
On Sat, Nov 20, 2021 at 8:16 AM Chris Angelico wrote:
>
> On Sat, Nov 20, 2021 at 8:13 AM Marco Sulla
> wrote:
> >
> > (venv_3_10) marco@buzz:~$ python
> > Python 3.10.0 (heads/3.10-dirty:f6e8b80d20, Nov 18 2021, 19:16:18)
> > [GCC 10.1.1 20200718] on linux
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>> a = frozenset((3, 4))
> > >>> a
> > frozenset({3, 4})
> > >>> a |= {5,}
> > >>> a
> > frozenset({3, 4, 5})
>
> That's the same as how "x = 4; x += 1" can "alter" four into five.
>
> >>> a = frozenset((3, 4))
> >>> id(a), a
> (140545764976096, frozenset({3, 4}))
> >>> a |= {5,}
> >>> id(a), a
> (140545763014944, frozenset({3, 4, 5}))
>
> It's a different frozenset.
>
Oh, even better test:
>>> a = frozenset((3, 4)); b = a
>>> id(a), a, id(b), b
(140602825123296, frozenset({3, 4}), 140602825123296, frozenset({3, 4}))
>>> a |= {5,}
>>> id(a), a, id(b), b
(140602825254144, frozenset({3, 4, 5}), 140602825123296, frozenset({3, 4}))
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Unexpected behaviour of math.floor, round and int functions (rounding)
On Sat, Nov 20, 2021 at 7:39 AM dn via Python-list wrote: > >> >>> 0.3 + 0.3 + 0.3 == 0.9 > >> False > > > > That's because 0.3 is not 3/10. It's not because floats are > > "unreliable" or "inaccurate". It's because the ones you're entering > > are not what you think they are. > > > > When will people understand this? > > (Probably never. Sigh.) > > > Am not aware of any institution which teaches the inner-workings of a > CPU/ALU/FPU/GPU in a general programming class, ie "Programming" and > particularly "Coding", have diverged from "Computer Science" - in at > least this respect. > I think what I find annoying about this sort of thing is that people triumphantly announce that the computer is WRONG. It's the numeric equivalent of XKCD 169, and people get smug for the exact same reason, and perhaps unfortunately, do not get their arms cut off. "Computer, give me a number as close as possible to three tenths." "HAH! THAT ISN'T THREE TENTHS! Hah you suck!" *slash* ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: frozenset can be altered by |=
Mh. Now I'm thinking that I've done
a = "Marco "
a += "Sulla"
many times without bothering.
On Fri, 19 Nov 2021 at 22:22, Chris Angelico wrote:
>
> On Sat, Nov 20, 2021 at 8:16 AM Chris Angelico wrote:
> >
> > On Sat, Nov 20, 2021 at 8:13 AM Marco Sulla
> > wrote:
> > >
> > > (venv_3_10) marco@buzz:~$ python
> > > Python 3.10.0 (heads/3.10-dirty:f6e8b80d20, Nov 18 2021, 19:16:18)
> > > [GCC 10.1.1 20200718] on linux
> > > Type "help", "copyright", "credits" or "license" for more information.
> > > >>> a = frozenset((3, 4))
> > > >>> a
> > > frozenset({3, 4})
> > > >>> a |= {5,}
> > > >>> a
> > > frozenset({3, 4, 5})
> >
> > That's the same as how "x = 4; x += 1" can "alter" four into five.
> >
> > >>> a = frozenset((3, 4))
> > >>> id(a), a
> > (140545764976096, frozenset({3, 4}))
> > >>> a |= {5,}
> > >>> id(a), a
> > (140545763014944, frozenset({3, 4, 5}))
> >
> > It's a different frozenset.
> >
>
> Oh, even better test:
>
> >>> a = frozenset((3, 4)); b = a
> >>> id(a), a, id(b), b
> (140602825123296, frozenset({3, 4}), 140602825123296, frozenset({3, 4}))
> >>> a |= {5,}
> >>> id(a), a, id(b), b
> (140602825254144, frozenset({3, 4, 5}), 140602825123296, frozenset({3, 4}))
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: frozenset can be altered by |=
On 2021-11-19 21:11, Marco Sulla wrote:
(venv_3_10) marco@buzz:~$ python
Python 3.10.0 (heads/3.10-dirty:f6e8b80d20, Nov 18 2021, 19:16:18)
[GCC 10.1.1 20200718] on linux
Type "help", "copyright", "credits" or "license" for more information.
a = frozenset((3, 4))
a
frozenset({3, 4})
a |= {5,}
a
frozenset({3, 4, 5})
I'll counter with:
Python 3.10.0 (tags/v3.10.0:b494f59, Oct 4 2021, 19:00:18) [MSC v.1929
64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> a = frozenset((3, 4))
>>> a
frozenset({3, 4})
>>> b = a
>>> a |= {5,}
>>> a
frozenset({3, 4, 5})
>>> b
frozenset({3, 4})
frozenset doesn't support mutation, so:
a |= {5,}
falls back to:
a = a | {5,}
The same kind of thing happens with tuples:
>>> a = (3, 4)
>>> a
(3, 4)
>>> a += (5,)
>>> a
(3, 4, 5)
--
https://mail.python.org/mailman/listinfo/python-list
Re: Unexpected behaviour of math.floor, round and int functions (rounding)
Chris Angelico writes: > On Sat, Nov 20, 2021 at 5:08 AM ast wrote: >> >>> 0.3 + 0.3 + 0.3 == 0.9 >> False > > That's because 0.3 is not 3/10. It's not because floats are > "unreliable" or "inaccurate". It's because the ones you're entering > are not what you think they are. > > When will people understand this? > > (Probably never. Sigh.) Most people understand what's going on when it's explained to them. And I think that being initially baffled is not unreasonable. After all, almost everyone comes to computers after learning that 3/10 can be written as 0.3. And Python "plays along" with the fiction to some extent. 0.3 prints as 0.3, 3/10 prints as 0.3 and 0.3 == 3/10 is True. The language (a language, not Python) could tell you that you were not getting the value you asked for. Every 0.3 could come with a warning that 0.3 can not be represented exactly as a floating point value. To avoid the warning, the programmer would write ~0.3 meaning, exactly, the binary (or whatever the base really is) floating point number closest to 0.3. -- Ben. -- https://mail.python.org/mailman/listinfo/python-list
Re: Unexpected behaviour of math.floor, round and int functions (rounding)
On Sat, Nov 20, 2021 at 9:07 AM Ben Bacarisse wrote:
>
> Chris Angelico writes:
>
> > On Sat, Nov 20, 2021 at 5:08 AM ast wrote:
>
> >> >>> 0.3 + 0.3 + 0.3 == 0.9
> >> False
> >
> > That's because 0.3 is not 3/10. It's not because floats are
> > "unreliable" or "inaccurate". It's because the ones you're entering
> > are not what you think they are.
> >
> > When will people understand this?
> >
> > (Probably never. Sigh.)
>
> Most people understand what's going on when it's explained to them. And
> I think that being initially baffled is not unreasonable. After all,
> almost everyone comes to computers after learning that 3/10 can be
> written as 0.3. And Python "plays along" with the fiction to some
> extent. 0.3 prints as 0.3, 3/10 prints as 0.3 and 0.3 == 3/10 is True.
In grade school, we learn that not everything can be written that way,
and 1/3 isn't actually equal to 0.33. Yet somehow people
understand that computers speak binary ("have you learned to count
yet, or are you still on zeroes and ones?" -- insult to a machine
empire, in Stellaris), but don't seem to appreciate that floats are
absolutely accurate and reliable, just in binary.
But lack of knowledge is never a problem. (Or rather, it's a solvable
problem, and I'm always happy to explain things to people.) The
problem is when, following that lack of understanding, people assume
that floats are "unreliable" or "inaccurate", and that you should
never ever compare two floats for equality, because they're never
"really equal". That's what leads to horrible coding practices and
badly-defined "approximately equal" checks that cause far more harm
than a simple misunderstanding ever could on its own.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Unexpected behaviour of math.floor, round and int functions (rounding)
On 20/11/2021 10.21, Chris Angelico wrote:
> On Sat, Nov 20, 2021 at 7:39 AM dn via Python-list
> wrote:
>>> 0.3 + 0.3 + 0.3 == 0.9
False
>>>
>>> That's because 0.3 is not 3/10. It's not because floats are
>>> "unreliable" or "inaccurate". It's because the ones you're entering
>>> are not what you think they are.
>>>
>>> When will people understand this?
>>> (Probably never. Sigh.)
>>
>>
>> Am not aware of any institution which teaches the inner-workings of a
>> CPU/ALU/FPU/GPU in a general programming class, ie "Programming" and
>> particularly "Coding", have diverged from "Computer Science" - in at
>> least this respect.
>>
>
> I think what I find annoying about this sort of thing is that people
> triumphantly announce that the computer is WRONG. It's the numeric
> equivalent of XKCD 169, and people get smug for the exact same reason,
> and perhaps unfortunately, do not get their arms cut off.
I guess I'm a little more familiar with the 'arrogant' side of this
phenomenon because in formal courses, eg uni, I'd predict at least one
such personality per group of 50~60 students!
However, cultural-components should be considered. There is no
requirement that the OP possess an advanced command of the English
language! (nor of Python, nor of computers/computing, ...) - see OP's name!
That said, it can be frustrating for those of us who see particular
problems over-and-over. If it were evidenced in a course, I would be
addressing a short-coming in the training materials - but there is no
one Python-course to suit us all...
Another behavior is to assume that because 'I' learned something
years-ago, so should 'everyone else'. Even superficial reflection
reveals that this just isn't true - particularly when we have an
education system which is largely based upon 'date of manufacture'!
(just because I'm older than you doesn't make me 'smarter' - although I
am definitely better-looking!)
The problem is not limited to the limitations of (hah!) floating-point
arithmetic. We've seen other questions 'here' and on "Tutor", which
illustrate lack(s) of understanding of basic relationships between
Python (software) and the hardware which runs its instructions, eg the
basic simplicity of using a 'paper computer' for debugging, the order of
precedence, that the RHS must happen before we can consider anything on
the LHS, ... I've met 'graduates' who have never even peered 'under the
hood' of a computer!
(Incidentally, I recently recommended against employing one such
otherwise extremely promising ("on paper" and 'in person') candidate.
Quizzed by A.N.Other on the panel (who was suitably-impressed), the
justification was an apparent total lack of curiosity - a personality
essential for problem-solvers! (IMHO)
Another one of my 'pet-peeves' (one of the many! Although, perhaps not
to the extent of chopping-off people's arms - no matter how Monty
Python-ic that might be, Sir Knight; is the lack of
spelling/typing/proof-reading skills evidenced by many. Criticising a
(school) teacher of my acquaintance for exactly this, I was bluntly told
"but you knew what I meant" and thus he felt "communication" had been
achieved. Um, yes, er, quite true - but at what effort, and who's effort?
I find this phenomenon in computer-people particularly fascinating,
given that Python (insert any other language-name here) is very
particular and finds "far" instead of "for" far-from acceptable (hah!
again!). Does this mean that their coding-technique is to
(almost-literally) throw stuff at Python and have the computer 'find'
all the typos and spelling-errors? (and if so, is this good use of
time/good technique?)
Thus, and on this, the members of the aforementioned panel had agreed
without discussion: any documents forwarded by a candidate which
contained spelling mistakes were 'binned' (trashed) without the usual
acceptance/tolerance/mercy one would apply to work-documents.
Training/education takes time and costs money. Accordingly, we have this
constant 'battle' between wanting to educate/acquire knowledge versus
the cost-benefit of some fact/skill which may never/rarely be used - and
the 'factory component' of choosing what is good for a group/for 'the
average trainee' cf what is so for the individual.
Entries on the back of a post-card...
--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list
Re: pytest segfault, not with -v
On Fri, 19 Nov 2021 at 20:38, MRAB wrote: > > On 2021-11-19 17:48, Marco Sulla wrote: > > I have a battery of tests done with pytest. My tests break with a > > segfault if I run them normally. If I run them using pytest -v, the > > segfault does not happen. > > > > What could cause this quantical phenomenon? > > > Are you testing an extension that you're compiling? That kind of problem > can occur if there's an uninitialised variable or incorrect reference > counting (Py_INCREF/Py_DECREF). Ok, I know. But why can't it be reproduced if I do pytest -v? This way I don't know which test fails. Furthermore I noticed that if I remove the __pycache__ dir of tests, pytest does not crash, until I re-ran it with the __pycache__ dir present. This way is very hard for me to understand what caused the segfault. I'm starting to think pytest is not good for testing C extensions. -- https://mail.python.org/mailman/listinfo/python-list
Re: pytest segfault, not with -v
On 2021-11-19 23:44, Marco Sulla wrote: On Fri, 19 Nov 2021 at 20:38, MRAB wrote: On 2021-11-19 17:48, Marco Sulla wrote: > I have a battery of tests done with pytest. My tests break with a > segfault if I run them normally. If I run them using pytest -v, the > segfault does not happen. > > What could cause this quantical phenomenon? > Are you testing an extension that you're compiling? That kind of problem can occur if there's an uninitialised variable or incorrect reference counting (Py_INCREF/Py_DECREF). Ok, I know. But why can't it be reproduced if I do pytest -v? This way I don't know which test fails. Furthermore I noticed that if I remove the __pycache__ dir of tests, pytest does not crash, until I re-ran it with the __pycache__ dir present. This way is very hard for me to understand what caused the segfault. I'm starting to think pytest is not good for testing C extensions. If there are too few Py_INCREF or too many Py_DECREF, it'll free the object too soon, and whether or when that will cause a segfault will depend on whatever other code is running. That's the nature of the beast: it's unpredictable! You could try running each of the tests in a loop to see which one causes a segfault. (Trying several in a loop will let you narrow it down more quickly.) pytest et al. are good for testing behaviour, but not for narrowing down segfaults. -- https://mail.python.org/mailman/listinfo/python-list
Re: Unexpected behaviour of math.floor, round and int functions (rounding)
Chris Angelico writes:
> On Sat, Nov 20, 2021 at 9:07 AM Ben Bacarisse wrote:
>>
>> Chris Angelico writes:
>>
>> > On Sat, Nov 20, 2021 at 5:08 AM ast wrote:
>>
>> >> >>> 0.3 + 0.3 + 0.3 == 0.9
>> >> False
>> >
>> > That's because 0.3 is not 3/10. It's not because floats are
>> > "unreliable" or "inaccurate". It's because the ones you're entering
>> > are not what you think they are.
>> >
>> > When will people understand this?
>> >
>> > (Probably never. Sigh.)
>>
>> Most people understand what's going on when it's explained to them. And
>> I think that being initially baffled is not unreasonable. After all,
>> almost everyone comes to computers after learning that 3/10 can be
>> written as 0.3. And Python "plays along" with the fiction to some
>> extent. 0.3 prints as 0.3, 3/10 prints as 0.3 and 0.3 == 3/10 is True.
>
> In grade school, we learn that not everything can be written that way,
> and 1/3 isn't actually equal to 0.33.
Yes. We learn early on that 0.33 means 33/100.
We don't learn that 0.33 is a special notation for machines that
have something called "binary floating point hardware" that does not
mean 33/100. That has to be learned later. And every
generation has to learn it afresh.
> Yet somehow people
> understand that computers speak binary ("have you learned to count
> yet, or are you still on zeroes and ones?" -- insult to a machine
> empire, in Stellaris), but don't seem to appreciate that floats are
> absolutely accurate and reliable, just in binary.
Yes, agreed, but I was not commenting on the odd (and incorrect) view
that floating point operations are not reliable and well-defined, but on
the reasonable assumption that a clever programming language might take
0.3 to mean what I was taught it meant in grade school.
As an old hand, I know it won't (in most languages), and I know why it
won't, and I know why that's usually the right design choice for the
language. But I can also appreciate that it's by no means obvious that,
to a beginner, "binary" implies the particular kind of representation
that makes 0.3 not mean 3/10. After all, the rational 3/10 can be
represented exactly in binary in many different ways.
> But lack of knowledge is never a problem. (Or rather, it's a solvable
> problem, and I'm always happy to explain things to people.) The
> problem is when, following that lack of understanding, people assume
> that floats are "unreliable" or "inaccurate", and that you should
> never ever compare two floats for equality, because they're never
> "really equal". That's what leads to horrible coding practices and
> badly-defined "approximately equal" checks that cause far more harm
> than a simple misunderstanding ever could on its own.
Agreed. Often, the "explanations" just make things worse.
--
Ben.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Unexpected behaviour of math.floor, round and int functions (rounding)
On Sat, Nov 20, 2021 at 12:43 PM Ben Bacarisse wrote: > > Chris Angelico writes: > > > On Sat, Nov 20, 2021 at 9:07 AM Ben Bacarisse wrote: > >> > >> Chris Angelico writes: > >> > >> > On Sat, Nov 20, 2021 at 5:08 AM ast wrote: > >> > >> >> >>> 0.3 + 0.3 + 0.3 == 0.9 > >> >> False > >> > > >> > That's because 0.3 is not 3/10. It's not because floats are > >> > "unreliable" or "inaccurate". It's because the ones you're entering > >> > are not what you think they are. > >> > > >> > When will people understand this? > >> > > >> > (Probably never. Sigh.) > >> > >> Most people understand what's going on when it's explained to them. And > >> I think that being initially baffled is not unreasonable. After all, > >> almost everyone comes to computers after learning that 3/10 can be > >> written as 0.3. And Python "plays along" with the fiction to some > >> extent. 0.3 prints as 0.3, 3/10 prints as 0.3 and 0.3 == 3/10 is True. > > > > In grade school, we learn that not everything can be written that way, > > and 1/3 isn't actually equal to 0.33. > > Yes. We learn early on that 0.33 means 33/100. > We don't learn that 0.33 is a special notation for machines that > have something called "binary floating point hardware" that does not > mean 33/100. That has to be learned later. And every > generation has to learn it afresh. But you learn that it isn't the same as 1/3. That's my point. You already understand that it is *impossible* to write out 1/3 in decimal. Is it such a stretch to discover that you cannot write 3/10 in binary? Every generation has to learn about repeating fractions, but most of us learn them in grade school. Every generation learns that computers talk in binary. Yet, putting those two concepts together seems beyond many people, to the point that they feel that floating point can't be trusted. > Yes, agreed, but I was not commenting on the odd (and incorrect) view > that floating point operations are not reliable and well-defined, but on > the reasonable assumption that a clever programming language might take > 0.3 to mean what I was taught it meant in grade school. It does mean exactly what it meant in grade school, just as 1/3 means exactly what it meant in grade school. Now try to represent 1/3 on a blackboard, as a decimal fraction. If that's impossible, does it mean that 1/3 doesn't mean 1/3, or that 1/3 can't be represented? > > But lack of knowledge is never a problem. (Or rather, it's a solvable > > problem, and I'm always happy to explain things to people.) The > > problem is when, following that lack of understanding, people assume > > that floats are "unreliable" or "inaccurate", and that you should > > never ever compare two floats for equality, because they're never > > "really equal". That's what leads to horrible coding practices and > > badly-defined "approximately equal" checks that cause far more harm > > than a simple misunderstanding ever could on its own. > > Agreed. Often, the "explanations" just make things worse. > When they're based on a fear of floats, yes. Explanations like "never use == with floats because 0.1+0.2!=0.3" are worse than useless, because they create that fear in a way that creates awful cargo-cult programming practices. If someone does something in Python, gets a weird result, and comes to the list saying "I don't understand this", that's not a problem. We get it all the time with mutables. Recent question about frozensets appearing to be mutable, same thing. I have no problem with someone humbly asking "what's happening?", based on an internal assumption that there's a reason things are the way they are. For some reason, floats don't get that same respect from many people. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Unexpected behaviour of math.floor, round and int functions (rounding)
Chris Angelico writes: > On Sat, Nov 20, 2021 at 12:43 PM Ben Bacarisse wrote: >> >> Chris Angelico writes: >> >> > On Sat, Nov 20, 2021 at 9:07 AM Ben Bacarisse wrote: >> >> >> >> Chris Angelico writes: >> >> >> >> > On Sat, Nov 20, 2021 at 5:08 AM ast wrote: >> >> >> >> >> >>> 0.3 + 0.3 + 0.3 == 0.9 >> >> >> False >> >> > >> >> > That's because 0.3 is not 3/10. It's not because floats are >> >> > "unreliable" or "inaccurate". It's because the ones you're entering >> >> > are not what you think they are. >> >> > >> >> > When will people understand this? >> >> > >> >> > (Probably never. Sigh.) >> >> >> >> Most people understand what's going on when it's explained to them. And >> >> I think that being initially baffled is not unreasonable. After all, >> >> almost everyone comes to computers after learning that 3/10 can be >> >> written as 0.3. And Python "plays along" with the fiction to some >> >> extent. 0.3 prints as 0.3, 3/10 prints as 0.3 and 0.3 == 3/10 is True. >> > >> > In grade school, we learn that not everything can be written that way, >> > and 1/3 isn't actually equal to 0.33. >> >> Yes. We learn early on that 0.33 means 33/100. >> We don't learn that 0.33 is a special notation for machines that >> have something called "binary floating point hardware" that does not >> mean 33/100. That has to be learned later. And every >> generation has to learn it afresh. > > But you learn that it isn't the same as 1/3. That's my point. You > already understand that it is *impossible* to write out 1/3 in > decimal. Is it such a stretch to discover that you cannot write 3/10 > in binary? > > Every generation has to learn about repeating fractions, but most of > us learn them in grade school. Every generation learns that computers > talk in binary. Yet, putting those two concepts together seems beyond > many people, to the point that they feel that floating point can't be > trusted. Binary is a bit of a red herring here. It's the floating point format that needs to be understood. Three tenths can be represented in many binary formats, and even decimal floating point will have some surprises for the novice. >> Yes, agreed, but I was not commenting on the odd (and incorrect) view >> that floating point operations are not reliable and well-defined, but on >> the reasonable assumption that a clever programming language might take >> 0.3 to mean what I was taught it meant in grade school. > > It does mean exactly what it meant in grade school, just as 1/3 means > exactly what it meant in grade school. Now try to represent 1/3 on a > blackboard, as a decimal fraction. If that's impossible, does it mean > that 1/3 doesn't mean 1/3, or that 1/3 can't be represented? As you know, it is possible, but let's say we outlaw any finite notation for repeated digits... Why should I convert 1/3 to this particular apparently unsuitable representation? I will write 1/3 and manipulate that number using factional notation. The novice programmer might similarly expect that when they write 0.3, the program will manipulate that number as the faction it clearly is. They may well be surprised by the fact that it must get put into a format that can't represent what those three characters mean, just as I would be surprised if you insisted I write 1/3 as a finite decimal (with no repeat notation). I'm not saying your analogy would not help someone understand, but you first have to explain why 0.3 is not treated as three tenths -- why I (to use your analogy) must not keep 1/3 as a proper fraction, but I must instead write it using a finite number of decimal digits. Neither is, in my view, obvious to the beginner. >> > But lack of knowledge is never a problem. (Or rather, it's a solvable >> > problem, and I'm always happy to explain things to people.) The >> > problem is when, following that lack of understanding, people assume >> > that floats are "unreliable" or "inaccurate", and that you should >> > never ever compare two floats for equality, because they're never >> > "really equal". That's what leads to horrible coding practices and >> > badly-defined "approximately equal" checks that cause far more harm >> > than a simple misunderstanding ever could on its own. >> >> Agreed. Often, the "explanations" just make things worse. > > When they're based on a fear of floats, yes. Explanations like "never > use == with floats because 0.1+0.2!=0.3" are worse than useless, > because they create that fear in a way that creates awful cargo-cult > programming practices. > > If someone does something in Python, gets a weird result, and comes to > the list saying "I don't understand this", that's not a problem. We > get it all the time with mutables. Recent question about frozensets > appearing to be mutable, same thing. I have no problem with someone > humbly asking "what's happening?", based on an internal assumption > that there's a reason things are the way they are
Re: Unexpected behaviour of math.floor, round and int functions (rounding)
On Sat, Nov 20, 2021 at 3:41 PM Ben Bacarisse wrote: > > Chris Angelico writes: > > > On Sat, Nov 20, 2021 at 12:43 PM Ben Bacarisse wrote: > >> > >> Chris Angelico writes: > >> > >> > On Sat, Nov 20, 2021 at 9:07 AM Ben Bacarisse > >> > wrote: > >> >> > >> >> Chris Angelico writes: > >> >> > >> >> > On Sat, Nov 20, 2021 at 5:08 AM ast wrote: > >> >> > >> >> >> >>> 0.3 + 0.3 + 0.3 == 0.9 > >> >> >> False > >> >> > > >> >> > That's because 0.3 is not 3/10. It's not because floats are > >> >> > "unreliable" or "inaccurate". It's because the ones you're entering > >> >> > are not what you think they are. > >> >> > > >> >> > When will people understand this? > >> >> > > >> >> > (Probably never. Sigh.) > >> >> > >> >> Most people understand what's going on when it's explained to them. And > >> >> I think that being initially baffled is not unreasonable. After all, > >> >> almost everyone comes to computers after learning that 3/10 can be > >> >> written as 0.3. And Python "plays along" with the fiction to some > >> >> extent. 0.3 prints as 0.3, 3/10 prints as 0.3 and 0.3 == 3/10 is True. > >> > > >> > In grade school, we learn that not everything can be written that way, > >> > and 1/3 isn't actually equal to 0.33. > >> > >> Yes. We learn early on that 0.33 means 33/100. > >> We don't learn that 0.33 is a special notation for machines that > >> have something called "binary floating point hardware" that does not > >> mean 33/100. That has to be learned later. And every > >> generation has to learn it afresh. > > > > But you learn that it isn't the same as 1/3. That's my point. You > > already understand that it is *impossible* to write out 1/3 in > > decimal. Is it such a stretch to discover that you cannot write 3/10 > > in binary? > > > > Every generation has to learn about repeating fractions, but most of > > us learn them in grade school. Every generation learns that computers > > talk in binary. Yet, putting those two concepts together seems beyond > > many people, to the point that they feel that floating point can't be > > trusted. > > Binary is a bit of a red herring here. It's the floating point format > that needs to be understood. Three tenths can be represented in many > binary formats, and even decimal floating point will have some surprises > for the novice. Not completely a red herring; binary floating-point as used in Python (IEEE double-precision) is defined as a binary mantissa and a scale, just as "blackboard arithmetic" is generally defined as a decimal mantissa and a scale. (At least, I don't think I've ever seen anyone doing arithmetic on a blackboard in hex or octal.) > >> Yes, agreed, but I was not commenting on the odd (and incorrect) view > >> that floating point operations are not reliable and well-defined, but on > >> the reasonable assumption that a clever programming language might take > >> 0.3 to mean what I was taught it meant in grade school. > > > > It does mean exactly what it meant in grade school, just as 1/3 means > > exactly what it meant in grade school. Now try to represent 1/3 on a > > blackboard, as a decimal fraction. If that's impossible, does it mean > > that 1/3 doesn't mean 1/3, or that 1/3 can't be represented? > > As you know, it is possible, but let's say we outlaw any finite notation > for repeated digits... Why should I convert 1/3 to this particular > apparently unsuitable representation? I will write 1/3 and manipulate > that number using factional notation. If you want that, the fractions module is there for you. And again, grade school, we learned about ratios as well as decimals (or vulgar fractions and decimal fractions). They have different tradeoffs. For instance, I learned pi as both 22/7 and 3.14, because sometimes it'd be convenient to use the rational form and other times the decimal. > The novice programmer might similarly expect that when they write 0.3, > the program will manipulate that number as the faction it clearly is. > They may well be surprised by the fact that it must get put into a > format that can't represent what those three characters mean, just as I > would be surprised if you insisted I write 1/3 as a finite decimal (with > no repeat notation). Except that 0.3 isn't written as a fraction, it's written as a decimal. > I'm not saying your analogy would not help someone understand, but you > first have to explain why 0.3 is not treated as three tenths -- why I > (to use your analogy) must not keep 1/3 as a proper fraction, but I must > instead write it using a finite number of decimal digits. Neither is, > in my view, obvious to the beginner. Try adding 1/3 + e; either you have to convert 1/3 to a decimal, or find a rational approximation for e (there aren't any really good ones but 193/71 seems promising - that's 2.7183, close enough) and then go to the work of rational addition. No, more likely you'll go for a finite number of decimal digits. >
