C Module's '1.#INF' changes to 'inf' at Python
Hi All,
My problem is that I've noticed a strange behavior in Python while handling
FPEs on Windows after switching compilers (msvc8 to msvc9) and I am trying to
find out how Python handles INF values to figure out where the problem might
be.
The problem appeared when we noticed that behavior of Numeric (older version of
NumPy) library has changed when dealing with Floating-Point Exceptions. We are
building our own slightly modified version of Python in-house (including
Numeric/NumPy) and when we switched from the msvc8 to msvc9 compiler, Numeric
started to return '-inf' as a result of 'Numeric.log10(0.0)' instead of rasing
an OverflowError. I know that Numeric is using umath instead of the math
library and since math.log10(0.0) is still raising an Error (which is
ValueError in 2.6 and OverflowError in 2.4, but still an error) I thought that
it might have something to do with how umath or Numeric handles the input or
return values of log functions.
Before hunting blindly I wanted to see how Python interprets INF results. I
built a simple module with only one function that causes an FPE with log10(0.0)
using C's math library and prints the results with printf. Then I imported that
module from python, called the function and printed the result in python too to
see if there are any differences between two values (which I found out that
there are)
Here is the that part of the code from my C Python module:
static PyObject *badNum(PyObject *self, PyObject *args)
{
int sts = 0; //Not used, just dummy
double a = 0.0;
double c = 0.0;
if (!PyArg_ParseTuple(args, "i", &sts)) //Dummy check, no use
return NULL;
disableFPEs();
c = log10(a); //Since FPEs disabled, should return '-1.#INF'
enableFPEs();
printf("C-Val: %f\n", c);
return Py_BuildValue("d", c);
}
After I build the module and imported it into Python, I called the function and
printed the return value in Python. The output was different in two versions of
Python:
Output1 (Module was built with msvc8 and run by python24)
C-Val: -1.#INF
Py-Val: -1.#INF
Output2 (Module was built with msvc9 and run by python26)
C-Val: -1.#INF
Py-Val: -inf
I now know that compiler change didn't cause any change in return value of
log10(0.0) at C level (both compilations resulted in '-1.#INF') so I am
thinking that somehow at somewhere in Python core the value is translated into
'-inf' and that might be the reason why Numeric/NumPy is missing the problem
and having issues raising OverflowError like it's supposed to.
I started debugging Python.exe (with my module loaded) in Visual Studio. While
tracing what's happening after the line "return Py_BuildValue("d", c);" I went
down to the "floatobject.c " file (which I thought is highly relevant to
floating-point value conversions) and in particular "format_float()" function
but since I am not that familiar with Python internals, I am having trouble
pinpointing the issue and the code where the "conversion" is happening and I
can't even be sure if I am looking at the right place.
I would appreciate if someone could help me out or at least could point me to
the right direction so that I can figure out the rest.
Thanks,
Best Regards,
Ali Erman CELEN
Platform Specialists / Porting
Dassault Systèmes I www.3ds.com
RE: C Module's '1.#INF' changes to 'inf' at Python
> Numeric.log10() will check to see if the errno was set to ERANGE. It does not
> check if a floating point exception flag was set, which is tricky to do
> across
> platforms. The newer numpy can do it because we've finally managed to
> implement
> all of that platform-specific code, but the earlier Numeric does not.
> Presumably, the msvc8 C runtime's implementation of log10() sets errno and
> the
> msvc9 runtime's version does not.
It doesn't seem like C part is changed. I confirmed that the behavior of
log10(0.0) in C's standard math.h library didn't change between compilers
(msvc8 and msvc9 both sets the errno to 34(ERANGE)). Now I'm thinking that this
setting errno problem is happening somewhere in Numeric's log10 wrappers.
As I see, Numeric's "PyUFunc_GenericFunction" checks errno value to see if the
called function has set it and if it is non-zero, it calls math_error which
raises the exception. The problem is that now errno is not being set but I
can't see why since I can't step into that redirected function call ("*(double
*)op = ((DoubleUnaryFunc *)func)(*(double *)ip1)" where function value is
"0x01c4ede0 log10").
I am wondering which functions are actually being called when I call
log10(0.0). Could you (or anybody) point me where this errno is supposed to be
set in Numeric or umath when I call log10(0.0) so that I can take a look at why
this is not being the case.
(I also noticed that this behavior is same under standard NumPy 1.4 with
standard Python 2.6 on Windows. If you call numpy.log10(0.0) you will get an
"-inf" and no exceptions will be raised. Which is not the case with Python's
standard math.log10(0.0) which will raise a ValueError)
Best Regards,
Ali Erman CELEN
Platform Specialists / Porting
This email and any attachments are intended solely for the use of the
individual or entity to whom it is addressed and may be confidential and/or
privileged. If you are not one of the named recipients or have received this
email in error, (i) you should not read, disclose, or copy it, (ii) please
notify sender of your receipt by reply email and delete this email and all
attachments, (iii) Dassault Systemes does not accept or assume any liability or
responsibility for any use of or reliance on this email.For other languages, go
to http://www.3ds.com/terms/email-disclaimer.
--
http://mail.python.org/mailman/listinfo/python-list
RE: C Module's '1.#INF' changes to 'inf' at Python
>> (I also noticed that this behavior is same under standard NumPy 1.4
>> with standard Python 2.6 on Windows. If you call numpy.log10(0.0) you
>> will get an "-inf" and no exceptions will be raised. Which is not the
>> case with Python's standard math.log10(0.0) which will raise a
>> ValueError)
>
> Correct. This is numpy's intended behavior. See numpy.seterr() to enable
> exceptions if you want them.
Numpy.seterr() doesn't seem to be working in case of log10(0.0) (output with
all standard: Python2.6 with NumPy1.4 on Windows-32bit is below)
Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]
on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.seterr()
{'over': 'ignore', 'divide': 'ignore', 'invalid': 'ignore', 'under': 'ignore'}
>>> numpy.int16(32000) * numpy.int16(3)
30464
>>> numpy.log10(0.0)
-inf
>>> numpy.seterr(all='raise')
{'over': 'ignore', 'divide': 'ignore', 'invalid': 'ignore', 'under': 'ignore'}
>>> numpy.int16(32000) * numpy.int16(3)
Traceback (most recent call last):
File "", line 1, in
FloatingPointError: overflow encountered in short_scalars
>>> numpy.log10(0.0)
-inf
>>> numpy.log10(-1.0)
nan
>>> numpy.seterr()
{'over': 'raise', 'divide': 'raise', 'invalid': 'raise', 'under': 'raise'}
>>>
Also I'm pretty sure C's log10 (math.h) still sets errno to ERANGE (34) with
both msvc8 and msvc9. That's why I thought a Numeric/Numpy wrapper might be
involved but you are saying, if I'm not mistaken, math library's log10()
function is called directly (without any special log10 wrappers) and only time
Numeric changes errno is where it sets errno=0 before calling C's log10.
Somehow, errno is not being set or gets changed/masked during this whole log10
call procedure and this might also be the reason why current version of Numpy
is missing the error even if we do numpy.seterr(all='raise').
So, why do you think errno is not being set under Python with Numeric.log10 (or
numpy.log10() which also seems to have the same 'letting overflow error
through' issue) ?
Thanks for your help,
Best Regards,
Ali Erman CELEN
Platform Specialists / Porting
This email and any attachments are intended solely for the use of the
individual or entity to whom it is addressed and may be confidential and/or
privileged. If you are not one of the named recipients or have received this
email in error, (i) you should not read, disclose, or copy it, (ii) please
notify sender of your receipt by reply email and delete this email and all
attachments, (iii) Dassault Systemes does not accept or assume any liability or
responsibility for any use of or reliance on this email.For other languages, go
to http://www.3ds.com/terms/email-disclaimer.
--
http://mail.python.org/mailman/listinfo/python-list
