[Numpy-discussion] Type declaration to include all valid numerical NumPy types for Cython

2020-08-09 Thread Ilhan Polat
Hi all,

As you might have seen my recent mails in Cython list, I'm trying to cook
up an input validator for the linalg.solve() function. The machinery of
SciPy linalg is as follows:

Some input comes in passes through np.asarray() then depending on the
resulting dtype of the numpy array we choose a LAPACK flavor (s,d,c,z) and
off it goes through f2py to lalaland and comes back with some result.

For the backslash polyalgorithm I need the arrays to be contiguous (C- or
F- doesn't matter) and any of the four (possibly via making new copies)
float, double, float complex, double complex after the intake because we
are using wrapped fortran code (LAPACK) in SciPy. So my difficulty is how
to type such function input, say,

ctypedef fused numeric_numpy_t:
bint
cnp.npy_bool
cnp.int_t
cnp.intp_t
cnp.int8_t
cnp.int16_t
cnp.int32_t
cnp.int64_t
cnp.uint8_t
cnp.uint16_t
cnp.uint32_t
cnp.uint64_t
cnp.float32_t
cnp.float64_t
cnp.complex64_t
cnp.complex128_t

Is this acceptable or something else needs to be used? Then there is the
storyof np.complex256 and mysterious np.float16. Then there is the Linux vs
Windows platform dependence issue and possibly some more that I can't
comprehend. Then there are datetime, str, unicode etc. that need to be
rejected. So this is quickly getting out of hand for my small brain.

To be honest, I am a bit running out of steam working with this issue even
though I managed to finish the actual difficult algorithmic part but got
stuck here. I am quite surprised how fantastically complicated and
confusing both NumPy and Cython docs about this stuff. Shouldn't we keep a
generic fused type for such usage? Or maybe there already exists but I
don't know and would be really grateful for pointers.

Here I wrote a dummy typed Cython function just for type checking:

cpdef inline bint ncc( numeric_numpy_t[:, :] a):
print(a.is_f_contig())
print(a.is_c_contig())

return a.is_f_contig() or a.is_c_contig()

And this is a dummy loop (with aliases) just to check whether fused type is
working or not (on windows I couldn't make it work for float16).

for x in (np.uint, np.uintc, np.uintp, np.uint0, np.uint8, np.uint16,
np.uint32,
  np.uint64, np.int, np.intc, np.intp, np.int0, np.int8, np.int16,
  np.int32,np.int64, np.float, np.float32, np.float64, np.float_,
  np.complex, np.complex64, np.complex128, np.complex_):
print(x)
C = np.arange(25., dtype=x).reshape(5, 5)
ncc(C)


Thanks in advance,
ilhan
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Type declaration to include all valid numerical NumPy types for Cython

2020-08-09 Thread Eric Moore
If that is really all you need, then the version in python is:

def convert_one(a):
"""
Converts input with arbitrary layout and dtype to a blas/lapack
compatible dtype with either C or F order.  Acceptable objects are
passed
through without making copies.
"""

a_arr = np.asarray(a)
dtype = np.result_type(a_arr, 1.0)

# need to handle these separately
if dtype == np.longdouble:
dtype = np.dtype('d')
elif dtype == np.clongdouble:
dtype = np.dtype('D')
elif dtype == np.float16:
dtype = np.dtype('f')

# explicitly force a copy if a_arr isn't one segment
return np.array(a_arr, dtype, copy=not a_arr.flags.forc, order='K')

In Cython, you could just run exactly this code and it's probably fine.
The could also be rewritten using the C calls if you really wanted.

You need to either provide your own or use a casting table and the copy /
conversion routines from somewhere.  Cython, to my knowledge, doesn't
provide these things, but Numpy does.

Eric

On Sun, Aug 9, 2020 at 6:16 PM Ilhan Polat  wrote:

> Hi all,
>
> As you might have seen my recent mails in Cython list, I'm trying to cook
> up an input validator for the linalg.solve() function. The machinery of
> SciPy linalg is as follows:
>
> Some input comes in passes through np.asarray() then depending on the
> resulting dtype of the numpy array we choose a LAPACK flavor (s,d,c,z) and
> off it goes through f2py to lalaland and comes back with some result.
>
> For the backslash polyalgorithm I need the arrays to be contiguous (C- or
> F- doesn't matter) and any of the four (possibly via making new copies)
> float, double, float complex, double complex after the intake because we
> are using wrapped fortran code (LAPACK) in SciPy. So my difficulty is how
> to type such function input, say,
>
> ctypedef fused numeric_numpy_t:
> bint
> cnp.npy_bool
> cnp.int_t
> cnp.intp_t
> cnp.int8_t
> cnp.int16_t
> cnp.int32_t
> cnp.int64_t
> cnp.uint8_t
> cnp.uint16_t
> cnp.uint32_t
> cnp.uint64_t
> cnp.float32_t
> cnp.float64_t
> cnp.complex64_t
> cnp.complex128_t
>
> Is this acceptable or something else needs to be used? Then there is the
> storyof np.complex256 and mysterious np.float16. Then there is the Linux vs
> Windows platform dependence issue and possibly some more that I can't
> comprehend. Then there are datetime, str, unicode etc. that need to be
> rejected. So this is quickly getting out of hand for my small brain.
>
> To be honest, I am a bit running out of steam working with this issue even
> though I managed to finish the actual difficult algorithmic part but got
> stuck here. I am quite surprised how fantastically complicated and
> confusing both NumPy and Cython docs about this stuff. Shouldn't we keep a
> generic fused type for such usage? Or maybe there already exists but I
> don't know and would be really grateful for pointers.
>
> Here I wrote a dummy typed Cython function just for type checking:
>
> cpdef inline bint ncc( numeric_numpy_t[:, :] a):
> print(a.is_f_contig())
> print(a.is_c_contig())
>
> return a.is_f_contig() or a.is_c_contig()
>
> And this is a dummy loop (with aliases) just to check whether fused type
> is working or not (on windows I couldn't make it work for float16).
>
> for x in (np.uint, np.uintc, np.uintp, np.uint0, np.uint8, np.uint16,
> np.uint32,
>   np.uint64, np.int, np.intc, np.intp, np.int0, np.int8, np.int16,
>   np.int32,np.int64, np.float, np.float32, np.float64, np.float_,
>   np.complex, np.complex64, np.complex128, np.complex_):
> print(x)
> C = np.arange(25., dtype=x).reshape(5, 5)
> ncc(C)
>
>
> Thanks in advance,
> ilhan
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion