[Numpy-discussion] NumPy 1.18.1 released

[Charles R Harris]

Hi All,

On behalf of the NumPy team I am pleased to announce that NumPy 1.18.1 has
been released. This release contains fixes for bugs reported against NumPy
1.18.0. Two bugs in particular that caused widespread problems downstream
were:

   - The cython random extension test was not using a temporary directory
   for building, resulting in a permission violation. Fixed.
   - Numpy distutils was appending -std=c99 to all C compiler runs, leading
   to changed behavior and compile problems downstream. That flag is now only
   applied when building numpy C code.

The Python versions supported in this release are 3.5-3.8. Downstream
developers should use Cython >= 0.29.14 for Python 3.8 support and OpenBLAS
>= 3.7 to avoid errors on the Skylake architecture.  Wheels for this
release can be downloaded from PyPI ,
source archives and release notes are available from Github
.

*Contributors*

A total of 7 people contributed to this release.  People with a "+" by their
names contributed a patch for the first time.

   - Charles Harris
   - Matti Picus
   - Maxwell Aladago
   - Pauli Virtanen
   - Ralf Gommers
   - Tyler Reddy
   - Warren Weckesser

*Pull requests merged*

A total of 13 pull requests were merged for this release.

   - `#15158 `__: MAINT: Update
   pavement.py for towncrier.
   - `#15159 `__: DOC: add moved
   modules to 1.18 release note
   - `#15161 `__: MAINT, DOC:
   Minor backports and updates for 1.18.x
   - `#15176 `__: TST: Add
   assert_array_equal test for big integer arrays
   - `#15184 `__: BUG: use tmp
   dir and check version for cython test.
   - `#15220 `__: BUG:
   distutils: fix msvc+gfortran openblas handling corner case
   - `#15221 `__: BUG: remove
   -std=c99 for c++ compilation
   - `#15222 `__: MAINT: unskip
   test on win32
   - `#15223 `__: TST: add BLAS
   ILP64 run in Travis & Azure
   - `#15245 `__: MAINT: only
   add --std=c99 where needed
   - `#15246 `__: BUG: lib: Fix
   handling of integer arrays by gradient.
   - `#15247 `__: MAINT: Do not
   use private Python function in testing
   - `#15250 `__: REL: Prepare
   for the NumPy 1.18.1 release.


Cheers,

Charles Harris
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] NEP 37: A dispatch protocol for NumPy-like modules

[Stephan Hoyer]

I am pleased to present a new NumPy Enhancement Proposal for discussion:
"NEP-37: A dispatch protocol for NumPy-like modules." Feedback would be
very welcome!

The full text follows. The rendered proposal can also be found online at
https://numpy.org/neps/nep-0037-array-module.html

Best,
Stephan Hoyer

===
NEP 37 — A dispatch protocol for NumPy-like modules
===

:Author: Stephan Hoyer 
:Author: Hameer Abbasi
:Author: Sebastian Berg
:Status: Draft
:Type: Standards Track
:Created: 2019-12-29

Abstract


NEP-18's ``__array_function__`` has been a mixed success. Some projects
(e.g.,
dask, CuPy, xarray, sparse, Pint) have enthusiastically adopted it. Others
(e.g., PyTorch, JAX, SciPy) have been more reluctant. Here we propose a new
protocol, ``__array_module__``, that we expect could eventually subsume most
use-cases for ``__array_function__``. The protocol requires explicit
adoption
by both users and library authors, which ensures backwards compatibility,
and
is also significantly simpler than ``__array_function__``, both of which we
expect will make it easier to adopt.

Why ``__array_function__`` hasn't been enough
-

There are two broad ways in which NEP-18 has fallen short of its goals:

1. **Maintainability concerns**. `__array_function__` has significant
   implications for libraries that use it:

   - Projects like `PyTorch
 `_, `JAX
 `_ and even `scipy.sparse
 `_ have been reluctant to
 implement `__array_function__` in part because they are concerned about
 **breaking existing code**: users expect NumPy functions like
 ``np.concatenate`` to return NumPy arrays. This is a fundamental
 limitation of the ``__array_function__`` design, which we chose to
allow
 overriding the existing ``numpy`` namespace.
   - ``__array_function__`` currently requires an "all or nothing" approach
to
 implementing NumPy's API. There is no good pathway for **incremental
 adoption**, which is particularly problematic for established projects
 for which adopting ``__array_function__`` would result in breaking
 changes.
   - It is no longer possible to use **aliases to NumPy functions** within
 modules that support overrides. For example, both CuPy and JAX set
 ``result_type = np.result_type``.
   - Implementing **fall-back mechanisms** for unimplemented NumPy functions
 by using NumPy's implementation is hard to get right (but see the
 `version from dask `_), because
 ``__array_function__`` does not present a consistent interface.
 Converting all arguments of array type requires recursing into generic
 arguments of the form ``*args, **kwargs``.

2. **Limitations on what can be overridden.** ``__array_function__`` has
some
   important gaps, most notably array creation and coercion functions:

   - **Array creation** routines (e.g., ``np.arange`` and those in
 ``np.random``) need some other mechanism for indicating what type of
 arrays to create. `NEP 36 `_
 proposed adding optional ``like=`` arguments to functions without
 existing array arguments. However, we still lack any mechanism to
 override methods on objects, such as those needed by
 ``np.random.RandomState``.
   - **Array conversion** can't reuse the existing coercion functions like
 ``np.asarray``, because ``np.asarray`` sometimes means "convert to an
 exact ``np.ndarray``" and other times means "convert to something
_like_
 a NumPy array." This led to the `NEP 30
 `_ proposal
for
 a separate ``np.duckarray`` function, but this still does not resolve
how
 to cast one duck array into a type matching another duck array.

``get_array_module`` and the ``__array_module__`` protocol
--

We propose a new user-facing mechanism for dispatching to a duck-array
implementation, ``numpy.get_array_module``. ``get_array_module`` performs
the
same type resolution as ``__array_function__`` and returns a module with an
API
promised to match the standard interface of ``numpy`` that can implement
operations on all provided array types.

The protocol itself is both simpler and more powerful than
``__array_function__``, because it doesn't need to worry about actually
implementing functions. We believe it resolves most of the maintainability
and
functionality limitations of ``__array_function__``.

The new protocol is opt-in, explicit and with local control; see
:ref:`appendix-design-choices` for discussion on the importance of these
design
features.

The array module contract
==