ENH: Add registration for sorting loops using new ufunc convenience function by MaanasArora · Pull Request #29900 · numpy/numpy · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions doc/release/upcoming_changes/29900.c_api.rst
9 changes: 9 additions & 0 deletions doc/release/upcoming_changes/29900.new_feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
DType sorting and argsorting supports the ArrayMethod API
---------------------------------------------------------
User-defined dtypes can now implement custom sorting and argsorting using
the ArrayMethod API. This mechanism can be used in place of the `PyArray_ArrFuncs`
slots which may be deprecated in the future.

The sorting and argsorting methods are registered by passing the arraymethod
specs that implement the operations to the new `PyUFunc_AddLoopsFromSpecs` function.
See the ArrayMethod API documentation for details.
58 changes: 58 additions & 0 deletions doc/source/reference/c-api/array.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1886,6 +1886,34 @@ with the rest of the ArrayMethod API.
the main ufunc registration function. This adds a new implementation/loop
to a ufunc. It replaces `PyUFunc_RegisterLoopForType`.

.. c:type:: PyUFunc_LoopSlot

Structure used to add multiple loops to ufuncs from ArrayMethod specs.
This is used in `PyUFunc_AddLoopsFromSpecs`.

.. c:struct:: PyUFunc_LoopSlot

.. c:member:: const char *name

The name of the ufunc to add the loop to.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is slightly vague. Where is this name found? Could one register for a ufunc defined outside numpy? I think by default it should be looked up on np, but one should be able to give a name including package. I guess one can test this using some of the private test ufuncs.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could do this, I think. The logic would be to allow for a :, if given, you import the part before the : and then use GetAttr for the part after (in principle should support . in the second part too).
It might be that the import mechanism can already do both in one go, but I am not sure. If it is tedious in C, could create a little Python helper to do it since it is very easy there, e.g. something like this

And if there is no : we would just go with numpy (and maybe the internal ufunc module as well to robustify it).

I am wondering if it makes sense to provision for when the object isn't found (or not a ufunc). We could for example allow for "?numpy.strings:..." to make it easier to support across versions.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I keep thinking about nice ways to expand, but the more I do so, the more it is clear that we should not go there now!

So, just a small comment: I would call this entry plain "name" -- it is already in the "UFunc_LoopSlot" namespace, so adding ufunc_ seems superfluous.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense - I agree adding more detail could be a bit broad at this time maybe! I've changed the ufunc_name to name, thanks!

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am happy to not do it here, but I am curious about your reasoning :). I suppose it may be weird enough to support for non-numpy and honestly, writing this helper isn't hard (although this type of convenience seems to make more of a dent than I tend to expect).

Anyway, one thing we don't yet support, but happy to also defer is adding ufuncs that are not available in the main namespace.


.. c:member:: PyArrayMethod_Spec *spec

The ArrayMethod spec to use to create the loop.

.. c:function:: int PyUFunc_AddLoopsFromSpecs( \
PyUFunc_LoopSlot *slots)

.. versionadded:: 2.4

Add multiple loops to ufuncs from ArrayMethod specs. This also
handles the registration of methods for the ufunc-like functions
``sort`` and ``argsort``. See :ref:`array-methods-sorting` for details.

The ``slots`` argument must be a NULL-terminated array of
`PyUFunc_LoopSlot` (see above), which give the name of the
ufunc and spec needed to create the loop.
Comment thread
mhvk marked this conversation as resolved.

.. c:function:: int PyUFunc_AddPromoter( \
PyObject *ufunc, PyObject *DType_tuple, PyObject *promoter)

Expand Down Expand Up @@ -2036,6 +2064,36 @@ code:
Py_INCREF(loop_descrs[2]);
}

.. _array-methods-sorting:

Sorting and Argsorting
Comment thread
mhvk marked this conversation as resolved.
~~~~~~~~~~~~~~~~~~~~~~~

Sorting and argsorting methods for dtypes can be registered using the
ArrayMethod API. This is done by adding an ArrayMethod spec with the name
``"sort"`` or ``"argsort"`` respectively. The spec must have ``nin=1``
and ``nout=1`` for both sort and argsort. Sorting is inplace, hence we
enforce that ``data[0] == data[1]``. Argsorting returns a new array of
indices, so the output must be of ``NPY_INTP`` type.

The ``context`` passed to the loop contains the ``parameters`` field which
for these operations is a ``PyArrayMethod_SortParameters *`` struct. This
struct contains a ``flags`` field which is a bitwise OR of ``NPY_SORTKIND``
values indicating the kind of sort to perform (that is, whether it is a
stable and/or descending sort). If the strided loop depends on the flags,
a good way to deal with this is to define :c:macro:`NPY_METH_get_loop`,
and not set any of the other loop slots.

.. c:struct:: PyArrayMethod_SortParameters

.. c:member:: NPY_SORTKIND flags

The flags passed to the sort operation. This is a bitwise OR of
``NPY_SORTKIND`` values indicating the kind of sort to perform.

These specs can be registered using :c:func:`PyUFunc_AddLoopsFromSpecs`
along with other ufunc loops.

API for calling array methods
-----------------------------

Expand Down
6 changes: 4 additions & 2 deletions numpy/_core/code_generators/cversions.txt
Original file line number Diff line number Diff line change
Expand Up @@ -80,5 +80,7 @@
0x00000013 = 2b8f1f4da822491ff030b2b37dff07e3
# Version 20 (NumPy 2.3.0)
0x00000014 = e56b74d32a934d085e7c3414cb9999b8,
# Version 21 (NumPy 2.4.0) Add 'same_value' casting, header additions
0x00000015 = e56b74d32a934d085e7c3414cb9999b8,
# Version 21 (NumPy 2.4.0)
# Add 'same_value' casting, header additions.
# General loop registration for ufuncs, sort, and argsort
0x00000015 = fbd24fc5b2ba4f7cd3606ec6128de7a5
4 changes: 3 additions & 1 deletion numpy/_core/code_generators/numpy_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -412,7 +412,7 @@ def get_annotations():
}

ufunc_types_api = {
'PyUFunc_Type': (0,)
'PyUFunc_Type': (0,),
}

ufunc_funcs_api = {
Expand Down Expand Up @@ -468,6 +468,8 @@ def get_annotations():
'PyUFunc_AddPromoter': (44, MinVersion("2.0")),
'PyUFunc_AddWrappingLoop': (45, MinVersion("2.0")),
'PyUFunc_GiveFloatingpointErrors': (46, MinVersion("2.0")),
# End 2.0 API
'PyUFunc_AddLoopsFromSpecs': (47, MinVersion("2.4")),
}

# List of all the dicts which define the C API
Expand Down
7 changes: 7 additions & 0 deletions numpy/_core/include/numpy/dtype_api.h
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,13 @@ typedef struct {
} PyArrayMethod_Spec;


// This is used for the convenience function `PyUFunc_AddLoopsFromSpecs`
typedef struct {
const char *name;
PyArrayMethod_Spec *spec;
} PyUFunc_LoopSlot;


/*
* ArrayMethod slots
* -----------------
Expand Down
10 changes: 10 additions & 0 deletions numpy/_core/src/common/npy_import.h
Loading
Loading