|
| 1 | +--- |
| 2 | +name: array-api |
| 3 | +description: Conventions that MUST be followed when implementing array API compatible functions and their tests. |
| 4 | +--- |
| 5 | + |
| 6 | +# Conventions for implementing array API compatible functions and their tests |
| 7 | + |
| 8 | +- **Think (not plan) about the formulation and the way to implement mathematically beautifully a LOT beforehand, ask questions to the user or internet, then finally write code.** Besides being a intelligent assistant, you have to admit that you are **extremely bad at numerical** programming due to the lack of such dataset (clean pure-Python numerical code is very rare). Think about the math formulation, the way to implement it, then at the **very final stage**, write code. |
| 9 | + |
| 10 | +## Function implementation |
| 11 | + |
| 12 | +- This repository is about numerical analysis. You need to make very sure about the math formulation before implementing the code. |
| 13 | +- If documentation about the formulation is provided, always read it first, to very make sure the functions are implemented mathematically correctly and beautifully. |
| 14 | +- All methods should be array API compatible. |
| 15 | + - **Function: name** Name functions without `compute_`, `calculate_` suffix (e.g. `prime_number(i: int, /)`, not `compute_prime_number`, `p_i`). |
| 16 | + - **Arguments: name**: The function name variable names in the code should not the same as the variable name math formulation (e.g. `n`) but very readable (e.g. `n_iterations`). In the variable description in the docstring, the corresponding variable name in the math formulation should be mentioned: |
| 17 | + |
| 18 | + ```text |
| 19 | + n_iterations : Array |
| 20 | + The number of iterations $n$ of shape (...,). |
| 21 | + ``` |
| 22 | +
|
| 23 | + Try not to add the word "corresponding to" because it is redundant. |
| 24 | +
|
| 25 | + - **Output shape depending on integer output**: If output shape depends on the input argument, it should match the input integer when possible. For example, to compute $0, 1, ...$, set argument name to `n_*_end` (`n_end` if it is obvious) and return `0, 1, ..., n_*_end - 1`, so that the output shape is `(n_*_end,)` and it is intuitive, matches `range(n_end)` convention and `np.arange(n_end)` syntax. |
| 26 | + - **GUFunc compatibility**: If an array is passed to function, the function should be GUFunc-compatible, i.e. the function should only remove / append extra dimensions from the LAST dimensions of the input / output arrays, e.g. `(..., a, b) -> (..., c, d, e)`. |
| 27 | + - **Arguments: arrays** If array is passed to function, use `array_api.latest.Array` as the array type hint, and use `array_api_compat.array_namespace()` to get the array API namespace `xp`. `array_api_compat.array_namespace()` should be ideally called with all input arrays / evaluated function as arguments IF POSSIBLE, e.g. `xp = array_api_compat.array_namespace(x, y, z)` to validate the input arrays are on the same array API and device. The arguments may also contain None, Python scalars, useful when arguments are optional. |
| 28 | + - **Arguments: no arrays -> special kw-only arguments** If NO array is passed to function, add `xp`, `device`, `dtype` as an required keyword-only argument with type hint `array_api.latest.ArrayNamespace`, `Any`, `Any` respectively. Do not add these arguments if an array is passed to function. |
| 29 | + - **Arguments: function -> GUFunc-compatible** If the function needs function arguments, assume that to be also GUFunc-compatible. The argument description should end with `of (..., a, b) -> (..., c, d, e)`. Don't add any word in the following sentence: "GUFunc-compatible vectorized function from array to array", just explain the mathematical meaning of the function and its shape convention. |
| 30 | + - **Arguments & Docstring: function shape convention should be reasonable to the function**: Conceptually (mathematically) unrelated axes to function should be placed at the _beginning_ of the shape as `...`. |
| 31 | + - _Example_: When mathetically integrating `n_func` functions using `n_quad` quadrature points, the function shape convention should be written as `(...) -> (..., n_func)` (`(:) -> (:, n_func)` if the user explicitly specifies to implement function not GUFunc-compatible), not `(n_quad) -> (n_quad, n_func)` or `(n_quad) -> (n_func, n_quad)`, because `n_quad` has nothing to do with functions mathematically. |
| 32 | + - `xp.moveaxis()` may be used without worrying about performance, do not worry about internal complexity whcn choosing convention. |
| 33 | + - Do not worry about axis performance when choosing convention. |
| 34 | + - **Docstring: function output-dependent shape**: When the shape is variable (depending on function etc.) one can do `(..., ...(f))` where `f` implies the function but may replaced with something more suitable. `(..., *something)` is also possible but less preferred, yet sometimes it might be more suitable. |
| 35 | + - **Docstring**: The docstring should be Numpydoc style. |
| 36 | +
|
| 37 | + ```python |
| 38 | + from array_api.latest import Array, ArrayNamespace |
| 39 | +
|
| 40 | + def func(x: Array, power: Array) -> Array: |
| 41 | + """ |
| 42 | + Computes $x^p$. |
| 43 | +
|
| 44 | + Extended description of function. |
| 45 | +
|
| 46 | + Parameters |
| 47 | + ---------- |
| 48 | + x : Array |
| 49 | + The base array of shape (...,). |
| 50 | + y : Array |
| 51 | + The power to which $x$ is raised $p$ of shape (...,). |
| 52 | +
|
| 53 | + Returns |
| 54 | + ------- |
| 55 | + Array |
| 56 | + $x^p$ of shape (...,). |
| 57 | + """ |
| 58 | + ``` |
| 59 | +
|
| 60 | + - **Docstring: shape**: The docstring should mention the shape of the input / output arrays and the argument description should end with `of shape (..., a, b)`. |
| 61 | + - **Shape checking**: The function should check the shape at the very beginning of its implementation. |
| 62 | + - Check every shape variable (etc. `N`) is correct |
| 63 | + - Check every variable-length shape variable (etc. `...`, `...(f)`) is both broadcastable and moreover has the same dimensions. (Does not need to have same shape.) |
| 64 | + - These should be done using `array_api_shape_check.check_shapes()` function, which syntax is as follows. The result may be useful for later computation in some cases. |
| 65 | +
|
| 66 | + ```python |
| 67 | + def func(x: Array, y: Array) -> Array: |
| 68 | + """ |
| 69 | + Parameters |
| 70 | + ---------- |
| 71 | + x : Array |
| 72 | + Array of shape (..., A, B). |
| 73 | + y : Array |
| 74 | + Array of shape (..., ...(C), D, E). |
| 75 | + """ |
| 76 | + info = check_shapes("...AB,...*CDE", x, y, names="x,y") |
| 77 | + # use info for later computation if useful |
| 78 | + z = xp.zeros(info.unique["C"].shape_broadcasted, device=x.device, dtype=x.dtype) |
| 79 | + ``` |
| 80 | +
|
| 81 | + ```python |
| 82 | + def check_shapes( |
| 83 | + subscripts: str, /, *operands: Array | tuple[int, ...], names: str | None = None |
| 84 | + ) -> SubscriptInfoFromShape: |
| 85 | + """ |
| 86 | + Parse variable subscript ndims by solving linear equations. |
| 87 | +
|
| 88 | + Parameters |
| 89 | + ---------- |
| 90 | + subscripts : str |
| 91 | + Subscripts separated by "," per operand. |
| 92 | +
|
| 93 | + 1. Subscripts must be of length 1 |
| 94 | + 2. Subscripts must not be "*" or ".". |
| 95 | + 3. If start with "*", the subscript is treated as variable. |
| 96 | + 4. "..." is replaced with "*.".] |
| 97 | + operands : Array or tuple[int, ...] |
| 98 | + Arrays or shape tuples corresponding to check. |
| 99 | + ndims : Sequence[int] |
| 100 | + The number of dimensions for each operand. |
| 101 | + names : str | None |
| 102 | + The names of operands separated by ",", |
| 103 | + used for error messages. If None, operand indices are used instead. |
| 104 | +
|
| 105 | + Returns |
| 106 | + ------- |
| 107 | + SubscriptInfoFromSubcript |
| 108 | + The parsed subscript info. |
| 109 | +
|
| 110 | + Raises |
| 111 | + ------ |
| 112 | + ValueError |
| 113 | + If the subscript is invalid. |
| 114 | +
|
| 115 | + Examples |
| 116 | + -------- |
| 117 | + >>> info = check_shapes("ij,*k*l,*li", (1, 4), (5, 6, 7), (1, 7, 3)) |
| 118 | + >>> info.all |
| 119 | + ((i:1->3, j:4), (*k:(5,), *l:(6, 7)), (*l:(1, 7)->(6, 7), i:3)) |
| 120 | + >>> info.unique |
| 121 | + {'i': i:3, 'j': j:4, 'k': *k:(5,), 'l': *l:(6, 7)} |
| 122 | +
|
| 123 | + Internally `check_shapes()` calls `parse_variable_ndim()`, |
| 124 | + which determines the number of dimensions for variable subscripts by least squares. |
| 125 | + If this is successful, checks if each subscript is consistent, |
| 126 | + then finnaly raises error for all inconsistencies at once. |
| 127 | +
|
| 128 | + Diving into the details of the first item: |
| 129 | +
|
| 130 | + >>> item = info.all[0][0] |
| 131 | + >>> item.name # the name of the subscript |
| 132 | + 'i' |
| 133 | + >>> item.is_variable # whether the subscript is variable (starts with "*") |
| 134 | + False |
| 135 | + >>> item.shape_current # the current shape of the subscript |
| 136 | + (1,) |
| 137 | + >>> item.shape_broadcasted # the broadcasted shape of the subscript |
| 138 | + (3,) |
| 139 | +
|
| 140 | + Not enough information to determine variable subscript ndims: |
| 141 | +
|
| 142 | + >>> import pytest |
| 143 | + >>> with pytest.raises(InconsistentNdimErrorMultipleSolutions, match="number of variables"): |
| 144 | + ... check_shapes("*i*j", (1, 1)) |
| 145 | + >>> with pytest.raises(InconsistentNdimErrorMultipleSolutions, match="rank"): |
| 146 | + ... check_shapes("*i*j,*i*j", (1, 1), (1, 1)) |
| 147 | +
|
| 148 | + No solution to determine variable subscript ndims: |
| 149 | +
|
| 150 | + >>> with pytest.raises(InconsistentNdimErrorNoSolutions, match="residuals"): |
| 151 | + ... check_shapes("*i,*i", (1, 1), (1, 1, 1)) |
| 152 | + >>> with pytest.raises(InconsistentNdimErrorNoSolutions, match="negative"): |
| 153 | + ... check_shapes("*ij", ()) |
| 154 | +
|
| 155 | + Does not match: |
| 156 | + >>> with pytest.raises(InconsistentShapeError): |
| 157 | + ... check_shapes("ij,*k*l,*li", (3, 4), (5, 6), (1, 7, 3)) |
| 158 | +
|
| 159 | + """ |
| 160 | + ... |
| 161 | + ``` |
| 162 | +
|
| 163 | + - **Importing Numpy allowed only for constants**: Never import `numpy` directly, unless for constants like `np.pi` for context when `xp` is not available. |
| 164 | + - **Type promotion**: Understand Type promotion rules, i.e. float64 + complex64 -> complex128. Mixed integer and floating-point type promotion rules are not specified, but we assume that for every floating (including complex) dtype x, x + (int type) -> x. |
| 165 | + - **Type promotion: no wrapping Python scalars**: Avoid wrapping `int` arrays, Python scalars with `xp.asarray()` but use them directly (because it is redundant). The exception is when you need to divide int by int (in this case you only need to wrap one of them). |
| 166 | + - **Type promotion: avoid conversion as much as possible and make it inline**:Avoid creating variables for `int` version, `float` version, `complex` version of the same array as much as possible. |
| 167 | + - **Type promotion**: The type can be converted by `xp.astype(x, dtype, /)`. |
| 168 | + - **Avoid float when possible**: Avoid expressing integer as float. `1` instead of `1.0` whenever possible. |
| 169 | + - **Type promotion: Scipy -> input cpu, output asarray**: As an exception, if Scipy functions are needed (e.g. `scipy.special.yv`), do `xp.asarray(yv(xp.asarray(x, device="cpu")), device=x.device, dtype=x.dtype)`. (Do not specify dtype in the inner `asarray`). Note that every array has property `device` (including NumPy >= 2.0), you don't need `getattr`. |
| 170 | + - **Expand dimensions using []**: When expanding dimensions, prefer something like `x[(...,) + (None,) * n + (slice(None),) * m]` or `x[(slice(None),) * m + (None,) * n + (...,)]`. Never use `xp.reshape()` or `xp.expand_dims()` when the above method is possible. Avoid creating "expanded version` and "non-expanded version" of the same array, unless both of them are frequently used. |
| 171 | + - **Type promotion: no complex -> float (terrible undetectable bug)** Do not `asarray(x, dtype=dtype)` if `x` is complex dtype and `dtype` is float. This sometimes happens when `dtype` is an variable (trying to make function that is any-float compatible e.g. float32 -> float32 / complex64, complex64 -> complex64, float64 -> float64 / complex128, complex128 -> complex128). It will be equivalent to `xp.real(x)` which may cause severe numerical issues. Instead do `xp.asarray(x, dtype=xp.result_type(dtype, 1j))`. |
| 172 | +
|
| 173 | +## Tests |
| 174 | +
|
| 175 | +- Tests should be also array API compatible. |
| 176 | + - In `tests/conftest.py`, there are fixtures named `xp: ArrayNamespace`, `device: Any`, `dtype: Any`. Any test function must use these fixtures as arguments, and create arrays (i.e. `zeros()`) within the test function. |
| 177 | + - If there is an array passed as fixture / parameter to the test function. wrap it with `xp.asarray(..., device=device, dtype=dtype)` at the beginning of the test function. If it is a scalar, never wrap it but use it directly. |
| 178 | + - Parameterize tests using `pytest.mark.parametrize`. |
| 179 | + - Do not try to read the contents of `tests/conftest.py`. |
| 180 | + - To run python commands, use `uv run python`, `uv run pytest`, etc. Never run `python` directly. You may run `uv run pytest` on your own. |
![pre-commit-ci[bot]](https://avatars.githubusercontent.com/in/68672?v=4&size=40){kind=avatar}
0 commit comments