You should start with defining what you want in terms of shape. I tried to understand it from the question, please add more details if it is not correct.
So here we have (1) empty array is not allowed and (2) no more than two dimensions. It translates the following way:
def is_allowed(arr):
return arr.shape != (0, ) and len(arr.shape) <= 2
The first condition just compares you array's shape with the shape of an empty array. the second condition checks that an array has no more than two dimensions.
With inner dimensions there is a problem. Some of the lists you provided as an example are not numpy arrays. If you cast np.array([[1,2,3],[2,3],[2,43,32]]), you get just an array where each element is the list. It is not a "real" numpy array with direct access to all the elements. See example:
>>> np.array([[1,2,3],[2,3],[2,43,32]])
array([list([1, 2, 3]), list([2, 3]), list([2, 43, 32])], dtype=object)
>>> np.array([[1,2,3],[2,3, None],[2,43,32]])
array([[1, 2, 3],
[2, 3, None],
[2, 43, 32]], dtype=object)
So I would recommend (if you are operating with usual lists) check that all arrays have the same length without numpy.
Answer from sooobus on Stack OverflowFirst, I recommend avoiding the matrix type altogether. The problems it causes outweighs the minor syntactical convenience it provides, in my (and quite a few others') opinion.
Second, the idiomatic way to do this kind of thing in numpy is not to assert that the inputs are particular types, but to coerce them to an ndarray. np.asarray() does this efficiently; if the input is an ndarray, the input passes through untouched; if it is an ndarray subclass, then a new ndarray will be created but only as a view on the original data so no data will be copied.
import numpy as np
def sigmoid(x):
# No need to use vectorize() for this. It's already vectorized.
return 1.0 / (1.0 + np.exp(-x))
def prob_hOn_givenV(v, b, w):
v = np.asarray(v)
b = np.asarray(b)
w = np.asarray(w)
# Don't bother checking the shape. `np.dot()` will do that for us.
# Strictly speaking, the `asarray(v)` and `asarray(w)` aren't necessary
# since `np.dot(v, w)` already does that internally. But it does no harm.
return sigmoid(b + np.dot(v, w))
These are the best links to pythonic ways of doing various checks:
Pre- and Post-conditions and Type Enforcement
You can use particular sets of asserts (or any other piece of code for more complicated checks, for example checking if an list of ints truly contains all ints) as a function, and then use that function as a parameter to the decorator, making a very clean description of what you're checking.
The accept/returns decorator is a quick and clean way of checking your inputs. The pre- and post-condition checking decorator is more suited for larger or more complicated sets of checks, whether it's type enforcement, or parameter range, or any other relationship between the data, like if an array's elements are sorted.
I used to include all these checks in the function alone, but with some functions it makes the code rather unreadable, as the real functionality gets lost in all the checks. This is a fairly simple way to organize your checks, and make your code easier to read.
+1 for not trusting duck typing ;)
Currently, numpy.typing.NDArray only accepts a dtype, like so: numpy.typing.NDArray[numpy.int32]. You have some options though.
Use typing.Annotated
typing.Annotated allows you to create an alias for a type and to bundle some extra information with it.
In some my_types.py you would write all variations of shapes you want to hint:
from typing import Annotated, Literal, TypeVar
import numpy as np
import numpy.typing as npt
DType = TypeVar("DType", bound=np.generic)
Array4 = Annotated[npt.NDArray[DType], Literal[4]]
Array3x3 = Annotated[npt.NDArray[DType], Literal[3, 3]]
ArrayNxNx3 = Annotated[npt.NDArray[DType], Literal["N", "N", 3]]
And then in foo.py, you can supply a numpy dtype and use them as typehint:
import numpy as np
from my_types import Array4
def foo(arr: Array4[np.int32]):
assert arr.shape == (4,)
MyPy will recognize arr to be an np.ndarray and will check it as such. The shape checking can be done at runtime only, like in this example with an assert.
If you don't like the assertion, you can use your creativity to define a function to do the checking for you.
def assert_match(arr, array_type):
hinted_shape = array_type.__metadata__[0].__args__
hinted_dtype_type = array_type.__args__[0].__args__[1]
hinted_dtype = hinted_dtype_type.__args__[0]
assert np.issubdtype(arr.dtype, hinted_dtype), "DType does not match"
assert arr.shape == hinted_shape, "Shape does not match"
assert_match(some_array, Array4[np.int32])
Use nptyping
Another option would be to use 3th party lib nptyping (yes, I am the author).
You would drop my_types.py as it would be of no use anymore.
Your foo.py would become something like:
from nptyping import NDArray, Shape, Int32
def foo(arr: NDArray[Shape["4"], Int32]):
assert isinstance(arr, NDArray[Shape["4"], Int32])
Use beartype + typing.Annotated
There is also another 3th party lib called beartype that you could use. It can take a variant of the typing.Annotated approach and will do the runtime checking for you.
You would reinstate your my_types.py with content similar to:
from beartype import beartype
from beartype.vale import Is
from typing import Annotated
import numpy as np
Int32Array4 = Annotated[np.ndarray, Is[lambda array:
array.shape == (4,) and np.issubdtype(array.dtype, np.int32)]]
Int32Array3x3 = Annotated[np.ndarray, Is[lambda array:
array.shape == (3,3) and np.issubdtype(array.dtype, np.int32)]]
And your foo.py would become:
import numpy as np
from beartype import beartype
from my_types import Int32Array4
@beartype
def foo(arr: Int32Array4):
... # Runtime type checked by beartype.
Use beartype + nptyping
You could also stack up both libraries.
Your my_types.py can be removed again and your foo.py would become something like:
from nptyping import NDArray, Shape, Int32
from beartype import beartype
@beartype
def foo(arr: NDArray[Shape["4"], Int32]):
... # Runtime type checked by beartype.
EDIT: Shape typing has been added to numpy. Types are covariant and they must be tuple[int, ...]. This allows things like NamedTuple, tuple[Literal[1]], tuple[NewType("Frame", int)]. Your example it would be:
import numpy as np
from typing import Literal
def foo(x: np.ndarray[tuple[Literal[4]], np.dtype[np.int32]]):
Currently, shape type information is reflected in ndarray.shape. However, most numpy functions that change the dimension or size of an array, however, don't necessarily know how to handle different axes and sizes in typing. As a result something like: arr[:] will lose the shape type information from arr.
Old answer: Prior to numpy 2.1, you could put anything as the first type argument to np.ndarray[_ShapeLike, _DtypeLike_co]. There was some consideration of TypeVarTuple (see PEP 646) for array shapes. However, bounds and variance is not implemented for TypeVarTuple, and therefore it cannot currently be used as the shape parameter.
When doing input validation for NumPy code, I always use np.asarray:
>>> np.asarray(np.array([1,2,3]))
array([1, 2, 3])
>>> np.asarray([1,2,3])
array([1, 2, 3])
>>> np.asarray((1,2,3))
array([1, 2, 3])
>>> np.asarray(1)
array(1)
>>> np.asarray(1).shape
()
This function has the nice feature that it only copies data when necessary; if the input is already an ndarray, the data is left in-place (only the type may be changed, because it also gets rid of that pesky np.matrix).
The docstring for ndim says
That's the docstring for the function np.ndim, not the ndim attribute, which non-NumPy objects don't have. You could use that function, but the effect would be that the data might be copied twice, so instead do:
S = np.asarray(S)
(p, p2) = S.shape
This will raise a ValueError if S.ndim != 2.
[Final note: you don't need ; in Python if you just follow the indentation rules. In fact, Python programmers eschew the semicolon.]
Given the comments to @larsmans answer, you could try:
if not isinstance(S, np.ndarray):
raise TypeError("Input not a ndarray")
if S.ndim == 0:
S = np.reshape(S, (1,1))
(p, p2) = S.shape
First, you check explicitly whether S is a (subclass of) ndarray. Then, you use the np.reshape to copy your data (and reshaping it, of course) if needed. At last, you get the dimension.
Note that in most cases, the np functions will first try to access the corresponding method of a ndarray, then attempt to convert the input to a ndarray (sometimes keeping it a subclass, as in np.asanyarray, sometimes not (as in np.asarray(...)). In other terms, it's always more efficient to use the method rather than the function: that's why we're using S.shape and not np.shape(S).
Another point: the np.asarray, np.asanyarray, np.atleast_1D... are all particular cases of the more generic function np.array. For example, asarray sets the optional copy argument of array to False, asanyarray does the same and sets subok=True, atleast_1D sets ndmin=1, atleast_2d sets ndmin=2... In other terms, it's always easier to use np.array with the appropriate arguments. But as mentioned in some comments, it's a matter of style. Shortcuts can often improve readability, which is always an objective to keep in mind.
In any case, when you use np.array(..., copy=True), you're explicitly asking for a copy of your initial data, a bit like doing a list([....]). Even if nothing else changed, your data will be copied. That has the advantages of its drawbacks (as we say in French), you could for example change the order from row-first C to column-first F. But anyway, you get the copy you wanted.
With np.array(input, copy=False), a new array is always created. It will either point to the same block of memory as input if this latter was already a ndarray (that is, no waste of memory), or will create a new one "from scratch" if input wasn't. The interesting case is of course if input was a ndarray.
Using this new array in a function may or may not change the original input, depending on the function. You have to check the documentation of the function you want to use to see whether it returns a copy or not. The NumPy developers try hard to limit unnecessary copies (following the Python example), but sometimes it can't be avoided. The documentation should tell explicitly what happens, if it doesn't or it's unclear, please mention it.
np.array(...) may raise some exceptions if something goes awry. For example, trying to use a dtype=float with an input like ["STRING", 1] will raise a ValueError. However, I must admit I can't remember which exceptions in all the cases, please edit this post accordingly.