You should start with defining what you want in terms of shape. I tried to understand it from the question, please add more details if it is not correct.

So here we have (1) empty array is not allowed and (2) no more than two dimensions. It translates the following way:

def is_allowed(arr):
  return arr.shape != (0, ) and len(arr.shape) <= 2

The first condition just compares you array's shape with the shape of an empty array. the second condition checks that an array has no more than two dimensions.

With inner dimensions there is a problem. Some of the lists you provided as an example are not numpy arrays. If you cast np.array([[1,2,3],[2,3],[2,43,32]]), you get just an array where each element is the list. It is not a "real" numpy array with direct access to all the elements. See example:

>>> np.array([[1,2,3],[2,3],[2,43,32]])
array([list([1, 2, 3]), list([2, 3]), list([2, 43, 32])], dtype=object)
>>> np.array([[1,2,3],[2,3, None],[2,43,32]])
array([[1, 2, 3],
       [2, 3, None],
       [2, 43, 32]], dtype=object)

So I would recommend (if you are operating with usual lists) check that all arrays have the same length without numpy.

Answer from sooobus on Stack Overflow
🌐
Medium
medium.com › @nearlydaniel › assertion-of-arbitrary-array-shapes-in-python-3c96f6b7ccb4
Assertion of arbitrary array shapes in Python | by Daniel | Medium
September 15, 2019 - A problem I’ve run into continually as a data scientist is the need to ensure an array (numpy, pytorch, tensorflow, etc) is of a specific dimension. For instance, making sure that all images loaded from a dataset are exactly 3x224x224 (channel, height, width) in dimension. This isn’t that hard to do, you can just hardcode an assertion:
Top answer
1 of 2
2

First, I recommend avoiding the matrix type altogether. The problems it causes outweighs the minor syntactical convenience it provides, in my (and quite a few others') opinion.

Second, the idiomatic way to do this kind of thing in numpy is not to assert that the inputs are particular types, but to coerce them to an ndarray. np.asarray() does this efficiently; if the input is an ndarray, the input passes through untouched; if it is an ndarray subclass, then a new ndarray will be created but only as a view on the original data so no data will be copied.

import numpy as np

def sigmoid(x):
    # No need to use vectorize() for this. It's already vectorized.
    return 1.0 / (1.0 + np.exp(-x))

def prob_hOn_givenV(v, b, w):
    v = np.asarray(v)
    b = np.asarray(b)
    w = np.asarray(w)
    # Don't bother checking the shape. `np.dot()` will do that for us.
    # Strictly speaking, the `asarray(v)` and `asarray(w)` aren't necessary
    # since `np.dot(v, w)` already does that internally. But it does no harm.
    return sigmoid(b + np.dot(v, w))
2 of 2
0

These are the best links to pythonic ways of doing various checks:

Pre- and Post-conditions and Type Enforcement

You can use particular sets of asserts (or any other piece of code for more complicated checks, for example checking if an list of ints truly contains all ints) as a function, and then use that function as a parameter to the decorator, making a very clean description of what you're checking.

The accept/returns decorator is a quick and clean way of checking your inputs. The pre- and post-condition checking decorator is more suited for larger or more complicated sets of checks, whether it's type enforcement, or parameter range, or any other relationship between the data, like if an array's elements are sorted.

I used to include all these checks in the function alone, but with some functions it makes the code rather unreadable, as the real functionality gets lost in all the checks. This is a fairly simple way to organize your checks, and make your code easier to read.

+1 for not trusting duck typing ;)

🌐
NumPy
numpy.org › devdocs › reference › generated › numpy.testing.assert_array_equal.html
numpy.testing.assert_array_equal — NumPy v2.6.dev0 Manual
Raises an AssertionError if two array_like objects are not equal. Given two array_like objects, check that the shape is equal and all elements of these objects are equal (but see the Notes for the special handling of a scalar). An exception is raised at shape mismatch or conflicting values.
🌐
NumPy
numpy.org › doc › stable › reference › generated › numpy.testing.assert_array_equal.html
numpy.testing.assert_array_equal — NumPy v2.5 Manual
Raises an AssertionError if two array_like objects are not equal. Given two array_like objects, check that the shape is equal and all elements of these objects are equal (but see the Notes for the special handling of a scalar). An exception is raised at shape mismatch or conflicting values.
Top answer
1 of 3
4

When doing input validation for NumPy code, I always use np.asarray:

>>> np.asarray(np.array([1,2,3]))
array([1, 2, 3])
>>> np.asarray([1,2,3])
array([1, 2, 3])
>>> np.asarray((1,2,3))
array([1, 2, 3])
>>> np.asarray(1)
array(1)
>>> np.asarray(1).shape
()

This function has the nice feature that it only copies data when necessary; if the input is already an ndarray, the data is left in-place (only the type may be changed, because it also gets rid of that pesky np.matrix).

The docstring for ndim says

That's the docstring for the function np.ndim, not the ndim attribute, which non-NumPy objects don't have. You could use that function, but the effect would be that the data might be copied twice, so instead do:

S = np.asarray(S)
(p, p2) = S.shape

This will raise a ValueError if S.ndim != 2.

[Final note: you don't need ; in Python if you just follow the indentation rules. In fact, Python programmers eschew the semicolon.]

2 of 3
3

Given the comments to @larsmans answer, you could try:

if not isinstance(S, np.ndarray):
    raise TypeError("Input not a ndarray")
if S.ndim == 0:
    S = np.reshape(S, (1,1))
(p, p2) = S.shape

First, you check explicitly whether S is a (subclass of) ndarray. Then, you use the np.reshape to copy your data (and reshaping it, of course) if needed. At last, you get the dimension.

Note that in most cases, the np functions will first try to access the corresponding method of a ndarray, then attempt to convert the input to a ndarray (sometimes keeping it a subclass, as in np.asanyarray, sometimes not (as in np.asarray(...)). In other terms, it's always more efficient to use the method rather than the function: that's why we're using S.shape and not np.shape(S).

Another point: the np.asarray, np.asanyarray, np.atleast_1D... are all particular cases of the more generic function np.array. For example, asarray sets the optional copy argument of array to False, asanyarray does the same and sets subok=True, atleast_1D sets ndmin=1, atleast_2d sets ndmin=2... In other terms, it's always easier to use np.array with the appropriate arguments. But as mentioned in some comments, it's a matter of style. Shortcuts can often improve readability, which is always an objective to keep in mind.

In any case, when you use np.array(..., copy=True), you're explicitly asking for a copy of your initial data, a bit like doing a list([....]). Even if nothing else changed, your data will be copied. That has the advantages of its drawbacks (as we say in French), you could for example change the order from row-first C to column-first F. But anyway, you get the copy you wanted.

With np.array(input, copy=False), a new array is always created. It will either point to the same block of memory as input if this latter was already a ndarray (that is, no waste of memory), or will create a new one "from scratch" if input wasn't. The interesting case is of course if input was a ndarray.

Using this new array in a function may or may not change the original input, depending on the function. You have to check the documentation of the function you want to use to see whether it returns a copy or not. The NumPy developers try hard to limit unnecessary copies (following the Python example), but sometimes it can't be avoided. The documentation should tell explicitly what happens, if it doesn't or it's unclear, please mention it.

np.array(...) may raise some exceptions if something goes awry. For example, trying to use a dtype=float with an input like ["STRING", 1] will raise a ValueError. However, I must admit I can't remember which exceptions in all the cases, please edit this post accordingly.

Find elsewhere
🌐
NumPy
numpy.org › doc › stable › reference › generated › numpy.ndarray.shape.html
numpy.ndarray.shape — NumPy v2.5 Manual
The shape property is usually used to get the current shape of an array, but may also be used to reshape the array in-place by assigning a tuple of array dimensions to it. As with numpy.reshape, one of the new shape dimensions can be -1, in which case its value is inferred from the size of ...
🌐
W3Schools
w3schools.com › python › numpy › numpy_array_shape.asp
NumPy Array Shape
The shape of an array is the number of elements in each dimension. NumPy arrays have an attribute called shape that returns a tuple with each index having the number of corresponding elements.
🌐
NumPy
numpy.org › doc › 2.2 › reference › generated › numpy.ndarray.shape.html
numpy.ndarray.shape — NumPy v2.2 Manual
The shape property is usually used to get the current shape of an array, but may also be used to reshape the array in-place by assigning a tuple of array dimensions to it. As with numpy.reshape, one of the new shape dimensions can be -1, in which case its value is inferred from the size of ...
🌐
GitHub
github.com › numpy › numpy › issues › 11101
Testing array assertion shape check fails with float/1d array comparison · Issue #11101 · numpy/numpy · GitHub
May 15, 2018 - Current behavior: any of the following lines passes np.testing.assert_array_almost_equal(42, [42]) np.testing.assert_array_almost_equal([42], 42) np.testing.assert_array_equal(42, [42]) np.testing.assert_array_equal([42], 42) Both array_almost_equal and array_almost_equal are supposed to check shapes, but it skips the check when comparing floats to 1D arrays. This behavior comes from line 711 in numpy/testing/_private/utils.py: cond = (x.shape == () or y.shape == ()) or x.shape == y.shape It specifically skips the shape check when one of the inputs is a float.
Author: numpy
🌐
NumPy
numpy.org › doc › 2.1 › reference › generated › numpy.equal.html
numpy.equal — NumPy v2.1 Manual
A location into which the result is stored. If provided, it must have a shape that the inputs broadcast to. If not provided or None, a freshly-allocated array is returned.
🌐
NumPy
numpy.org › doc › 2.1 › reference › generated › numpy.testing.assert_equal.html
numpy.testing.assert_equal — NumPy v2.1 Manual
If True and either of the actual and desired arguments is an array, raise an AssertionError when either the shape or the data type of the arguments does not match.
🌐
NumPy
numpy.org › doc › stable › reference › testing.html
Testing guidelines — NumPy v2.5 Manual
By default, these assertion functions only compare the numerical values in the arrays. Consider using the strict=True option to check the array dtype and shape, too.
🌐
SciPy
docs.scipy.org › doc › › numpy-1.16.1 › reference › generated › numpy.testing.assert_array_equal.html
numpy.testing.assert_array_equal — NumPy v1.16 Manual
January 31, 2019 - Raises an AssertionError if two array_like objects are not equal. Given two array_like objects, check that the shape is equal and all elements of these objects are equal. An exception is raised at shape mismatch or conflicting values. In contrast to the standard usage in numpy, NaNs are compared ...
🌐
NumPy
numpy.org › doc › stable › reference › generated › numpy.testing.assert_array_less.html
numpy.testing.assert_array_less — NumPy v2.4 Manual
Raises an AssertionError if two array_like objects are not ordered by less than. Given two array_like objects x and y, check that the shape is equal and all elements of x are strictly less than the corresponding elements of y (but see the Notes for the special handling of a scalar).
Top answer
1 of 6
2

You can use vstack(array1, array2) from numpy which stacks arrays vertically. For example:

A = np.random.randint(2, size = (2, 16))
B = np.random.randint(2, size = (5, 16))

print A.shape
print B.shape
if A.shape[0] < B.shape[0]:
    A = np.vstack((A, np.zeros((B.shape[0] - A.shape[0], 16))))
elif A.shape[0] > B.shape[0]:
    B = np.vstack((B, np.zeros((A.shape[0] - B.shape[0], 16)))) 

print A.shape   
print A

In your case:

if data.shape[0] < data_sys.shape[0]:
    data = np.vstack((data, np.zeros((data_sys.shape[0] - data.shape[0], 16))))
elif data.shape[0] > data_sys.shape[0]:
    data_sys = np.vstack((data_sys, np.zeros((data.shape[0] - data_sys.shape[0], 16)))) 

I assume that your matrices have always the same number of columns, if not you can similarly use hstack to stack them horizontally.

2 of 6
2

If you have only two files, and their shapes differ in just the 0th dimension, a simple check and copy is probably easiest, though it lacks generality:

import numpy as np

data = np.genfromtxt('./test1.csv', dtype=float, delimiter=',')
data_sys = np.genfromtxt('./test2.csv', dtype=float, delimiter=',')   

fill_value = 0 # could be np.nan or something else instead

if data.shape[0]>data_sys.shape[0]:
    temp = data_sys
    data_sys = np.ones(data.shape)*fill_value
    data_sys[:temp.shape[0],:] = temp
elif data.shape[0]<data_sys.shape[0]:
    temp = data
    data = np.ones(data_sys.shape)*fill_value
    data[:temp.shape[0],:] = temp

print 'Using conditional:'
print data.shape
print data_sys.shape
if data.shape != data_sys.shape:
        print "we have an error"

A much more general solution is a custom class--overkill for your two files but much easier if you have lots of files to handle. The basic idea is that static class variables sx and sy keep track of the largest widths and heights, and are used when get_data is called, to output a standard shape array. This is pre-filled with your desired fill value, and the actual data from the corresponding file are copied into the upper left corner of the standard shape array:

import numpy as np

class IsomorphicArray:

    sy = 0 # static class variable
    sx = 0 # static class variable
    fill_value = 0.0

    def __init__(self,csv_filename):
        self.data = np.genfromtxt(csv_filename,dtype=float,delimiter=',')
        self.instance_sy,self.instance_sx = self.data.shape
        if self.instance_sy>IsomorphicArray.sy:
            IsomorphicArray.sy = self.instance_sy
        if self.instance_sx>IsomorphicArray.sx:
            IsomorphicArray.sx = self.instance_sx

    def get_data(self):
        out = np.ones((IsomorphicArray.sy,IsomorphicArray.sx))*self.fill_value
        out[:self.instance_sy,:self.instance_sx] = self.data
        return out

isomorphic_array_list = []

for filename in ['./test1.csv','./test2.csv']:
    isomorphic_array_list.append(IsomorphicArray(filename))

numpy_array_list = []

for isomorphic_array in isomorphic_array_list:
    numpy_array_list.append(isomorphic_array.get_data())


print 'Using custom class:'
for numpy_array in numpy_array_list:
    print numpy_array.shape
🌐
NumPy
numpy.org › doc › stable › reference › generated › numpy.testing.assert_equal.html
numpy.testing.assert_equal — NumPy v2.5 Manual
If True and either of the actual and desired arguments is an array, raise an AssertionError when either the shape or the data type of the arguments does not match.
🌐
NumPy
numpy.org › doc › 2.3 › reference › generated › numpy.testing.assert_array_equal.html
numpy.testing.assert_array_equal — NumPy v2.3 Manual
Raises an AssertionError if two array_like objects are not equal. Given two array_like objects, check that the shape is equal and all elements of these objects are equal (but see the Notes for the special handling of a scalar). An exception is raised at shape mismatch or conflicting values.