If you have a list of lists, you only needed to use ...
import numpy as np
...
npa = np.asarray(someListOfLists, dtype=np.float32)
per this LINK in the scipy / numpy documentation. You just needed to define dtype inside the call to asarray.
Answer from Thom Ives on Stack OverflowIf you have a list of lists, you only needed to use ...
import numpy as np
...
npa = np.asarray(someListOfLists, dtype=np.float32)
per this LINK in the scipy / numpy documentation. You just needed to define dtype inside the call to asarray.
If you're converting a list into an array, you'll need to make a new copy anyway, so
arr = np.array(my_list, dtype='float32')
also works.
One use case is when my_list is not actually a list but some other list-like object; in which case explicitly casting to list beforehand might be helpful.
arr = np.array(list(my_list), dtype='float32')
For example,
my_list = pd.Series([[1], [2], [3]]).values
np.array(my_list) # jagged array; not OK
np.array(list(my_list)) # OK
np.array(my_list.tolist()) # OK
my_list = {'a': [1], 'b': [2]}.values()
np.array(my_list) # jagged array; not OK
np.array(list(my_list)) # OK
To cast nested list into an array, the shapes of the sublists must match. If they don't maybe you want to concatenate the sublists along some axis. Try np.concatenate/np.r_/np.c_ etc. instead.
python - How to create a numpy array of lists? - Stack Overflow
How do I convert a list to a numpy array?
python - List of lists into numpy array - Stack Overflow
Is it ever advantageous to use a standard Python list vs a numpy array when all elements are the same type?
As you discovered, np.array tries to create a 2d array when given something like
A = np.array([[1,2],[3,4]],dtype=object)
You have apply some tricks to get around this default behavior.
One is to make the sublists variable in length. It can't make a 2d array from these, so it resorts to the object array:
In [43]: A=np.array([[1,2],[],[1,2,3,4]])
In [44]: A
Out[44]: array([[1, 2], [], [1, 2, 3, 4]], dtype=object)
And you can then append values to each of those lists:
In [45]: for i in A: i.append(34)
In [46]: A
Out[46]: array([[1, 2, 34], [34], [1, 2, 3, 4, 34]], dtype=object)
np.empty also creates an object array:
In [47]: A=np.empty((3,),dtype=object)
In [48]: A
Out[48]: array([None, None, None], dtype=object)
But you then have to be careful how you change the elements to lists. np.fill is tempting, but has problems:
In [49]: A.fill([])
In [50]: A
Out[50]: array([[], [], []], dtype=object)
In [51]: for i in A: i.append(34)
In [52]: A
Out[52]: array([[34, 34, 34], [34, 34, 34], [34, 34, 34]], dtype=object)
It turns out that fill puts the same list in all slots, so modifying one modifies all the others. You can get the same problem with a list of lists:
In [53]: B=[[]]*3
In [54]: B
Out[54]: [[], [], []]
In [55]: for i in B: i.append(34)
In [56]: B
Out[56]: [[34, 34, 34], [34, 34, 34], [34, 34, 34]]
The proper way to initial the empty A is with an iteration, e.g.
In [65]: A=np.empty((3,),dtype=object)
In [66]: for i,v in enumerate(A): A[i]=[v,i]
In [67]: A
Out[67]: array([[None, 0], [None, 1], [None, 2]], dtype=object)
In [68]: for v in A: v.append(34)
In [69]: A
Out[69]: array([[None, 0, 34], [None, 1, 34], [None, 2, 34]], dtype=object)
It's a little unclear from the question and comments whether you want to append to the lists, or append lists to the array. I've just demonstrated appending to the lists.
There is an np.append function, which new users often misuse. It isn't a substitute for list append. It is a front end to np.concatenate. It is not an in-place operation; it returns a new array.
Also defining a list to add with it can be tricky:
In [72]: np.append(A,[[1,23]])
Out[72]: array([[None, 0, 34], [None, 1, 34], [None, 2, 34], 1, 23], dtype=object)
You need to construct another object array to concatenate to the original, e.g.
In [76]: np.append(A,np.empty((1,),dtype=object))
Out[76]: array([[None, 0, 34], [None, 1, 34], [None, 2, 34], None], dtype=object)
In all of this, an array of lists is harder to construct than a list of lists, and no easier, or faster, to manipulate. You have to make it a 2d array of lists to derive some benefit.
In [78]: A[:,None]
Out[78]:
array([[[None, 0, 34]],
[[None, 1, 34]],
[[None, 2, 34]]], dtype=object)
You can reshape, transpose, etc an object array, where as creating and manipulating a list of lists of lists gets more complicated.
In [79]: A[:,None].tolist()
Out[79]: [[[None, 0, 34]], [[None, 1, 34]], [[None, 2, 34]]]
===
As shown in https://stackoverflow.com/a/57364472/901925, np.frompyfunc is a good tool for creating an array of objects.
np.frompyfunc(list, 0, 1)(np.empty((3,2), dtype=object))
If you really need a 1-d array of lists you will have to wrap your lists in your own class as numpy will always try to convert your lists to arrays inside of an array (which is more efficient but obviously requires constant size-elements), for example through
class mylist:
def __init__(self, l):
self.l=l
def __repr__(self):
return repr(self.l)
def append(self, x):
self.l.append(x)
and then you can change any element without changing the dimension of others
>>> x = mylist([1,2,3])
>>> y = mylist([1,2,3])
>>> import numpy as np
>>> data = np.array([x,y])
>>> data
array([[1,2,3], [1,2,3]], dtype=object)
>>> data[0].append(2)
>>> data
array([[1,2,3,2], [1,2,3]], dtype=object)
Update
As suggested by ali_m there is actually a way to force numpy to simply create a 1-d array for references and then feed them with actual lists
>>> data = np.empty(2, dtype=np.object)
>>> data[:] = [1, 2, 3], [1, 2, 3]
>>> data
array([[1, 2, 3], [1, 2, 3]], dtype=object)
>>> data[0].append(4)
>>> data
array([[1, 2, 3, 4], [1, 2, 3]], dtype=object)
The list I am working with has around 50 entries, I cannot simply do
Example:
list = [1,2,3,4,5,6,......50]
arr = numpy.array([ list ])
because it will place the entire list in 1 address. I am trying to use loops to append the values to the array but it is not working. Please help, thank you.
If your list of lists contains lists with varying number of elements then the answer of Ignacio Vazquez-Abrams will not work. Instead there are at least 3 options:
1) Make an array of arrays:
x=[[1,2],[1,2,3],[1]]
y=numpy.array([numpy.array(xi) for xi in x])
type(y)
>>><type 'numpy.ndarray'>
type(y[0])
>>><type 'numpy.ndarray'>
2) Make an array of lists:
x=[[1,2],[1,2,3],[1]]
y=numpy.array(x)
type(y)
>>><type 'numpy.ndarray'>
type(y[0])
>>><type 'list'>
3) First make the lists equal in length:
x=[[1,2],[1,2,3],[1]]
length = max(map(len, x))
y=numpy.array([xi+[None]*(length-len(xi)) for xi in x])
y
>>>array([[1, 2, None],
>>> [1, 2, 3],
>>> [1, None, None]], dtype=object)
>>> numpy.array([[1, 2], [3, 4]])
array([[1, 2], [3, 4]])