For a more general answer (if you need to discard several columns):
import numpy
x = numpy.array(data)[:,range(0,6)+range(7,18)]
Answer from nicolas on Stack OverflowFor a more general answer (if you need to discard several columns):
import numpy
x = numpy.array(data)[:,range(0,6)+range(7,18)]
The numpy.delete function returns a new array with the specified columns deleted, along whichever axis you want. The following is equivalent to the first statement you posted above:
x = numpy.delete(data, 6, axis=1)
python - How to create an array from two columns in pandas - Stack Overflow
How can I assign multiple rows and columns of one array to the same rows and columns of another array in Python? - Stack Overflow
pandas - Python: How to assign elements of an array to multiple columns in DataFrame? - Stack Overflow
numpy - Putting multiple columns of data into one array, python - Stack Overflow
You can convert your dataframe in this way:
import pandas as pd
import numpy as np
df = pd.DataFrame({0:[[2387, 1098], [1873, 6792],], 1:[0,1]})
arr = np.array(df.loc[:,0].to_list())
df2 = pd.DataFrame({0:arr[:,0], 1:arr[:,1], 2:df.loc[:,1]})
print(df2)
The result is:
0 1 2
0 2387 1098 0
1 1873 6792 1
A second way to solve the problem (with a "moon" sample) is:
import sklearn
import sklearn.datasets
X, y = sklearn.datasets.make_moons()
pd.DataFrame({'x0':X[:,0], 'x1': X[:,1], 'y':y})
and the result is:
x0 x1 y
0 0.981559 0.191159 0
1 0.967948 -0.499486 1
2 0.018441 0.308841 1
3 -0.981559 0.191159 0
4 0.967295 0.253655 0
.. ... ... ..
95 0.238554 -0.148228 1
96 0.096023 0.995379 0
97 0.327699 -0.240278 1
98 0.900969 0.433884 0
99 1.981559 0.308841 1
[100 rows x 3 columns]
EDIT:
Maybe it looks strange but you can use .str[0] to get first column from lists in DataFrame.
import pandas as pd
df = pd.DataFrame({0:[[2387, 1098], [1873, 6792],], 1:[0,1]})
new_df = pd.DataFrame({
0: df[0].str[0],
1: df[0].str[1],
2: df[1]
})
print(new_df)
OLDER:
Using apply() with pandas.Series you can convert first column into new DataFrame with two columns
import pandas as pd
df = pd.DataFrame({0:[[2387, 1098], [1873, 6792],], 1:[0,1]})
new_df = df[0].apply(pd.Series)
print(new_df)
Result:
0 1
0 2387 1098
1 1873 6792
And later you can assing them back to old `DataFrame
df[2] = df[1] # move `[0,1,...]` to column 2
df[[0,1]] = new_df # put `new_df` in columns 0,1
Result:
0 1 2
0 2387 1098 0
1 1873 6792 1
Or you can copy column [0,1,...] from old df to new_df
import pandas as pd
df = pd.DataFrame({0:[[2387, 1098], [1873, 6792],], 1:[0,1]})
new_df = df[0].apply(pd.Series)
new_df[2] = df[1]
print(new_df)
You can access the underlying numpy array via the to_numpy method:
df[['col1', 'col2']].to_numpy()
Out:
array([[0, 1],
[2, 3],
[4, 5]])
.values attribute will do the same if you are on an earlier version (before v0.24).
You can also achieve the same output with the below code.
import numpy as np
np.array(df[['col1','col2']])
Out[60]:
array([[0, 1],
[2, 3],
[4, 5]])
The error is telling you that you are trying to assign a sequence of values to a sequence of keys. You can use the zip function to create a dictionary from the two lists. Then you can use the dictionary to assign the values to the columns.
This is because the return value of apply is a pd.Series which comes with 1 column while you try to assign it to 5. See this example:
import numpy as np
import pandas as pd
def do_something(row, args):
arr = np.array([1,2,3,4,5])
return arr
df = pd.DataFrame(np.ones((2,6)),columns = ['A', 'B', 'C', 'D', 'E', 'some other column'])
df.apply(lambda row: do_something(row, 2), axis=1)
0 [1, 2, 3, 4, 5]
1 [1, 2, 3, 4, 5]
dtype: object
The solution is to convert it to 5 columns so you could do the assign later:
df[['A', 'B', 'C', 'D', 'E']] = df.apply(lambda row: do_something(row, 2), axis=1).to_list()
You might use python's built-in zip for that following way:
import pandas as pd
arrayA = ['f','d','g']
arrayB = ['1','2','3']
arrayC = [4,5,6]
df = pd.DataFrame(zip(arrayA, arrayB, arrayC), columns=['AA','NN','gg'])
print(df)
Output:
AA NN gg
0 f 1 4
1 d 2 5
2 g 3 6
Zip is a great solution in this case as pointed out by Daweo, but alternatively you can use a dictionary for readability purposes:
import pandas as pd
arrayA = ['f','d','g']
arrayB = ['1','2','3']
arrayC = [4,5,6]
my_dict = {
'AA': arrayA,
'NN': arrayB,
'gg': arrayC
}
df = pd.DataFrame(my_dict)
print(df)
Output
AA NN gg
0 f 1 4
1 d 2 5
2 g 3 6
>>> import numpy as np
>>> A = np.array([[1,2,3,4],[5,6,7,8]])
>>> A
array([[1, 2, 3, 4],
[5, 6, 7, 8]])
>>> A[:,2] # returns the third columm
array([3, 7])
See also: "numpy.arange" and "reshape" to allocate memory
Example: (Allocating a array with shaping of matrix (3x4))
nrows = 3
ncols = 4
my_array = numpy.arange(nrows*ncols, dtype='double')
my_array = my_array.reshape(nrows, ncols)
Could it be that you're using a NumPy array? Python has the array module, but that does not support multi-dimensional arrays. Normal Python lists are single-dimensional too.
However, if you have a simple two-dimensional list like this:
A = [[1,2,3,4],
[5,6,7,8]]
then you can extract a column like this:
def column(matrix, i):
return [row[i] for row in matrix]
Extracting the second column (index 1):
>>> column(A, 1)
[2, 6]
Or alternatively, simply:
>>> [row[1] for row in A]
[2, 6]
Does something like this work?
import numpy as np
x = np.array([[1,2],[3,4]])
y = np.array([[5,6],[7,8]])
result
>>> x
array([[1, 2],
[3, 4]])
>>> y
array([[5, 6],
[7, 8]])
>>> x[:,1] + y[:,1]
array([ 8, 12])
>>> x[:, 1] += y[:, 1] # using +=
>>> x[:, 1]
array([ 8, 12])
Update:
I think this should work for you:
src = np.array([["a", "b"], ["c", "d"], ["e", "f"]], dtype='|S8')
src2 = np.array([["x"], ["y"], ["z"]], dtype='|S8')
def add_columns(x, y):
return [a + b for a,b in zip(x, y)]
def update_array(source_array, col_num, add_col):
temp_col = add_columns(source_array[:, col_num], add_col)
source_array[:, col_num] = temp_col
return source_array
Result:
>>> update_array(src, 1, src2[:,0])
array([['a', 'bx'],
['c', 'dy'],
['e', 'fz']],
dtype='|S8')
When you need to debug this kind of thing, it's useful to break it down into simpler steps. Are you getting the slices wrong, adding two incompatible array types, adding two types but trying to stick the results into an incompatible type (using += when + is OK but = is not), or adding incompatible data values? Any one of those could raise a TypeError, so how do we know which one you're doing?
Well, just do them at a time and see:
Slicing:
>>> src[:, 1]
array(['b', 'd', 'f'], dtype='|S1')
>>> src[:, 1] = ['x', 'y', 'z']
>>> src
>>> array([['a', 'x'], ['c', 'y'], ['e', 'z']], dtype='|S1')
That's fine. What about adding?
>>> src + src2
TypeError: unsupported operand type(s) for +: 'numpy.ndarray' and 'numpy.ndarray'
So, we've already found the same error as your more complicated case, without the slicing, and without the +=, which makes things much easier to debug. Let's make it even simpler:
>>> s1, s2 = np.array('a'), np.array('b')
>>> s1 + s2
TypeError: unsupported operand type(s) for +: 'numpy.ndarray' and 'numpy.ndarray'
So even adding two 0D arrays fails! Can't get any simpler than that.
Maybe it's the data types. What happens if we use integers?
>>> n1, n2 = np.array(1), np.array(2)
>>> n1 + n2
3
And you can go all the way back to your original example, using integers instead of strings, and it still works fine:
>>> m1 = np.array([[1,2], [3,4], [5,6]])
>>> m2 = np.array([[7], [8], [9]])
>>> m1[:, 1] += m2[:, 0]
>>> array([[ 1, 9],
[ 3, 12],
[ 5, 15]])
That should make it obvious that the problem is with data types. So, what is the data type? Just print out the array and see what numpy thinks it is:
>>> src = numpy.array([["a", "b"], ["c", "d"], ["e", "f"]])
>>> src
array([['a', 'b'], ['c', 'd'], ['e', 'f']], dtype='|S1')
That '|S1' isn't one of the friendly data types you see in the User Guide section on Data types, it's a structure definition, as explained in the section on Structured arrays. What it means is a 1-character fixed length string.
And that makes the problem obvious: You can't add two 1-character fixed-length strings, because the result isn't a 1-character fixed-length string.
If you really want to make this work as-is, the simple solution is to leave them as Python strings:
>>> src = numpy.array([["a", "b"], ["c", "d"], ["e", "f"]], dtype=object)
>>> src2 = numpy.array([["x"], ["y"], ["z"]], dtype=object)
>>> src[:, 1] += src2[:, 0]
No more TypeError.
Alternatively, if you explicitly give src a dtype of |S2, numpy will allow that, and the second character will just be blank. It won't let you add another |S1 into it, but you can loop in Python, or maybe find a complicated way to fix numpy into doing it for you. Either way, you're not getting any of the usual time performance benefits of numpy of course, but you are still getting the space performance benefits of using packed fixed-size cells.
But you might want to step back and ask what you're trying to get out of numpy here. What is your actual higher-level goal here? Most of the benefit of numpy comes from using strict C/Fortran-style data types that numpy knows how to work with—it can pack them in tightly, access them without an extra dereference (and without refcounting), operate on in various ways from multiplying to copying to printing without any help from Python, etc. But it can't do string manipulation. If you're trying to vectorize string manipulation, you're using the wrong library to do it. If you're just using numpy because someone said it's fast, well, that's true in many cases, but not in this one. If you're using numpy because some other code is handing you numpy data, but you don't want to treat it in a numpy way, there's nothing stopping you from converting it to pure Python data.
Im currently trying to learn how to code and im learning with Python. I'm on arrays but I dont understand how to combine lists like this. What im doing just tacks on the second list to the end making one really long list thats 1 column. Also what is this process called when combining the two lists into seperate columns in one array?