Transpose, then unpack:
>>> x, y, z = data.T
>>> x
array([1, 4, 7])
Transpose, then unpack:
>>> x, y, z = data.T
>>> x
array([1, 4, 7])
You don't need to slice it.
>>> import numpy as np
>>> data = np.array([[1,2,3],[4,5,6],[7,8,9]])
>>> x, y, z = data.T
>>> x
array([1, 4, 7])
>>> y
array([2, 5, 8])
>>> z
array([3, 6, 9])
You can unpack the rows and columns into a list with:
res1, res2 = [*a], [*a.T]
print(res1)
[array([5, 0, 3, 3]),
array([7, 9, 3, 5]),
array([2, 4, 7, 6]),
array([8, 8, 1, 6])]
print(res2)
[array([5, 7, 2, 8]),
array([0, 9, 4, 8]),
array([3, 3, 7, 1]),
array([3, 5, 6, 6])]
Extended iterable unpacking was introduced in python 3.0, for older versions you can call the list constructor as in @U9-Forward 's answer
As it seems you're on Python 2:
>>> l1, l2 = list(a), list(a.T)
>>> l1
[array([5, 0, 3, 3]), array([7, 9, 3, 5]), array([2, 4, 7, 6]), array([8, 8, 1, 6])]
>>> l2
[array([5, 7, 2, 8]), array([0, 9, 4, 8]), array([3, 3, 7, 1]), array([3, 5, 6, 6])]
>>>
python - Splitting each item in an array into a separate data frame column - Stack Overflow
python - split array rows into columns from commas - Stack Overflow
How to split array with 1 column into multiple columns (numpy array) - Stack Overflow
Separate large arrays into separate columns of pandas dataframe
Assuming the array is called points and numpy has already been imported:
newpoints = numpy.array([x.split(',') for x in points], dtype=numpy.float)
The elements in your array are strings rather than numbers. You can loop over each row in this (53,) array of strings, use split(',') to split each row at the commas, and put the result in a new numpy array with a numeric data type:
a = np.array(['1,2,3','4,5,6','7,8,9','10,11,12'])
b = np.array([l.split(',') for l in a],dtype=np.float32)
So I have a pandas dataframe which contains around 10 columns. Each of the 1000+ (we'll just label it 'z') elements of two of these columns is a list of n values, and when matched they form sets of (x, y) pairs. So those two columns look something like this:
| Column 1 | Column 2 | |
|---|---|---|
| 1 | (x0, x1, ..., xn) | (y0, y1, ..., yn) |
| ... | ... | ... |
| z | (x0, x1, ..., xn) | (y0, y1, ..., yn) |
I want to separate these columns into individual elements. In other words rather than just the two columns, I want a new dataframe to have 2n columns, one for each x and one for each y. I thought of using numpy's array_split(), but I'm not sure how to implement that in a large enough scale without hardcoding it, which isn't really an option. Is anyone aware of a way to go about this?
You can use the DataFrame constructor with lists created by to_list:
import pandas as pd
d1 = {'teams': [['SF', 'NYG'],['SF', 'NYG'],['SF', 'NYG'],
['SF', 'NYG'],['SF', 'NYG'],['SF', 'NYG'],['SF', 'NYG']]}
df2 = pd.DataFrame(d1)
print (df2)
teams
0 [SF, NYG]
1 [SF, NYG]
2 [SF, NYG]
3 [SF, NYG]
4 [SF, NYG]
5 [SF, NYG]
6 [SF, NYG]
df2[['team1','team2']] = pd.DataFrame(df2.teams.tolist(), index= df2.index)
print (df2)
teams team1 team2
0 [SF, NYG] SF NYG
1 [SF, NYG] SF NYG
2 [SF, NYG] SF NYG
3 [SF, NYG] SF NYG
4 [SF, NYG] SF NYG
5 [SF, NYG] SF NYG
6 [SF, NYG] SF NYG
And for a new DataFrame:
df3 = pd.DataFrame(df2['teams'].to_list(), columns=['team1','team2'])
print (df3)
team1 team2
0 SF NYG
1 SF NYG
2 SF NYG
3 SF NYG
4 SF NYG
5 SF NYG
6 SF NYG
A solution with apply(pd.Series) is very slow:
#7k rows
df2 = pd.concat([df2]*1000).reset_index(drop=True)
In [121]: %timeit df2['teams'].apply(pd.Series)
1.79 s ± 52.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
In [122]: %timeit pd.DataFrame(df2['teams'].to_list(), columns=['team1','team2'])
1.63 ms ± 54.3 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
Much simpler solution:
pd.DataFrame(df2["teams"].to_list(), columns=['team1', 'team2'])
Yields,
team1 team2
-------------
0 SF NYG
1 SF NYG
2 SF NYG
3 SF NYG
4 SF NYG
5 SF NYG
6 SF NYG
7 SF NYG
If you wanted to split a column of delimited strings rather than lists, you could similarly do:
pd.DataFrame(df["teams"].str.split('<delim>', expand=True).values,
columns=['team1', 'team2'])
This is a job for the * operator on the zip method.
>>> asdf
[[1, 2], [3, 4], [5, 6]]
>>> zip(*asdf)
[(1, 3, 5), (2, 4, 6)]
So in the context of your data it might be something like:
handle = open(file,'r')
lines = [line.split() for line in handle if line[:4] not in ('time', 'Step')]
Xvals, Yvals = zip(*lines)
or if your really need to be able to mutate the data afterwards you could just call the list constructor on each tuple:
Xvals, Yvals = [list(block) for block in zip(*lines)]
One way to do it is:
Xvals=[]; Yvals=[]
i = open(file,'r')
for line in i:
x, y = line.split(' ', 1)
Xvals.append(float(x))
Yvals.append(float(y))
print Xvals,Yvals
Note the call to the float function, which will change the string you get from the file into a number.
You can use zip to unzip helpful into separate columns:
df['helpful_numerator'], df['helpful_denominator'] = zip(*df['helpful'])
Edit
As mentioned by @MaxU in the comments, if you want to drop the helpful column from your DataFrame, use pop when selecting the column in zip:
df['helpful_numerator'], df['helpful_denominator'] = zip(*df.pop('helpful'))
Timings
Using the following setup to create a larger sample DataFrame and functions to time against:
df = pd.DataFrame({'A': list('abc'), 'B': [[0,1],[2,3],[4,5]]})
df = pd.concat([df]*10**5, ignore_index=True)
def root(df):
df['C'], df['D'] = zip(*df['B'])
return df
def maxu(df):
return df.join(pd.DataFrame(df.pop('B').tolist(), columns=['C', 'D']))
def flyingmeatball(df):
df['C'] = df['B'].apply(lambda x: x[0])
df['D'] = df['B'].apply(lambda x: x[1])
return df
def psidom(df):
df['C'] = df.B.str[0]
df['D'] = df.B.str[1]
return df
I get the following timings:
%timeit root(df.copy())
10 loops, best of 3: 70.6 ms per loop
%timeit maxu(df.copy())
10 loops, best of 3: 151 ms per loop
%timeit flyingmeatball(df.copy())
1 loop, best of 3: 223 ms per loop
%timeit psidom(df.copy())
1 loop, best of 3: 283 ms per loop
If helpful is a column of lists, you can use str to access the element in the list:
df['helpful_numerator'] = df.helpful.str[0]
df['helpful_denominator'] = df.helpful.str[1]
df
