Another option with DataFrame.where():
df[['C1', 'V1']].where(df.Cond == "X", df[['C2', 'V2']].values)
# C1 V1
#0 1 2
#1 7 8
#2 9 10
Answer from akuiper on Stack OverflowAnother option with DataFrame.where():
df[['C1', 'V1']].where(df.Cond == "X", df[['C2', 'V2']].values)
# C1 V1
#0 1 2
#1 7 8
#2 9 10
I try create more general solution with filter and numpy.where, for new column names use extract:
#if necessary sort columns
df = df.sort_index(axis=1)
#filter df by 1 and 2
df1 = df.filter(like='1')
df2 = df.filter(like='2')
print (df1)
C1 V1
0 1 2
1 5 6
2 9 10
print (df2)
C2 V2
0 3 4
1 7 8
2 11 12
#np.where need same shape of mask as df1 and df2
mask = pd.concat([df.Cond == 'X']*len(df1.columns), axis=1)
print (mask)
Cond Cond
0 True True
1 False False
2 True True
cols = df1.columns.str.extract('([A-Za-z])', expand=False)
print (cols)
Index(['C', 'V'], dtype='object')
print (np.where(mask, df1,df2))
Index(['C', 'V'], dtype='object')
[[ 1 2]
[ 7 8]
[ 9 10]]
print (pd.DataFrame(np.where(mask, df1, df2), index=df.index, columns=cols))
C V
0 1 2
1 7 8
2 9 10
For some dataframe X
p A p B p C
0 0 0 0
1 0 0 0
2 0 0 1
3 0 0 0
4 0 0 0
5 0 0 0
6 1 0 0
If you can set up the names of the columns you want to test for in col_list
col_list = X.columns
You can then use np.any() to test with or between each:
X.loc[(X[col_list] == 1).any(axis=1)]
Which gives you:
p A p B p C
2 0 0 1
6 1 0 0
Informed you don't need loc and will still get the same answer, credit to @MaartynFabre for the info
X[(X[col_list] == 1).any(axis=1)]
p A p B p C
2 0 0 1
6 1 0 0
test Dataframe
col0 col1 col2
0 1 1 2
1 1 1 1
2 2 2 2
make a new dataframe with the test for all columns
result_s = d.concat((df['col%i'%i] == 1 for i in range(3)), axis=1).all(axis=1)
results in
0 False
1 True
2 False
dtype: bool
if you do df[result_s] you get
col0 col1 col2
1 1 1 1
this selects the rows where all columns are ==1 If one of the is enough, change the .all() to .any
col0 col1 col2
0 1 1 2
1 1 1 1
You can use all with boolean indexing:
print ((df == 'something1').all(1))
0 True
1 False
2 True
3 False
4 False
dtype: bool
print (df[(df == 'something1').all(1)])
col1 col2
0 something1 something1
2 something1 something1
EDIT:
If need select only some columns you can use isin with boolean indexing for selecting desired columns and then use subset - df[cols]:
print (df)
col1 col2 col3
0 something1 something1 a
1 something2 something3 s
2 something1 something1 r
3 something2 something3 a
4 something1 something2 a
cols = df.columns[df.columns.isin(['col1','col2'])]
print (cols)
Index(['col1', 'col2'], dtype='object')
print (df[(df[cols] == 'something1').all(1)])
col1 col2 col3
0 something1 something1 a
2 something1 something1 r
Why not:
df[(df.col1 == 'something1') | (df.col2 == 'something1')]
outputs:
col1 col2
0 something1 something1
2 something1 something1
4 something1 something2
I think below should do it, but its elegance is up for debate.
new_df = old_df[((old_df['C1'] > 0) & (old_df['C1'] < 20)) & ((old_df['C2'] > 0) & (old_df['C2'] < 20)) & ((old_df['C3'] > 0) & (old_df['C3'] < 20))]
Shorter version:
In [65]:
df[(df>=0)&(df<=20)].dropna()
Out[65]:
Name C1 C2 C3
1 BBBB 12 1 10