Using & operator, don't forget to wrap the sub-statements with ():
males = df[(df[Gender]=='Male') & (df[Year]==2014)]
To store your DataFrames in a dict using a for loop:
from collections import defaultdict
dic={}
for g in ['male', 'female']:
dic[g]=defaultdict(dict)
for y in [2013, 2014]:
dic[g][y]=df[(df[Gender]==g) & (df[Year]==y)] #store the DataFrames to a dict of dict
A demo for your getDF:
def getDF(dic, gender, year):
return dic[gender][year]
print genDF(dic, 'male', 2014)
Answer from zhangxaochen on Stack OverflowUsing & operator, don't forget to wrap the sub-statements with ():
males = df[(df[Gender]=='Male') & (df[Year]==2014)]
To store your DataFrames in a dict using a for loop:
from collections import defaultdict
dic={}
for g in ['male', 'female']:
dic[g]=defaultdict(dict)
for y in [2013, 2014]:
dic[g][y]=df[(df[Gender]==g) & (df[Year]==y)] #store the DataFrames to a dict of dict
A demo for your getDF:
def getDF(dic, gender, year):
return dic[gender][year]
print genDF(dic, 'male', 2014)
Start from pandas 0.13, this is the most efficient way.
df.query('Gender=="Male" & Year=="2014" ')
python - Filter Multiple Values using pandas - Stack Overflow
Pandas - Filter based on multiple conditions
python - Filter a column by multiple values - Stack Overflow
How to "pass through" multiple conditions in a pandas dataframe with query?
I'm new to Python. I learned C++ in college years ago and recently have been using VBA in excel. I've learned you can use Python to manipulate excel files and am trying to learn a new skill and streamline a weekly reporting requirement.
I'm trying to filter the data across multiple columns and multiple values in some columns.
So far I have:
OutputData = RawData [ (RawData['Column1']=='ABC') & (RawData['Column2']!='XYZ') ]
This works thus far, but how to I get:
Column3 == AAA or BBB
Also, how would I exclude values starting with CCC
.isin() works as well, more pythonic
country_list = ['brazil', 'poland', 'russia', 'countrydummy', 'usa']
filtered_df = df[df['Country Name'].isin(country_list)]
print(filtered_df)
You are missing a pair of parentheses to get comparable items on both sides of the | operator - which has higher precedence than ==:
df = df.loc[(df['Col 2'] == 'High') | (df['Col2'] == 'Medium')]
Hi,
I have a csv file with approx. 100 columns and I want to filter rows if two of the columns are set to a value of X and the other columns are blank / Nan values. In order to make the code more readable, I would like to specify the column names in a list and then use the variable name within the Pandas query e.g. something like the following:
my_file=/home/test.csv my_df=pd.read_csv(my_file) control_fields=['Is_Active','Is_Valid'] data_fields=['Age','DOB','Country','City']
As a starting point, I have tried the following but this isn't filtering the data at all:
my_new_df=my_df[my_df['control_fields']==1]
Can someone please explain why the above isn't working and also advise if there is a better way of achieving my requirement?
Thanks!
There are two ways to do this:
df[(df["value"]==2) | (df["value"]==4) | (df["value"]==5) | (df["value"]==9)]
OR
numbers = [2, 4, 5, 9]
df[df["value"].isin(numbers)]
1. For filtering single column you can use: df.loc[df['column_name'] == some_value]
df = pd.DataFrame({'A': 'foo bar foo bar foo bar foo foo'.split(),
'B': 'one one two three two two one three'.split(),
'C': np.random.randint(3, size=8), 'D':np.random.randint(6, size=8)})
df
#returns
# A B C D
#0 foo one 1 0
#1 bar one 1 4
#2 foo two 0 0
#3 bar three 2 5
#4 foo two 0 2
#5 bar two 2 4
#6 foo one 1 5
#7 foo three 1 0
df_filtered = df.loc[df['C'] == 1]
df_filtered #gives:
# A B C D
#0 foo one 1 0
#1 bar one 1 4
#6 foo one 1 5
#7 foo three 1 0
2. For filtering with more values of single column you can use the '|' operator (for multiple conditions): df.loc[(df['column_name'] >= A) | (df['column_name'] <= B)].
since you mention filter by single column values (the name of the single column you call 'values' which has different values 2, 4, 5, 9 for example), you can use this approach:
df_filtered = df.loc[(df['C'] == 1)| (df['C'] == 2)]
df_filtered #returns:
# A B C D
#0 foo one 1 0
#1 bar one 2 1
#2 foo two 1 1
#3 bar three 2 2
#4 foo two 1 1
#5 bar two 2 2
#6 foo one 2 2
3. You can even filter by multiple column values using the same approch:
df_filtered = df.loc[(df['C'] == 1)| (df['D'] == 2)]
df_filtered #gives:
# A B C D
#0 foo one 1 0
#2 foo two 1 1
#3 bar three 2 2
#4 foo two 1 1
#5 bar two 2 2
#6 foo one 2 2
# note that this time we filter all the df values corresponding to columns C == 1 & D == 2