in of a Series checks whether the value is in the index:
In [11]: s = pd.Series(list('abc'))
In [12]: s
Out[12]:
0 a
1 b
2 c
dtype: object
In [13]: 1 in s
Out[13]: True
In [14]: 'a' in s
Out[14]: False
One option is to see if it's in unique values:
In [21]: s.unique()
Out[21]: array(['a', 'b', 'c'], dtype=object)
In [22]: 'a' in s.unique()
Out[22]: True
or a python set:
In [23]: set(s)
Out[23]: {'a', 'b', 'c'}
In [24]: 'a' in set(s)
Out[24]: True
As pointed out by @DSM, it may be more efficient (especially if you're just doing this for one value) to just use in directly on the values:
In [31]: s.values
Out[31]: array(['a', 'b', 'c'], dtype=object)
In [32]: 'a' in s.values
Out[32]: True
Answer from Andy Hayden on Stack Overflowin of a Series checks whether the value is in the index:
In [11]: s = pd.Series(list('abc'))
In [12]: s
Out[12]:
0 a
1 b
2 c
dtype: object
In [13]: 1 in s
Out[13]: True
In [14]: 'a' in s
Out[14]: False
One option is to see if it's in unique values:
In [21]: s.unique()
Out[21]: array(['a', 'b', 'c'], dtype=object)
In [22]: 'a' in s.unique()
Out[22]: True
or a python set:
In [23]: set(s)
Out[23]: {'a', 'b', 'c'}
In [24]: 'a' in set(s)
Out[24]: True
As pointed out by @DSM, it may be more efficient (especially if you're just doing this for one value) to just use in directly on the values:
In [31]: s.values
Out[31]: array(['a', 'b', 'c'], dtype=object)
In [32]: 'a' in s.values
Out[32]: True
You can try this to check a particular value 'x' in a particular column named 'id'
if x in df['id'].values
How to Check if Any Value in a List Is in either of Two Columns With Pandas
python - Check if certain value is contained in a dataframe column in pandas - Stack Overflow
How do i check if a dataframe column contains lists?
How to test if all values in pandas dataframe column are equal?
Videos
I am loading a csv as a pandas dataframe.
The headers of the csv are: fname,lname,linkedin_url,work_email,personal_email,cell_phone,company_name,job_title,company_website,industry,location,gender,birth_day
I have a list of emails that I want to filter the dataframe with and keep the records that are in the list. I figured out how to do it based on one column using this:
datafilters = df['work_email'].isin(FilterThese)
How do I adjust the above so it checks if the values in the list "FilterThese" exist in either 'work_email' or 'personal_email'?
Thank you!
You can simply use this:
'07311954' in df.date.values which returns True or False
Here is the further explanation:
In pandas, using in check directly with DataFrame and Series (e.g. val in df or val in series ) will check whether the val is contained in the Index.
BUT you can still use in check for their values too (instead of Index)! Just using val in df.col_name.values
or val in series.values. In this way, you are actually checking the val with a Numpy array.
And .isin(vals) is the other way around, it checks whether the DataFrame/Series values are in the vals. Here vals must be set or list-like. So this is not the natural way to go for the question.
I think you need str.contains, if you need rows where values of column date contains string 07311954:
print df[df['date'].astype(str).str.contains('07311954')]
Or if type of date column is string:
print df[df['date'].str.contains('07311954')]
If you want check last 4 digits for string 1954 in column date:
print df[df['date'].astype(str).str[-4:].str.contains('1954')]
Sample:
print df['date']
0 8152007
1 9262007
2 7311954
3 2252011
4 2012011
5 2012011
6 2222011
7 2282011
Name: date, dtype: int64
print df['date'].astype(str).str[-4:].str.contains('1954')
0 False
1 False
2 True
3 False
4 False
5 False
6 False
7 False
Name: date, dtype: bool
print df[df['date'].astype(str).str[-4:].str.contains('1954')]
cmte_id trans_typ entity_typ state employer occupation date \
2 C00119040 24K CCM MD NaN NaN 7311954
amount fec_id cand_id
2 1000 C00140715 H2MD05155
I have a script that runs nightly extracting data from an API and cleaning it up into a dataframe. It has worked fine up until today when all of a sudden is failing with the following error:
TypeError: unhashable type: 'list'
The line of code that is throwing that error is
final_df.drop_duplicates(inplace=True)
A check on stack overflow says that lists in dataframes arent hashable hence the error. How can i find the offending column (or columns) in my dataframe and fix the error?