Use to_datetime. There is no need to specify the format in this case since the parser is able to figure it out.
In [51]:
pd.to_datetime(df['I_DATE'])
Out[51]:
0 2012-03-28 14:15:00
1 2012-03-28 14:17:28
2 2012-03-28 14:50:50
Name: I_DATE, dtype: datetime64[ns]
To access the date/day/time component use the dt accessor:
In [54]:
df['I_DATE'].dt.date
Out[54]:
0 2012-03-28
1 2012-03-28
2 2012-03-28
dtype: object
In [56]:
df['I_DATE'].dt.time
Out[56]:
0 14:15:00
1 14:17:28
2 14:50:50
dtype: object
You can use strings to filter as an example:
In [59]:
df = pd.DataFrame({'date':pd.date_range(start = dt.datetime(2015,1,1), end = dt.datetime.now())})
df[(df['date'] > '2015-02-04') & (df['date'] < '2015-02-10')]
Out[59]:
date
35 2015-02-05
36 2015-02-06
37 2015-02-07
38 2015-02-08
39 2015-02-09
Answer from EdChum on Stack Overflowpython - How to convert string to datetime format in pandas? - Stack Overflow
Converting string to datetime - when the year is only three digits.
python - Convert DataFrame column type from string to datetime - Stack Overflow
python - datetime to string with series in pandas - Stack Overflow
Videos
Use to_datetime. There is no need to specify the format in this case since the parser is able to figure it out.
In [51]:
pd.to_datetime(df['I_DATE'])
Out[51]:
0 2012-03-28 14:15:00
1 2012-03-28 14:17:28
2 2012-03-28 14:50:50
Name: I_DATE, dtype: datetime64[ns]
To access the date/day/time component use the dt accessor:
In [54]:
df['I_DATE'].dt.date
Out[54]:
0 2012-03-28
1 2012-03-28
2 2012-03-28
dtype: object
In [56]:
df['I_DATE'].dt.time
Out[56]:
0 14:15:00
1 14:17:28
2 14:50:50
dtype: object
You can use strings to filter as an example:
In [59]:
df = pd.DataFrame({'date':pd.date_range(start = dt.datetime(2015,1,1), end = dt.datetime.now())})
df[(df['date'] > '2015-02-04') & (df['date'] < '2015-02-10')]
Out[59]:
date
35 2015-02-05
36 2015-02-06
37 2015-02-07
38 2015-02-08
39 2015-02-09
Approach: 1
Given original string format: 2019/03/04 00:08:48
you can use
updated_df = df['timestamp'].astype('datetime64[ns]')
The result will be in this datetime format: 2019-03-04 00:08:48
Approach: 2
updated_df = df.astype({'timestamp':'datetime64[ns]'})
Hi, so I'm working on a genealogy project. I have ancestors back in the 400s onward.
I've imported them as a csv using pandas and need to convert just the years to datetime.
import pandas as pd
df = pd.read_csv('final_ancestry.csv')
df.Year1=df.Year1.astype(str)
df.Year2=df.Year2.astype(str)
pd.to_datetime(df.Year1, format='%Y')I keep getting "ValueError: time data '406' does not match format '%Y' (match)". I know the format is YYYY, but in the csv, I actually put it as 0406, etc. My next thought is to save it as a txt file and import the file that way, but is there something I'm missing?
I also don't know if I really need to convert to string, but it wasn't working as the INT64 that it imported as, so I thought I'd try string.
The easiest way is to use to_datetime:
df['col'] = pd.to_datetime(df['col'])
It also offers a dayfirst argument for European times (but beware this isn't strict).
Here it is in action:
In [11]: pd.to_datetime(pd.Series(['05/23/2005']))
Out[11]:
0 2005-05-23 00:00:00
dtype: datetime64[ns]
You can pass a specific format:
In [12]: pd.to_datetime(pd.Series(['05/23/2005']), format="%m/%d/%Y")
Out[12]:
0 2005-05-23
dtype: datetime64[ns]
If your date column is a string of the format '2017-01-01' you can use pandas astype to convert it to datetime.
df['date'] = df['date'].astype('datetime64[ns]')
or use datetime64[D] if you want Day precision and not nanoseconds
print(type(df['date'].iloc[0]))
yields
<class 'pandas._libs.tslib.Timestamp'>
the same as when you use pandas.to_datetime
You can try it with other formats then '%Y-%m-%d' but at least this works.
There is no .str accessor for datetimes and you can't do .astype(str) either.
Instead, use .dt.strftime:
>>> series = pd.Series(['20010101', '20010331'])
>>> dates = pd.to_datetime(series, format='%Y%m%d')
>>> dates.dt.strftime('%Y-%m-%d')
0 2001-01-01
1 2001-03-31
dtype: object
See the docs on customizing date string formats here: strftime() and strptime() Behavior.
For old pandas versions <0.17.0, one can instead can call .apply with the Python standard library's datetime.strftime:
>>> dates.apply(lambda x: x.strftime('%Y-%m-%d'))
0 2001-01-01
1 2001-03-31
dtype: object
As of pandas version 0.17.0, you can format with the dt accessor:
dates.dt.strftime('%Y-%m-%d')