pd.to_datetime returns a Series of datetime64 dtype, as described here:
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.to_datetime.html
df['DATE'] = df['Date'].dt.date
or this:
df['Date'].map(datetime.datetime.date)
Answer from Jose on Stack Overflowpd.to_datetime returns a Series of datetime64 dtype, as described here:
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.to_datetime.html
df['DATE'] = df['Date'].dt.date
or this:
df['Date'].map(datetime.datetime.date)
You can use apply function on the dataframe column to convert the necessary column to String. For example:
df['DATE'] = df['Date'].apply(lambda x: x.strftime('%Y-%m-%d'))
Make sure to import datetime module.
apply() will take each cell at a time for evaluation and apply the formatting as specified in the lambda function.
QST: How to convert dtype from datetime64[ns] to datetime
python - convert datetime64[ns, UTC] pandas column to datetime - Stack Overflow
python - Keep only date part when using pandas.to_datetime - Stack Overflow
python - How to convert a Pandas data frame column from np.datetime64 to datetime? - Stack Overflow
To remove timezone, use tz_localize:
df['timestamp'] = pd.to_datetime(df.timestamp).dt.tz_localize(None)
Output:
timestamp
0 2020-07-09 04:23:50.267
1 2020-07-09 11:21:55.536
2 2020-07-09 11:23:18.015
3 2020-07-09 04:03:28.581
4 2020-07-09 04:03:33.874
Return of to_datetime depends [confusingly to me] on the type of input:
list-like: DatetimeIndex
Series: Series of datetime64 dtype
scalar: Timestamp
So the following fails
df["Time"] = pd.to_datetime(df["StringArray"])
xm = df["Time"] < pd.to_datetime("12/29/2020 9:09:37 PM")
but the following works just fine
df["Time"] = pd.to_datetime(df["StringArray"])
xm = df["Time"] < pd.to_datetime("12/29/2020 9:09:37 PM", utc=True)
This may help you avoid timezone problems. Regards,
Since version 0.15.0 this can now be easily done using .dt to access just the date component:
Copydf['just_date'] = df['dates'].dt.date
The above returns datetime.date, so object dtype. If you want to keep the dtype as datetime64 then you can just normalize:
Copydf['normalised_date'] = df['dates'].dt.normalize()
This sets the time component to midnight, i.e. 00:00:00, but the display shows just the date value.
pandas.Series.dt
Simple Solution:
Copydf['date_only'] = df['date_time_column'].dt.date
You can convert Series of dtype datetime64[ns] to a NumPy array of datetime.datetime objects by calling the .dt.to_pydatetime() method:
In [75]: df.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 252 entries, 0 to 251
Data columns (total 1 columns):
time 252 non-null datetime64[ns]<--the `time` column has dtype `datetime64[ns]`
dtypes: datetime64ns
memory usage: 2.0 KB
In [77]: df.head()
Out[77]:
time
0 2009-01-02
1 2009-01-05
2 2009-01-06
3 2009-01-07
4 2009-01-08
In [76]: df['time'].dt.to_pydatetime()[:5]
Out[76]:
array([datetime.datetime(2009, 1, 2, 0, 0),
datetime.datetime(2009, 1, 5, 0, 0),
datetime.datetime(2009, 1, 6, 0, 0),
datetime.datetime(2009, 1, 7, 0, 0),
datetime.datetime(2009, 1, 8, 0, 0)], dtype=object)
Note that NDFrames (such as Series and DataFrames) can only hold datetime-like objects as objects of dtype datetime64[ns]. The automatic conversion of all datetime-likes to a common dtype simplifies subsequent date computations. But it makes it impossible to store, say, Python datetime.datetime objects in a DataFrame column. Pandas core developer, Jeff Reback explains,
"We don't allow direct conversions because its simply too complicated to keep anything other than datetime64[ns] internally (nor necessary at all)."
Without your data set, I have to guess at some things. But, you should be able to repeat the same thing as what you demonstrated as having worked.
dt['datetime'] = datetime.utcfromtimestamp(dt['time'].values.astype(int)/1000000000))
For anyone who also stumbled across this when comparing a dataframe date to a variable date, and other answers did not exactly answer your question; you can use the code below.
Instead of:
self.df["date"] = pd.to_datetime(self.df["date"])
You can add .dt.date to the end like this:
self.df["date"] = pd.to_datetime(self.df["date"]).dt.date
You can use
pd.Timestamp('today')
or
pd.to_datetime('today')
But both of those give the date and time for 'now'.
Try this instead:
pd.Timestamp('today').floor('D')
or
pd.to_datetime('today').floor('D')
You could have also passed the datetime object to pandas.to_datetime but I like the other option more.
pd.to_datetime(datetime.datetime.today()).floor('D')
Pandas also has a Timedelta object
pd.Timestamp('now').floor('D') + pd.Timedelta(-3, unit='D')
Or you can use the offsets module
pd.Timestamp('now').floor('D') + pd.offsets.Day(-3)
To check for membership, try one of these
cur_date in df['date'].tolist()
Or
df['date'].eq(cur_date).any()