What about:

import pandas

x = pandas.DataFrame({
    'x_1': [0, 1, 2, 3, 0, 1, 2, 500, ],},
    index=[0, 1, 2, 3, 4, 5, 6, 7])

x['x_1'].rolling(window=2).apply(lambda x: x.iloc[1] - x.iloc[0])

in general you can replace the lambda function with your own function. Note that in this case the first item will be NaN.

Update

Defining the following:

n_steps = 2
def my_fun(x):
    return x.iloc[-1] - x.iloc[0]

x['x_1'].rolling(window=n_steps).apply(my_fun)

you can compute the differences between values at n_steps.

Answer from Pierluigi on Stack Overflow
🌐
Pandas
pandas.pydata.org › docs › reference › api › pandas.DataFrame.diff.html
pandas.DataFrame.diff — pandas 3.0.6 documentation
Calculates the difference of a DataFrame element compared with another element in the DataFrame (default is element in previous row).
🌐
Medium
medium.com › @whyamit101 › understanding-pandas-rolling-f8f6d6796c07
Understanding Pandas Rolling. If you think you need to spend $2,000… | by why amit | Medium
February 26, 2025 - Here, for every rolling window of 4 elements, we get the difference between the highest and lowest value. As you can see, once the window is full, you get consistent results, and this might be exactly what you need for analyzing certain patterns ...
🌐
Scaler
scaler.com › home › topics › pandas › resampling, rolling calculations, and differencing in pandas
Resampling, Rolling Calculations, and Differencing in Pandas - Scaler Topics
May 4, 2023 - For the second column, we can find how the stock changes each day as it gives the difference between the closing price and the opening price. We can do complex statistical analysis of data on Pandas. This is especially useful for time series data. We can resample the data for better accuracy. Resampling is required when the data is not centered around the mean. For more statistical information, we can use the rolling() and diff() methods of DataFrame.
🌐
Pandas
pandas.pydata.org › docs › reference › api › pandas.DataFrame.rolling.html
pandas.DataFrame.rolling — pandas 3.0.6 documentation
For a DataFrame, a column label or Index level on which to calculate the rolling window, rather than the DataFrame’s index.
Top answer
1 of 2
7

Based your code (your groupby/apply), it looks like (despite your example ... but maybe I misunderstand what you want and then what Andy did would be the best idea) that you're working with a 'date' column that is a datetime64 dtype and not an integer dtype in your actual data. Also it looks like you want compute the change in days as measured from the first observation of a given group/stage. I think this is a better set of example data (if I understand your goal correctly):

>>> df

  group       date stage  dur
0     A 2014-01-01   one    0
1     A 2014-01-03   one    2
2     A 2014-01-04   one    3
3     A 2014-01-05   two    0
4     B 2014-01-02  four    0
5     B 2014-01-06  five    0
6     B 2014-01-10  five    4
7     C 2014-01-03   two    0
8     C 2014-01-05   two    2

Given that you should get some speed-up from just modifying your apply (as Jeff suggests in his comment) by dividing through by the timedelta64 in a vectorized way after the apply (or you could do it in the apply):

>>> df['dur'] = df.groupby(['group','stage']).date.apply(lambda x: x - x.iloc[0])
>>> df['dur'] /= np.timedelta64(1,'D')
>>> df

  group       date stage  dur
0     A 2014-01-01   one    0
1     A 2014-01-03   one    2
2     A 2014-01-04   one    3
3     A 2014-01-05   two    0
4     B 2014-01-02  four    0
5     B 2014-01-06  five    0
6     B 2014-01-10  five    4
7     C 2014-01-03   two    0
8     C 2014-01-05   two    2

But you can also avoid the groupby/apply given your data is in group,stage,date order. The first date for every ['group','stage'] grouping happens when either the group changes or the stage changes. So I think you can do something like the following:

>>> beg = (df.group != df.group.shift(1)) | (df.stage != df.stage.shift(1))
>>> df['dur'] = (df['date'] - df['date'].where(beg).ffill())/np.timedelta64(1,'D')
>>> df

  group       date stage  dur
0     A 2014-01-01   one    0
1     A 2014-01-03   one    2
2     A 2014-01-04   one    3
3     A 2014-01-05   two    0
4     B 2014-01-02  four    0
5     B 2014-01-06  five    0
6     B 2014-01-10  five    4
7     C 2014-01-03   two    0
8     C 2014-01-05   two    2

Explanation: Note what df['date'].where(beg) creates:

>>> beg = (df.group != df.group.shift(1)) | (df.stage != df.stage.shift(1))
>>> df['date'].where(beg)

0   2014-01-01
1          NaT
2          NaT
3   2014-01-05
4   2014-01-02
5   2014-01-06
6          NaT
7   2014-01-03
8          NaT

And then I ffill the values and take the difference with the 'date' column.

Edit: As Andy points out you could also use transform:

>>> df['dur'] = df.date - df.groupby(['group','stage']).date.transform(lambda x: x.iloc[0])
>>> df['dur'] /= np.timedelta64(1,'D')

  group       date stage  dur
0     A 2014-01-01   one    0
1     A 2014-01-03   one    2
2     A 2014-01-04   one    3
3     A 2014-01-05   two    0
4     B 2014-01-02  four    0
5     B 2014-01-06  five    0
6     B 2014-01-10  five    4
7     C 2014-01-03   two    0
8     C 2014-01-05   two    2

Speed: I timed the two method using a similar dataframe with 400,000 observations:

Apply method:

1 loops, best of 3: 18.3 s per loop

Non-apply method:

1 loops, best of 3: 1.64 s per loop

So I think avoiding the apply could give some significant speed-ups

2 of 2
6

I think I'd use diff here:

In [11]: df.groupby('stage')['date'].diff().fillna(0)
Out[11]:
0    0
1    2
2    0
3    0
4    0
5    4
dtype: float64

(Assuming that the stages are contiguous.)

If you are just subtracting the first in each group, use a transform:

In [21]: df['date'] - df.groupby('stage')['date'].transform(lambda x: x.iloc[0])
Out[21]:
0    0
1    2
2    0
3    0
4    0
5    4
Name: date, dtype: int64

Note: this is probably significantly faster...

🌐
Stack Overflow
stackoverflow.com › questions › 52104500 › rolling-difference-using-pandas
python - Rolling Difference using Pandas - Stack Overflow
August 31, 2018 - First, convert the month column to a Categorical (because alphabetically, December is before January, etc). Next, calculate Net_Items as the difference between Adds and Subtracts.
Find elsewhere
🌐
Iditect
iditect.com › faq › python › rolling-difference-in-pandas.html
Rolling difference in Pandas
Description: This query focuses on calculating the difference between consecutive rows within a rolling window in Pandas.
🌐
Bitcoden
bitcoden.com › answers › rolling-difference-in-pandas
Rolling difference in Pandas
Ideally the step size would be editable (i.e. difference between current time step and n last steps). I've also written this, but for larger arrays, it is quite slow: def roll_diff(values,step): diff = [] for i in np.arange(step, len(values)-1): pers_window = np.arange(i-1,i-step-1,-1) diff.append(np.abs(values[i] - np.mean(values[pers_window]))) diff = np.pad(diff, (0, step+1), 'constant') return diff ... import pandas x = pandas.DataFrame({ 'x_1': [0, 1, 2, 3, 0, 1, 2, 500, ],}, index=[0, 1, 2, 3, 4, 5, 6, 7]) x['x_1'].rolling(window=2).apply(lambda x: x.iloc[1] - x.iloc[0])
🌐
GeeksforGeeks
geeksforgeeks.org › python-pandas-dataframe-rolling
Python | Pandas dataframe.rolling() - GeeksforGeeks
February 21, 2022 - Python is a great language for ... importing and analyzing data much easier. Pandas dataframe.rolling() function provides the feature of rolling window calculations....
🌐
Medium
ujjwal-dalmia.medium.com › data-wrangling-solutions-rolling-calculations-in-pandas-de112a16f1c5
Data Wrangling Solutions — Rolling Calculations In Pandas | by Ujjwal Dalmia | Medium
November 9, 2023 - Data Wrangling Solutions — Rolling Calculations In Pandas A simple solution to calculate rolling difference & percentage change. One of the frequently used data pre-processing actions is to create …
🌐
GitHub
github.com › pola-rs › polars › issues › 5325
rolling method returns different results in Polars and Pandas · Issue #5325 · pola-rs/polars
October 24, 2022 - As demonstrated in the reproducible example i have provided, you will notice that rolling mean returns different results in Pandas and in Polars. In my specific case the results of machine learning accuracy is significantly higher when i am using Pandas rolling mean.
Author: pola-rs
🌐
Spark By {Examples}
sparkbyexamples.com › home › pandas › pandas rolling() mean, average, sum examples
Pandas rolling() Mean, Average, Sum Examples - Spark By {Examples}
October 14, 2024 - pandas.DataFrame.rolling() function can be used to get the rolling mean, average, sum, median, max, min e.t.c for one or multiple columns. Rolling mean is
🌐
Pandas
pandas.pydata.org › docs › reference › api › pandas.Series.diff.html
pandas.Series.diff — pandas 3.0.5 documentation - PyData |
Calculates the difference of a Series element compared with another element in the Series (default is element in previous row).
🌐
APXML
apxml.com › courses › time-series-analysis-forecasting › chapter-1-intro-time-series-data › shifting-lagging-rolling
Time Shifting, Lagging & Rolling Windows
This operation, known as differencing, helps stabilize the mean of a time series by removing trends or changes in level. It highlights the period-to-period changes rather than the absolute values. We will see its importance for achieving stationarity in Chapter 2. Pandas offers the .diff() ...