Increase measure the increase in your window, and rate is the 'per second' rate.

So if you were to imagine that Prometheus write a database row each second of what the current value is. That helps reason through the metrics calculations.

1. If the total so far was 0 then the metric increased by 5 in the last 10 seconds, the increase is looking back 10 seconds and finding the value and reporting the increase.

2. The same as 1. but it is looking back 30 seconds.

3. The 0.5 comes from looking at the window size (10s), measuring the increase them normalizing it to a 'per second' rate. So the increase of 5/10 seconds: .5 calls per second

4. With the larger window, the 'per second' is spread over the larger window.

increase is easier to reason about, but rate standardizes on the 'per second' unit.

One problem with the 'per second' unit though is that it can be too small to talk through, but it makes it easy to multiple by 60 (to show a per minute) or 3600 (to show a per hour). The window is only used to calculate the window for how far to gather data for the calculation.

increase() will always (approximately) double the actual increase with your setup.

The reason is that (as currently implemented):

1. increase() is (as you observed) syntactic sugar for rate() i.e. it is the value that would be returned by rate() multiplied by the number of seconds in the range you specified. In your case, it is rate() * 240.
2. rate() uses extrapolation in its computation. In the vast majority of cases a 4 minute range will return exactly 2 data points, almost exactly 2 minutes apart. The rate is then computed as the difference between last and first (i.e. the 2 points in your case) divided by the time difference of the 2 points (around 120 seconds in 99.99% of cases) multiplied by the range you requested (exactly 240 seconds). So if the increase between the 2 points is zero, the rate is zero. If the increase between the 2 points is 1.0, the computed rate() will be close to 2.0 / 240 and, as a result, the increase() will be 2.0.

This approach works mostly fine with counters that increase smoothly (e.g. if you have a more or less fixed number of signups every 2 minutes). But with a counter that rarely increases (as does your signups counter) or a spiky counter (like CPU usage) you get weird overestimates (like the increase of 2 you are seeing).

You can essentially reverse engineer Prometheus' implementation and get (something very close to) the actual increase by multiplying with (requested_range - scrape interval) and dividing by requested_range, essentially walking back the extrapolation that Prometheus does.

In your case, this would mean

increase(signups_count[4m]) * (240 - 120) / 240

or, more succinctly,

increase(signups_count[4m]) / 2

It requires you to be aware both of the length of the range and the scrape interval, but it will give you what you want: "ones for ones, and twos for twos, most of the time". Sometimes you'll get 1.01 instead of 1.0 because the scrapes were 119 seconds, not 120 seconds apart and sometimes, if your evaluation is closely aligned with the scrape some points right on the boundary might be included or not in a data point calculation, but it's still a better answer than 2.0.

Prometheus calculates rate(counter[d]) at timestamp t in the following way:

1. It selects raw samples for the counter time series on the time range (t-d ... t]. Note that the t-d timestamp isn't included in the time range, while t timestamp is included in the time range. If the selected time range contains less than two raw samples, then Prometheus returns an empty value (a gap) at the timestamp t.
2. Then it calculates the increase of the selected raw samples. Usually it is calculated as the difference between the last selected sample and the first selected sample. Calculations become slightly complicated if the counter was reset to zero during the selected time range. Let's skip this for the sake of clarity.
3. Then the resulting increase can be extrapolated if timestamps for the first and/or the last raw samples are located too far from the bounds of the selected time range.
4. Then the rate is calculated by dividing the extrapolated increase by d.

Prometheus calculates increase(counter[d]) in the same way except the last step.

Let's look at a few examples applied to the original data:

second   counter_value    increase calculated by hand(call it ICH from now)
1             1                    1
2             3                    2
3             6                    3
4             7                    1
5            10                    3
6            14                    4
7            17                    3
8            21                    4
9            25                    4
10           30                    5

• The rate(counter[1s]) will return nothing at any timestamp t, since any time range (t-1s ... t] contains only a single raw sample, while Prometheus requires at least two samples for calculating both rate() and increase().

• The rate(counter[2s]) and increase(counter[2]) would return the following values per each timestamp t when extrapolation isn't applied:

t       counter_value    rate(counter[2s])        increase(counter[2s])
1             1                    -                       -
2             3               (3-1)/2=1.0                3-1=2
3             6               (6-3)/2=1.5                6-3=3
4             7               (7-6)/2=0.5                7-6=1
5            10              (10-7)/2=1.5               10-7=3
6            14             (14-10)/2=2                14-10=4
7            17             (17-14)/2=1.5              17-14=3
8            21             (21-17)/2=2                21-17=4
9            25             (25-21)/2=2                25-21=4
10           30             (30-25)/2=2.5              30-25=5

In reality Prometheus results for rate(counter[2s]) and increase(counter[2s]) may be slightly bigger because of extrapolation, since the first sample on the selected time range is located comparatively far from the start of the time range.

Such calculations have the following issues:

• Prometheus can return fractional results from increase() over time series, which contains only integer values. This is because of extrapolation. For example, Prometheus may return fractional results from increase(http_requests_total[5m]).

• Prometheus returns empty results (aka gaps) from increase(counter[d]) and rate(counter[d]) when the lookbehind window d doesn't cover at least two samples - see rate(counter[1s]) and increase(counter[1s]) example above.

• Prometheus completely misses the increase between the raw sample just before the (t-d ... t] interval and the first raw sample on this interval. This may result in inaccurate calculations. For example, increase(counter[1h]) doesn't equal to sum_over_time(increase(counter[1m])[1h:1m]).

Prometheus developers are aware of these issues - see this link. These issues are addressed in the system I work on - VictoriaMetrics - more specifically, in MetricsQL query language - see this comment and this article for technical details.