There's no increases there, the time series appear with the value 1. If they had increased from 0 to 1 rather than nothing to 1 then increase would show a non-zero value. I'd recommend initialising your metrics with the label values you know about.

Answer from brian-brazil on Stack Overflow
🌐
Google Groups
groups.google.com › g › prometheus-users › c › GLKaSYQN_gI › m › OGqeuNjhEAAJ
rate() occasionally returns zeroes on small time window change
You still have to fix the interval, but you do it in the Prometheus config, close to where you define your scrape and eval intervals. ... xrate() proposal is that because evaluation doesn't happen exactly every 10 seconds, it is still possible for some data points collected right around the time of an eval to be included in 2 successive rate calculations (or none).
Discussions

rate()[1m] does not return any data
What did you do? Query single metric for last 10 minutes. Metric is coming from cadvisor (kubelet build-in) used prometheus-query for CLI Query the same metric using rate() function with range vect... More on github.com
🌐 github.com
4
September 20, 2017
prometheus - Why is increase() showing only zero values when I can see the metric value increasing? - Stack Overflow
I am using Grafana to visualise a Prometheus time-series. When I simply set my stacked graph to visualise my_metric I get this: If I change to increase(my_metric[1h]) I get all zeroes: Everything... More on stackoverflow.com
🌐 stackoverflow.com
Do I understand Prometheus's rate vs increase functions correctly? - Stack Overflow
If the selected time range contains ... then Prometheus returns an empty value (a gap) at the timestamp t. Then it calculates the increase of the selected raw samples. Usually it is calculated as the difference between the last selected sample and the first selected sample. Calculations become slightly complicated if the counter was reset to zero during the ... More on stackoverflow.com
🌐 stackoverflow.com
Prometheus query expression 0/0 and 1-0/0 returns 0
Entering the expression 0/0 directly into explore view for prometheus queries (or in a panel) returns a time series with all zeroes. Entering the expression 1 - (0/0) unfortunately also returns zeroes. This causes problems with defaulting for SLI queries when traffic is low. E.g.: 1 - (( sum(rate(... More on github.com
🌐 github.com
5
November 27, 2022
🌐
DoiT
doit.com › home › blog › making peace with prometheus rate()
Making peace with Prometheus rate() | DoiT
February 17, 2023 - You see, depending on how stars align if our 1-minute range buckets land just on metric change boundaries, there is no change in the metric within that bucket at all and this is exactly why rate(), together with its sister increase(), returns zeroes.
🌐
GitHub
github.com › prometheus › prometheus › issues › 3194
rate()[1m] does not return any data · Issue #3194 · prometheus/prometheus
September 20, 2017 - What did you do? Query single metric for last 10 minutes. Metric is coming from cadvisor (kubelet build-in) used prometheus-query for CLI Query the same metric using rate() function with range vector of 1m What did you expect to see? Rat...
Author: prometheus
🌐
Prometheus
prometheus.io › docs › prometheus › latest › querying › functions
Query functions | Prometheus
Otherwise rate() cannot detect counter resets when your target restarts. For each input time series, resets(v range-vector) returns the number of counter resets within the provided time range as an instant vector. Any decrease in the value between two consecutive float samples is interpreted as a counter reset. A reset in a native histogram is detected in a more complex way: Any decrease in any bucket, including the zero ...
Top answer
1 of 3
273

In an ideal world (where your samples' timestamps are exactly on the second and your rule evaluation happens exactly on the second) rate(counter[1s]) would return exactly your ICH value and rate(counter[5s]) would return the average of that ICH and the previous 4. Except the ICH at second 1 is 0, not 1, because no one knows when your counter was zero: maybe it incremented right there, maybe it got incremented yesterday, and stayed at 1 since then. (This is the reason why you won't see an increase the first time a counter appears with a value of 1 -- because your code just created and incremented it.)

increase(counter[5s]) is exactly rate(counter[5s]) * 5 (and increase(counter[2s]) is exactly rate(counter[2s]) * 2).

Now what happens in the real world is that your samples are not collected exactly every second on the second and rule evaluation doesn't happen exactly on the second either. So if you have a bunch of samples that are (more or less) 1 second apart and you use Prometheus' rate(counter[1s]), you'll get no output. That's because what Prometheus does is it takes all the samples in the 1 second range [now() - 1s, now()] (which would be a single sample in the vast majority of cases), tries to compute a rate and fails.

If you query rate(counter[5s]) OTOH, Prometheus will pick all the samples in the range [now() - 5s, now] (5 samples, covering approximately 4 seconds on average, say [t1, v1], [t2, v2], [t3, v3], [t4, v4], [t5, v5]) and (assuming your counter doesn't reset within the interval) will return (v5 - v1) / (t5 - t1). I.e. it actually computes the rate of increase over ~4s rather than 5s.

increase(counter[5s]) will return (v5 - v1) / (t5 - t1) * 5, so the rate of increase over ~4 seconds, extrapolated to 5 seconds.

Due to the samples not being exactly spaced, both rate and increase will often return floating point values for integer counters (which makes obvious sense for rate, but not so much for increase).

2 of 3
50

Prometheus calculates rate(counter[d]) at timestamp t in the following way:

  1. It selects raw samples for the counter time series on the time range (t-d ... t]. Note that the t-d timestamp isn't included in the time range, while t timestamp is included in the time range. If the selected time range contains less than two raw samples, then Prometheus returns an empty value (a gap) at the timestamp t.
  2. Then it calculates the increase of the selected raw samples. Usually it is calculated as the difference between the last selected sample and the first selected sample. Calculations become slightly complicated if the counter was reset to zero during the selected time range. Let's skip this for the sake of clarity.
  3. Then the resulting increase can be extrapolated if timestamps for the first and/or the last raw samples are located too far from the bounds of the selected time range.
  4. Then the rate is calculated by dividing the extrapolated increase by d.

Prometheus calculates increase(counter[d]) in the same way except the last step.

Let's look at a few examples applied to the original data:

second   counter_value    increase calculated by hand(call it ICH from now)
1             1                    1
2             3                    2
3             6                    3
4             7                    1
5            10                    3
6            14                    4
7            17                    3
8            21                    4
9            25                    4
10           30                    5
  • The rate(counter[1s]) will return nothing at any timestamp t, since any time range (t-1s ... t] contains only a single raw sample, while Prometheus requires at least two samples for calculating both rate() and increase().

  • The rate(counter[2s]) and increase(counter[2]) would return the following values per each timestamp t when extrapolation isn't applied:

t       counter_value    rate(counter[2s])        increase(counter[2s])
1             1                    -                       -
2             3               (3-1)/2=1.0                3-1=2
3             6               (6-3)/2=1.5                6-3=3
4             7               (7-6)/2=0.5                7-6=1
5            10              (10-7)/2=1.5               10-7=3
6            14             (14-10)/2=2                14-10=4
7            17             (17-14)/2=1.5              17-14=3
8            21             (21-17)/2=2                21-17=4
9            25             (25-21)/2=2                25-21=4
10           30             (30-25)/2=2.5              30-25=5

In reality Prometheus results for rate(counter[2s]) and increase(counter[2s]) may be slightly bigger because of extrapolation, since the first sample on the selected time range is located comparatively far from the start of the time range.

Such calculations have the following issues:

  • Prometheus can return fractional results from increase() over time series, which contains only integer values. This is because of extrapolation. For example, Prometheus may return fractional results from increase(http_requests_total[5m]).

  • Prometheus returns empty results (aka gaps) from increase(counter[d]) and rate(counter[d]) when the lookbehind window d doesn't cover at least two samples - see rate(counter[1s]) and increase(counter[1s]) example above.

  • Prometheus completely misses the increase between the raw sample just before the (t-d ... t] interval and the first raw sample on this interval. This may result in inaccurate calculations. For example, increase(counter[1h]) doesn't equal to sum_over_time(increase(counter[1m])[1h:1m]).

Prometheus developers are aware of these issues - see this link. These issues are addressed in the system I work on - VictoriaMetrics - more specifically, in MetricsQL query language - see this comment and this article for technical details.

🌐
MetricFire
metricfire.com › blog › understanding-the-prometheus-rate-function
How the Prometheus rate() function works | MetricFire
March 12, 2026 - You can apply rate() to specific dimensions, making monitoring error rates for different backends useful. Before we discuss Prometheus functions too deeply, we first need to discuss some basic concepts and terminology.
Find elsewhere
🌐
GitHub
github.com › grafana › grafana › issues › 59349
Prometheus query expression 0/0 and 1-0/0 returns 0 · Issue #59349 · grafana/grafana
November 27, 2022 - 1 - (( sum(rate(http_server_sl... up showing zero during zero traffic scenarios. In prometheus, when omitting the > 0 the query does not return a value....
Author: grafana
🌐
Google Groups
groups.google.com › g › prometheus-users › c › uhJ9UUAF3Hg
First value of increase(Counter[Time]) doesn't count
July 4, 2023 - In general, you can't assume that counters start at zero. The counter may have been running and incrementing for a long time, before Prometheus starts scraping it; the first value you see could represent days or years of accumulation. That's why rate() and increase() only give a value if there are two or more adjacent data points.
🌐
GitHub
github.com › grafana › grafana › issues › 54857
Prometheus returning zero for some queries instead of NaN · Issue #54857 · grafana/grafana
September 7, 2022 - There was a breaking change in v9.0.0 that Prometheus would only return NaN but for some reason, zero is being returned in the following query. sum(avg_over_time(cluster_namespace_service_slo:successful_requests_total:rate5m{cluster="ops-us-east-0", namespace=~".+", service="cortex", slo="cardinality"}[$__rate_interval])) / sum(avg_over_time(cluster_namespace_service_slo:requests_total:rate5m{cluster="ops-us-east-0", namespace=~".+", service="cortex", slo="cardinality"}[$__rate_interval])) Original thread: https://raintank-corp.slack.com/archives/CPXKS6FB5/p1662559567729429 ·
Author: grafana
Top answer
1 of 3
34

AFAICT the cause for the weird results is (1) the fact that your counter actually only increases once every minute, even though you collect it every 15 seconds combined with (2) Prometheus' rate() implementation discarding every 4th counter increase (in your particular setup).

More precisely, you appear to be computing a 1 minute rate, every 1 minute over a counter scraped at 15 second resolution, increasing every 1 minute (on average).

What this means essentially is that Prometheus will basically slice your 1 hour interval into disjoint 1 minute ranges and estimate the rate over each range. The first value will be the extrapolated rate of increase between points 0 and 3, the second will be the extrapolated rate between points 4 and 7 and so on. Because your counter only actually increases once a minute, you can run into 2 different situations:

  1. Your counter increases happen between point pairs 3-4, 7-8 etc. In this case Prometheus sees an increase rate of zero (because there is no increase between points 0 and 3, points 4 and 7 etc. This seems to be happening in the first half of your first graph.
  2. Your counter increases happen somewhere between points 0-3, 4-7 etc. In this case Prometheus takes the difference between the last and first points in each interval (your actual counter increase), divides it by the time difference between the 2 points (on average 45 seconds), then extrapolates that to 1 minute (essentially overestimating it by a factor of 1.(3) -- I'm eyeballing an increase of ~200k over ~50 minutes, so an average rate of about 67 QPS, whereas rate() returns something closer to 90 QPS). This is what happens in the second half of your graph.

This is also why your graph looks wildly different across refreshes. The argument for the current implementation of rate() is that it is "correct on average". Which, if you look at the whole of your graph, across refreshes, is true. </sarcasm>

Essentially graphing a Prometheus rate() or increase() over a time range R with resolution R will result in aliasing, either overestimating (1.33x in your case) or underestimating (zero in your case) on anything but a smoothly increasing counter.

You can work around it by replacing your expression with:

rate(foo[75s]) / 75  * 60

This way you'll actually get the rate of increase between data points 1 minute apart (a 75 seconds range will almost always return exactly 5 points, so 4 counter increases) and reverse the extrapolation to 75 seconds that Prometheus does. There will be some noise in edge cases (e.g. if your evaluation is aligned with scraping times it's possible to get 6 points in one range and 4 in the next due to scrape interval jitter) but you're getting that anyway with rate().

BTW, you can see the aliasing by increasing the resolution of your graph to something like 1 second (anything 15 seconds or below should show it clearly).

2 of 3
2

What you say doesn't line up with the data, that raw data is only going up about once a minute. Are you sure you're scraping every 15s?

🌐
GitConnected
levelup.gitconnected.com › prometheus-counter-metrics-d6c393d86076
Working With Prometheus Counter Metrics | Level Up Coding
February 28, 2022 - The counter is reset to zero when the application restarts. Lucky for us, PromQL (the Prometheus Query Language) provides functions to get more insightful data from our counters. Prometheus’ rate function calculates at what rate the counter ...
🌐
Promlabs
promlabs.com › blog › 2023 › 09 › 13 › dealing-with-missing-time-series-in-prometheus
PromLabs | Blog - Dealing with Missing Time Series in Prometheus
Imagine a PromQL query for the total operations rate: sum without(optype) (rate(operations_total{job="my-job"}[5m])) If no operation has happened at all yet, the expression will return an empty result instead of a rate with the value of 0, as you ...
🌐
Medium
nklya.medium.com › promql-how-to-return-0-instead-of-no-data-9e49f7ccb80d
PromQL / How to return 0 instead of ‘no data’ - Nicolai Antiferov - Medium
October 26, 2025 - And to solve this, you just need to add OR on() vector(0) to the end of your promQL query. It will return 0 if the metric expression does not return anything. Explanation: Prometheus uses label matching in expressions.
🌐
Medium
medium.com › @rameshavutu › prometheus-rate-irate-increase-counters-gauges-explained-2e4a0eeedfa3
Prometheus rate() vs irate() vs increase(): Understanding Counters, Gauges, and Accurate Metrics | Medium
June 10, 2026 - Memory usage doesn't have a "rate of increase per second" in the same sense. what you want is the current value or maybe a delta, not a rate. When a Prometheus-instrumented process restarts, its in-process counters reset to zero.
🌐
Reddit
reddit.com › r/prometheusmonitoring › how to deal with increase function and no data points
r/PrometheusMonitoring on Reddit: How to deal with Increase function and no data points
September 3, 2021 -

I need to use the increase function, but the metric is very rare and when service restarts the counter is null for long time. The increase metric doesn’t play nicely with the null value and I can find a solution to consider null as zero.

Increase(my_counter[1h])

Works only if there are no null data points. Any other query I tried do not work like

Increase(my_metrics[1h] or vector(0))
🌐
Reddit
reddit.com › r/prometheusmonitoring › sum rate with missing values?
r/PrometheusMonitoring on Reddit: Sum rate with missing values?
April 20, 2022 -

I'm using mtail to track a job that logs to a file that gets transaction counts once every few days. Mtail keeps a running count of the transactions and while it's running i'm able to visualize/track rates of transactions. However, if mtail is restarted, no metrics are available until the next time the job runs. These missing metrics are preventing me from capturing the rate of change from the first job that runs after restart. Is there a way to default missing data to 0?

So that instead of:

_ _ _ 500

I get

0 0 0 500

and the rate, given the 4 data points is 500 between the last two instead of no data?

🌐
GitHub
github.com › prometheus › prometheus › issues › 2793
If throttled, don't `rate`, etc. with 0 values · Issue #2793 · prometheus/prometheus
June 1, 2017 - If throttled, don't rate, etc. with 0 values#2793 ... We've run into the "throttling mode" ° today. It seems that Prometheus (1.6.3) uses zero values for the throttled time and calculates wrong rates resulting in false alerts afterwards.
Author: prometheus