What should I use for time on irate/rate?
irate() Vs rate() Functions in Prometheus
Let’s say you have a counter with these values observed at 1m intervals:
0 60 120 600 720 780
Now rate over 5m will be:
(780-0)/5/60 = 2.6/sec
And irate over 5m will be (only last two data points are used which happen to be only 1m apart)
(780-720)/1/60 = 1/sec
Increasing the resolution does not affect the irate function because the last two observed values do not change when you look further back.
More on reddit.comRate and irate display very different values
prometheus - Rate of the metric aggregated by label - Stack Overflow
Can rate() be used with all types of Prometheus metrics?
How do you calculate request rates using the Prometheus rate function?
How do you calculate the increase of a counter over time using Prometheus functions?
Hi All,
I'm trying to understand how irate() & rate() functions work. Why does irate() produce a similar looking graph when the range / resolution is 24h or 5m ? While the difference in graph is clearly visible with rate() when using range as 24h (presents a smoothed out line) or 5m(more spikey).
In the below graph for irate() for 2 different resolutions the graph looks the same. As per prometheus docs irate() calculates the per second instant rate based on the last two data points. What does this mean if my range is 24h? Thank you.
https://preview.redd.it/08mc4pq2mye41.jpg?width=3206&format=pjpg&auto=webp&s=4aef0b681969c8b9831ec0f34310838511ea0350
As it was repeated numerous times, again and again rate must be applied before sum.
Additionally both rate and irate require at least two samples in range vector to return anything. I doubt that you have a scrape interval of less then 8 seconds. So most likely you range selector is incorrect, and you need something bigger like [30s] or anything, but at least twice your scrape interval. And if you are planning to use it in Grafana you can use [$__rate_interval] instead: Grafana will substitute best suited value itself.
Your query should look like
sum by (app) (rate(application_errors_total[30s]))
This could apply to more general use cases. So want to share my 2c.
As discussed, rate/irate/increase all requires two data points, so it's not usable for your use case.
We can hand calculate the increase, leveraging the offset feature of promQL.
Assuming the scrape-interval is 15 seconds, here is the promQL:
(sum(application_errors_total) by (app)) - (sum(application_errors_total) by (app) offset 15s)
If you really want rate, then you can divide the result by scrape-interval.
It still requires two data points to calculate the difference. But instead of requiring each app/error-type combination has at least two data points, it only requires each app has at least two data points for any error types.
Of course it's a poor man's increase/rate. It doesn't consider counter reset or extrapolation.