Prometheus doesn't provide functionality, which can be used for returning cumulative increase over multiple time series on the selected time range.
If you still need this functionality, then try VictoriaMetrics - Prometheus-like monitoring solution I work on. It allows calculating cumulative increase over multiple counters. For example, the following MetricsQL query returns cumulative increase over all the time series with http_requests_total name on the selected time range in Grafana:
running_sum(sum(increase(http_requests_total)))
How does it work?
It calculates increase per each time series with the
http_requests_totalname. Note that theincrease()in the query above doesn't contain lookbehind window in square brackets. VictoriaMetrics automatically sets the lookbehind window to thestepvalue, which is passed by Grafana to /api/v1/query_range endpoint. Thestepvalue is the interval between points on the graph.It sums increases returned at step 1 with the sum() function individually per each point on the graph.
It calculates cumulative increase over per-
stepincreases returned at step 2 with the running_sum function.
Prometheus doesn't provide functionality, which can be used for returning cumulative increase over multiple time series on the selected time range.
If you still need this functionality, then try VictoriaMetrics - Prometheus-like monitoring solution I work on. It allows calculating cumulative increase over multiple counters. For example, the following MetricsQL query returns cumulative increase over all the time series with http_requests_total name on the selected time range in Grafana:
running_sum(sum(increase(http_requests_total)))
How does it work?
It calculates increase per each time series with the
http_requests_totalname. Note that theincrease()in the query above doesn't contain lookbehind window in square brackets. VictoriaMetrics automatically sets the lookbehind window to thestepvalue, which is passed by Grafana to /api/v1/query_range endpoint. Thestepvalue is the interval between points on the graph.It sums increases returned at step 1 with the sum() function individually per each point on the graph.
It calculates cumulative increase over per-
stepincreases returned at step 2 with the running_sum function.
Using sum(increase(http_requests_total{method="POST",path="/resource/aaa",statusClass="2XX"}[$__interval])) as query is a good starting point and that's as far as Prometheus can go.
However, in order to get the cumulative graph, you need to leverage a "Transformation" in the Grafana UI as I'll describe below with a concrete example.
See for example this graph in the Grafana Playground
Let's select a single timeseries upper_95 that we want to plot the cumulative sum for:
Then select the "Transformations" tab and click "Add Transformation"
Choose the "Add field from calculation" transformation
Then choose Mode "Cumulative functions" and Calculation "Total" (should be the default). The documentation says:
Cumulative functions - Apply functions on the current row and all preceding rows. Total - Calculates the cumulative total up to and including the current row.
which is exactly what we want.
Since we have multiple timeseries we need to also select "upper_95" as Field, but if you only have one you can leave it as blank.
Now you have the cumulative sum graph available and you can select it in the legend to hide the other ones:
What is the difference between sum_over_time() and sum()?
How does sum_over_time() handle missing data points?
Can sum_over_time() cause high memory usage?
What you need is the increase() function, that will calculate the difference between the counter values at the start and at the end of the specified time interval. It also correctly handles counter resets during that time period (if any).
increase(http_requests_total[24h])
If you have multiple counters http_requests_total (e.g. from multiple instances) and you need to get the cumulative count of requests, use the sum() operator:
sum(increase(http_requests_total[24h]))
See also my answer to that part of the question about using Grafana's time range selection in queries.
SO won't let me comment on Yoory's answer so I have to make a new one...
In Grafana 5.3, they introduced $__range for Prometheus that's easier to use:
sum(rate(http_requests_total[$__range]))
This variable represents the range for the current dashboard. It is calculated by to - from
http://docs.grafana.org/features/datasources/prometheus/
Hello,
I have this graph monitoring the bandwidth of a VLAN on a switch every 1m using SNMP Exporter, but I also what to get the total/sum data over time, so if I select the last hour it will show x amount inbound and x amount outbound.
sum by(ifName) (irate(ifHCInOctets{instance=~"192.168.200.10", job="snmp_exporter", ifName=~".*(1001).*"}[1m])) * 8My current graph:
I'd like to duplicate and create a stat panel show how much data in total has passed over what period I choose that's all.
For the metric I'm not sure whether to use bytes(SI) or bytes(IEC), but are similar if I change to either.
Not sure how to calculate this, but I have this created for the past 1 hour.
by copying the PromQL in Grafana and changing to a stat panel and then editing to use this:
Not sure if this is ok as I'm not sure how to calculate it all, maths was never my best subject.
Any help would be great.
I think something like is close: with sum_over_time
sum by(ifName) (sum_over_time(ifHCInOctets{instance=~"192.168.200.10", job="snmp_exporter", ifName=~".*(1001).*"}[1m])) * 8but it comes back as 85.8 Pib when it should be 85.8 TB with my calculations.
EDIT
Observium:
What Grafana shows