Summary from

• Understanding Prometheus Range Vectors
• The Anatomy of a PromQL Query

• What’s a Vector?
  • Since Prometheus is a timeseries database, all data is in the context of some timestamp. The series that maps a timestamp to recorded data is called a timeseries
  • a set of related timeseries is called a vector
  • Ex.
    • http_requests_total is a vector representing the total number of http requests received by a service
    • http_requests_total{code="200"} http_requests_total refers to the entire set of timeseries that are named that. And by appending a {code="200"}, we’re selecting a subset.
• Types of Vectors
  • Instant vector - a set of timeseries where every timestamp maps to a single data point at that “instant”
    • Imagine evaluating the expression http_requests_total at a given timestamp. http_requests_total is an instant vector selector that selects the latest sample for any time series with the metric name http_requests_total. More specifically, "latest" means "at most 5 minutes old and not stale", relative to the evaluation timestamp. So this selector will only yield a result for series that have a sample at most 5 minutes prior to the evaluation timestamp, and where the last sample before the evaluation timestamp is not a stale marker (an explicit way of marking a series as terminating at a certain time in the Prometheus TSDB).
  • Range vector - a set of timeseries where every timestamp maps to a “range” of data points, recorded some duration into the past.
    • Range vector is mostly used for graphs, where you want to show a PromQL expression over a given time range. A range query works exactly like many completely independent instant queries that are evaluated at subsequent time steps over a given range of time. Of course, this is highly optimized under the hood and Prometheus doesn't actually run many independent instant queries in this case.
• Differences
  • Instant vectors can be charted; Range vectors cannot. This is because charting something involves displaying a data point on the y-axis for every timestamp on the x-axis. Instant vectors have a single value for every timestamp, while range vectors have many of them. For the purpose of charting a metric, it is undefined1 how to show multiple data points for a single timestamp in a timeseries.
  • Instant vectors can be compared and have arithmetic performed on them; Range vectors cannot. This is also due to the way comparison and arithmetic operators are defined. For every timestamp, if we have multiple values, we don’t know how to add1 or compare them to another timeseries of a similar nature.
  • Range Vectors for counters. We take the instant vector and append our duration [15m]. This part is called the range selector and it transforms the instant vector into a range vector. We then use a function like increase which effectively subtracts the data point at the start of the range from the one at the end. increase(http_requests_total{code="200",handler="/api/v1/query"}[15m]) represent it is the increase in the total number of requests over the past fifteen minutes

Solution

It's possible with the subquery-syntax (introduced in Prometheus version 2.7):

deriv(rate(varnish_main_client_req[2m])[5m:10s])

Warning: These subqueries are expensive, i.e. create very high load on Prometheus. Use recording-rules when you use these queries regularly (in alerts, etc.).

Subquery syntax

<instant_query>[<range>:<resolution>]

• instant_query: a PromQL-function which returns an instant-vector)
• range: offset (back in time) to start the first subquery
• resolution: the size of each of the subqueries.

It returns a range-vector.

In the example above, Prometheus runs rate() (= instant_query) 30 times (the first from 5 minutes ago to -4:50, ..., the last -0:10 to now). The resulting range-vector is input to the deriv()-function.

Another example (mostly available on all Prometheus instances):

deriv(rate(prometheus_http_request_duration_seconds_sum{job="prometheus"}[1m])[5m:10s])

Without the subquery-range ([5m:10s]), you'll get this error-message:

parse error at char 80: expected type range vector in call to function "deriv", got instant vector

The range vector selector is just an ordinary time series selector (which is confusingly named instant vector selector) with the added lookbehind window in square brackets.

An ordinary time series selector (aka instant vector selector) selects time series matching the given filter. For example, http_requests_total{path=~"/foo/bar|/baz"} selects time series with the name http_requests_total and the path label containing either /foo/bar or /baz values.

The corresponding range vector selector with the one hour lookbehind window looks like the following http_requests_total{path=~"/foo/bar|baz"}[1h].

The range vector selector can be used in the following places:

• It can be passed to /api/v1/query. In this case the API returns all the raw samples for matching time series on the interval (time-d ... time], where time is the query arg passed to /api/v1/query, while d is the specified lookbehind window in square brackets of range vector selector. See this article for details.

• It can be passed to e.g. rollup functions. These functions perform calculations over raw samples on the given lookbehind window in square brackets. The calculations are performed independently per each matching time series and per each requested point on the graph. Prometheus datasource in Grafana sends requests to /api/v1/query_range. This API accepts start, end and step query args and calculates N=1+(end-start)/step points per each matching time series at timestamps start, start+step, start+2*step, ..., start+(N-1)*step.

Let's look how rate(m[d]) is calculated on the start ... end time range with the given step:

• Prometheus selects all the time series matching m on the given time range (start-d .. end]. Note that the time range starts from start-d instead of start, where d is the provided lookbehind window in square brackets.

• Then Prometheus calculates the average per-second increase rate over the given lookbehind window d individually per each matching time series per each requested point on the graph.

See also this answer.

Is there any way to perform an instant vector operation on a range vector?

No. Prometheus doesn't allow anything like this.

But you can apply range selector over something other than vector selector using subquery syntax. So in your example it would be something like.

avg(avg_over_time((count_containers > 0)[1h:15s]))

Notice that in this case you must place : in the range selector to indicate usage of subquery.

And for this example I used resolution 15s, to indicate that result of the query should be calculated for each 15 seconds window. But you might want to adjust this to your needs, depending on needed precision, scrape interval, etc. Also, resolution can be omitted (while preserving :, like [1h:]): in that case value of evaluation_interval will be used.

we want to count the fraction of the time where we have at least one, we might try to do:

Your attempt of the query, even if it were supported, would not produced what you wanted. Rather it would calculate average number of containers when number of containers was positive.

To calculate percentage of the time when number of containers was positive use following

avg(avg_over_time( (count_containers > bool 0)[1h:15s] ))

Expression count_containers > bool 0 will return 1 if number of containers is positive, and 0 otherwise.