I found here this solution: {__name__=~"metricA|metricB|metricC",container_name=~"frontend|backend|db"}.
How to execute multiple queries in one call in Prometheus - Stack Overflow
grafana - Query multiple metrics in one Prometheus HTTP Call - Stack Overflow
How to get all the metrics of an instance with prometheus api? - Stack Overflow
promql - query Prometheus HTTP API by list of metrics - Stack Overflow
You can fetch all metrics with following curl expression:
url=http://{urIPorhostname}:9090
curl -s $url/api/v1/label/__name__/values | jq -r ".data[]" | sort
All the metrics for a particular instance IP:9100 can be obtained via the following PromQL query:
{instance="IP:9100"}
This query returns all the metrics, which have the given instance="IP:9100" label. See time series selector docs for more details. See also PromQL tutorial.
The {instance="IP:9100"} query can be sent to the following Prometheus querying APIs:
- /api/v1/query - this endpoint returns matching time series values at the given timestamp specified via
timequery arg. For example,curl 'http://prometheus:9090/api/v1/query?query={instance="IP:9100"}'. Make sure you properly encodedqueryarg with percent encoding. See this demo. - /api/v1/series - this endpoint returns matching time series without any data, e.g. only metric names and labels are returned. For example,
curl 'http://prometheus:9090/api/v1/series?match[]={instance="IP:9100"}'. Make sure you properly encodedmatch[]arg with percent encoding. See this demo - /api/v1/query_range - this endpoint returns calculated datapoints for matching time series on the selected time range
[start ... end]with the givenstepinterval between samples. See more details about calculated datapoints in these docs.
If you want returning just unique metric names without labels, then you can query /api/v1/label/__name__/values endpoint: curl http://prometheus:9090/api/v1/label/__name__/values . The __name__ is a special label name used in Prometheus for referring to metric names. Note that multiple time series may share the same metric name. For example, http_requests_total{path="/foo"} and http_requests_total{path="/bar"} time series share http_requests_total metric name, while they differ by path label values.
Hello!
I've got a unique situation that I'm looking to the community to see if anyone has done something similar.
Basically I've got a Prometheus/Grafana instance running scraping metrics. I've configured a lot of dashboards in Grafana via PromQL queries and it is working great.
I have another system where I'd like to import all of these metrics on an interval to combine with some other infrastructure items I have there. The best two paths I could come up with are:
-
Submit a GET request to the Prometheus API for each PromQL query I've defined and store those as key:value pairs (there are about 100 PromQL queries I'm looking to store on a fixed interval - let's say every 5 mins)
-
This gets the job done but isn't very efficient as I would be submitting 100 GET requests via a loop every 5 mins
-
-
Submit a single bulk GET request gathering all the metrics and somehow re-create the PromQL queries programmatically (sums, rates, etc) by manipulating the json response
-
This would be more efficient and less load on Prometheus itself, but that a lot of work sifting through that much json
-
Has anyone attempted to do the same or have any ideas that I might be missing? I'm pretty much limited to getting the metrics through the Prometheus API.
You can hack your way around it as long as you're not too picky about what you want to query. E.g. if you want to query all up time series and all cpu_utilization time series, you can use something like this:
{__name__=~"up|cpu_utilization"}
It will even work with range queries, as long as you want the same range for all time series:
{__name__=~"up|cpu_utilization"}[1m]
But not if you want different time ranges or want to add selectors (e.g. there's no easy way of combining up{job="prometheus} and cpu_utilization{instance="foo:8080"}).
You can't send multiple queries as query= qry1 qry2.. in a single HTTP API call. If you take a closer look to the response object:
$ curl 'http://localhost:9090/api/v1/query?query=up&time=2015-07-01T20:10:51.781Z'
{
"status" : "success",
"data" : {
"resultType" : "vector",
"result" : [
{
"metric" : {
"__name__" : "up",
"job" : "prometheus",
"instance" : "localhost:9090"
},
"value": [ 1435781451.781, "1" ]
},
{
"metric" : {
"__name__" : "up",
"job" : "node",
"instance" : "localhost:9100"
},
"value" : [ 1435781451.781, "0" ]
}
]
}
}
You will see that "data" field contains only two keys (i.e. "resultType" and "result") and it isn't an array itself. So there is no chance that "data" will hold the result of multiple queries.