You can use the argument list of group_left to include extra labels from the right operand (parentheses and indents for clarity):

(
  max(consul_health_service_status{status="critical"}) 
  by (service_name,status,node) == 1
)
   + on(service_name,node) group_left(env)
(
   0 * consul_service_tags
)

The important part here is the operation + on(service_name,node) group_left(env):

  • the + is "abused" as a join operator (fine since 0 * consul_service_tags always has the value 0)
  • group_left(env) is the modifier that includes the extra label env from the right (consul_service_tags)
Answer from user2361830 on Stack Overflow
🌐
Yannick Pereira-Reis
ypereirareis.github.io › blog › 2020 › 02 › 21 › how-to-join-prometheus-metrics-by-label-with-promql
How to join Prometheus metrics by label with PromQL – Yannick Pereira-Reis
The result we want is something ... => 30887053824 node1 => 36352176166912 · sum(node_disk_bytes_read * on(instance) group_left(node_name) node_meta{}) by (node_name) on(instance) => this is how to JOIN on label insta...
🌐
Prometheus
discuss.prometheus.io › promql
Using range with label_join - PromQL - Prometheus Monitoring System
October 26, 2022 - I have a set of metrics that are being sent only when there is an issue. I want to create an expression that will capture the number of issues in the last 15 minutes. For efficiency I would like to use a regex with the m…
Discussions

monitoring - How can I 'join' two metrics in a Prometheus query? - Stack Overflow
It is a good practice in Prometheus ecosystem to expose additional labels, which can be joined to multiple metrics, via a separate info-like metric as explained in this article. For example, consul_service_tags metric exposes a set of tags, which can be joined to metrics via (service_name, ... More on stackoverflow.com
🌐 stackoverflow.com
Mechanism Search based on new label created through label_join function
Proposal As you already know using label_join() and label_replace() function can be used to add new label dynamically. But there is way by to do matching/search on top of this newly added label. So this new label is useless if it can't b... More on github.com
🌐 github.com
24
January 15, 2019
simpler syntax for joins on labels with different names
Background When doing joins, I usually encounter cases where I want to join through a label that has different names on each side. For example, suppose I have: A container_fs_reads_total metric wit... More on github.com
🌐 github.com
8
October 10, 2024
Allow label_join to be used in aggregation operators
I have looked quite a bit and haven't found a way to do this but it seems simple so please let me know if I've missed something. I have a need to do capacity planning for failover events wh... More on github.com
🌐 github.com
12
November 23, 2021
🌐
GitHub
github.com › prometheus › prometheus › issues › 2204
Joining different labels · Issue #2204 · prometheus/prometheus
November 18, 2016 - The label naming might not be 100% aligned, i.e. in Kubernetes we get container metrics with a pod_name label from cAdvisor and metrics with a pod label from kube-state-metrics. The values are the same pod identifying names. However, we have to use label_replace() to be able to join them properly, which is always somewhat awkward and hard to decipher.
Author: prometheus
🌐
Grafana
grafana.com › blog › 2021 › 08 › 04 › how-to-use-promql-joins-for-more-effective-queries-of-prometheus-metrics-at-scale
How to use PromQL joins for more effective queries of Prometheus metrics at scale | Grafana Labs
August 5, 2021 - PromQL supports the ability to join two metrics together: You can append a label set from one metric and append it to another at query time. This can be useful in Prometheus rule evaluations, since it lets you generate a new metric for a series by appending labels from another info metric.
Top answer
1 of 3
76

You can use the argument list of group_left to include extra labels from the right operand (parentheses and indents for clarity):

(
  max(consul_health_service_status{status="critical"}) 
  by (service_name,status,node) == 1
)
   + on(service_name,node) group_left(env)
(
   0 * consul_service_tags
)

The important part here is the operation + on(service_name,node) group_left(env):

  • the + is "abused" as a join operator (fine since 0 * consul_service_tags always has the value 0)
  • group_left(env) is the modifier that includes the extra label env from the right (consul_service_tags)
2 of 3
15

It is a good practice in Prometheus ecosystem to expose additional labels, which can be joined to multiple metrics, via a separate info-like metric as explained in this article. For example, consul_service_tags metric exposes a set of tags, which can be joined to metrics via (service_name, node) labels.

The join is usually performed via on() and group_left() modifiers applied to * operation. The * doesn't modify values for time series on the left side because info-like metrics usually have constant 1 values. The on() modifier is used for limiting the labels used for finding matching time series on the left and the right side of *. The group_left() modifier is used for adding additional labels from time series on the right side of *. See these docs for details.

For example, the following PromQL query adds env label from consul_service_tags metric to consul_health_service_status metric with the same set of (service_name, node) labels:

consul_health_service_status
  * on(service_name, node) group_left(env)
consul_service_tags

Additional label filters can be added to consul_health_service_status if needed. For example, the following query returns only time series with status="critical" label:

consul_health_service_status{status="critical"}
  * on(service_name, node) group_left(env)
consul_service_tags
Authors: Joel BastosPedro Araújo
Published: 2019
Pages: 442
🌐
GitHub
github.com › prometheus › prometheus › issues › 5097
Mechanism Search based on new label created through label_join function · Issue #5097 · prometheus/prometheus
January 15, 2019 - label_join(sum(app:healthcheck:gauge{application="collectHealthies"} != bool 200) by (first_name, last_name), "full_name", " ", "first_name", "last_name") ... {first_name="Jack",full_name="Jack daniels",last_name="daniels"} 10 {first_name="...
Author: prometheus
🌐
GitHub
github.com › prometheus › prometheus › issues › 15146
simpler syntax for joins on labels with different names · Issue #15146 · prometheus/prometheus
October 10, 2024 - Background When doing joins, I usually encounter cases where I want to join through a label that has different names on each side. For example, suppose I have: A container_fs_reads_total metric with a pod label A some_custom_pod_info met...
Author: prometheus
Find elsewhere
🌐
GitHub
github.com › prometheus › prometheus › issues › 9850
Allow label_join to be used in aggregation operators · Issue #9850 · prometheus/prometheus
November 23, 2021 - {instance="datacenter0"} 1000 {instance="datacenter1"} 2000 sum by(label_join("instance", "datacenters", ",")) ( irate( tIPFilterParamsEgrHitByteCount{ instance=~"datacenter.*", }[10m] ) ) {datacenters="datacenter0,datacenter1"} 3000
Author: prometheus
🌐
Prometheus
prometheus.io › docs › prometheus › latest › querying › functions
Query functions | Prometheus
For each timeseries in v, label_join(v instant-vector, dst_label string, separator string, src_label_1 string, src_label_2 string, ...) joins all the values of all the src_labels using separator and returns the timeseries with the label dst_label containing the joined value.
🌐
OneUptime
oneuptime.com › home › blog › how to join two metrics in prometheus query
How to Join Two Metrics in Prometheus Query
December 17, 2025 - # Create composite key for matching label_join( container_memory_usage_bytes, "pod_container", "_", "pod", "container" )
🌐
SigNoz
signoz.io › blog › prometheus query tutorial with examples
Prometheus Query Tutorial with examples | SigNoz
August 1, 2022 - In this case, these are (a, b, c, z). There is no limit to the number of sources you can concatenate this way. The separator element should also be defined in the function. In this case, it is a simple comma separator (","). Prometheus ...
🌐
Iximiuz
iximiuz.com › en › posts › prometheus-vector-matching
Prometheus Cheat Sheet - How to Join Multiple Metrics (Vector Matching)
November 30, 2021 - How to join metrics on matching labels? Vector matching: one-to-one, one-to-many and many-to-one, many-to-many. What on/ignoring clause is for? group_left and group_right explained. Logical/set operations explained.
🌐
Google Groups
groups.google.com › g › prometheus-users › c › qlHK7r1hQVs
Joining two metrics in promtheus and having all labels from both
March 6, 2023 - To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/4c22a279-a51c-47db-993e-9107e54e883en@googlegroups.com. ... Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message ... But yes, you'll have to treat it as a many-to-one in order to pick up the extra labels.
🌐
OneUptime
oneuptime.com › home › blog › how to write prometheus queries that return label values
How to Write Prometheus Queries That Return Label Values
December 17, 2025 - Prometheus HTTP API - Direct access to label metadata · PromQL functions - label_replace() and label_join() for transformation
🌐
Ac-programming
ac-programming.com › content › Prom › Functions › label.html
Prometheus Label Functions
The regular expression "(.*)" captures the value of "source_label" and assigns it to "new_label". ... label_join(my_metric{job="job1"}, "new_label", ",", "label1", "label2") This example joins the values of "label1" and "label2" with a comma (",") as the delimiter and assigns the result to ...
🌐
Medium
sean22492249.medium.com › prometheus-label-replace-label-join-label-values-002efbdec87e
Prometheus label_replace, label_join, label_values | by Kiwi lee | Medium
June 28, 2024 - label_join(up, "combination", ",", "job", "instance") #Before up{instance="localhost:9090", job="prometheus"} #After up{instance="localhost:9090", job="prometheus", combination="prometheus,localhost:9090"} 將 job, instance 合併後,再擷...
🌐
Better Stack
betterstack.com › community › questions › how-to-join-two-metric-in-one-prometheus-query
How Can I 'Join' Two Metrics in a Prometheus Query? | Better Stack Community
Joining two metrics in Prometheus is commonly achieved through the use of the * operator or the group function, allowing you to perform operations between metrics based on their labels. Unlike traditional SQL databases, Prometheus does not have a direct JOIN function; instead, you leverage label matching and arithmetic operations to combine metric data.