you can use the label_replace function in promQL, but it also add the label, don't replace it
label_replace(
<vector_expr>, "<desired_label>", "$1", "<existing_label>", "(.+)"
)
label_replace(
node_systemd_unit_state{instance="server-01",job="node-exporters",name="kubelet.service",state="active"},
"unit_name","$1","name", "(.+)"
)
So, to avoid the repetition you can add:
sum(label_replace(
node_systemd_unit_state{instance="server-01",job="node-exporters",name="kubelet.service",state="active"},
"unit_name","$1","name", "(.+)"
)
)by(unit_name)
Answer from Chus on Stack OverflowI want to replace label with a unique something
Query label_replace function - PromQL - Prometheus Monitoring System
relabel and aggregate metrics
replace - How to write nested label_replace queries in prometheus? - Stack Overflow
How do I filter by label values in PromQL?
What is PromQL?
How do I predict future resource usage in PromQL?
you can use the label_replace function in promQL, but it also add the label, don't replace it
label_replace(
<vector_expr>, "<desired_label>", "$1", "<existing_label>", "(.+)"
)
label_replace(
node_systemd_unit_state{instance="server-01",job="node-exporters",name="kubelet.service",state="active"},
"unit_name","$1","name", "(.+)"
)
So, to avoid the repetition you can add:
sum(label_replace(
node_systemd_unit_state{instance="server-01",job="node-exporters",name="kubelet.service",state="active"},
"unit_name","$1","name", "(.+)"
)
)by(unit_name)
I got tired of all the fragmented documentation and I feel I provided a better answer in this post here: https://medium.com/@texasdave2/replace-and-remove-a-label-in-a-prometheus-query-9500faa302f0
Replace is not a true REPLACE
Your goal is to simply replace the old label name “old_job_id” with a new label name “new_task_id”. Prometheus label_replace will really “add” the new label name. It will preserve the old label name as well… So, that could be a problem, it’s not a true “replace in place”.
So if you want to “add” your new label name and “remove” the old label name, you need to do this:
sum without (old_job_id) (label_replace(metric, "new_task_id", "$1", "old_job_id", "(.*)"))
Here’s how this reads:
sum without (old_job_id) will remove the old label name from the query output
metric is your metric, like “node_filesystem_avail_bytes”
“new_task_id” is where you would put your new label name
“$1” is regex for using the string in new label name, don’t change this
“old_job_id” is where you’ll put your old label, the one you want to get rid of (.*……. that mess is regex that will replace the whole label name
Hi,
I have rabbitmq metrics which contains the `channel` label. Since this label has high cardinality I decided I want to drop it, but faced an issue.
When prometheus drops it, there will be duplicates, and prometheus just take one of them, the exact situation here https://grafana.com/blog/2022/10/20/how-to-manage-high-cardinality-metrics-in-prometheus-and-kubernetes/#3-begin-optimizing-metrics in the `Reduce labels` section.
From what I can see, I need recording rule that would sum these metrics but im not sure about the order of operations.
If I have a metric_relabeling_rule in the scrapping config and a recording rule, which one will be applied first?
Is there a sensible way of recalculating all of the metrics that contains the `channel` label and take the sum of them such that no data is being dropped?
Or do I have to create a new metric name with the channel summed?
Edit:
In this response they say "maybe you need to aggregate over the duplicate series", but i dont know if they mean recording rules or what