Relabeling in Prometheus

What is Relabeling?

Relabeling in Prometheus is a powerful feature that allows users to modify labels on scraped time series data. This can be useful for a variety of reasons, including changing label names, filtering out unwanted series, or altering label values based on certain conditions. Relabeling is performed during the scraping process, allowing for a cleaner and more organized set of metrics.

Why Use Relabeling?

Relabeling helps in managing and organizing metrics effectively. It provides the ability to:

• Rename labels for consistency across services.
• Drop unnecessary labels to reduce cardinality.
• Modify label values to improve clarity and utility.
• Aggregate metrics from different sources under a unified label structure.

How Does Relabeling Work?

Relabeling is configured in the scrape_config section of the Prometheus configuration file. The process involves defining a set of relabeling rules that dictate how the labels should be processed. Each rule can include criteria for when to execute the rule, what transformation to apply, and the conditions under which to keep or discard the labels.

Relabeling Configuration Example

Here is an example configuration snippet demonstrating how to use relabeling in Prometheus:

scrape_configs:
  - job_name: 'example-job'
    static_configs:
      - targets: ['localhost:9090']
    relabel_configs:
      - source_labels: [__meta_kubernetes_namespace]
        action: replace
        target_label: namespace
        replacement: 'default'
      - source_labels: [__address__]
        target_label: __address__
        replacement: 'my-service:80'
      - source_labels: [__meta_kubernetes_service_name]
        action: drop
        regex: 'unwanted-service'
            

In this example:

• The first rule replaces the __meta_kubernetes_namespace label with a static value of default.
• The second rule modifies the __address__ label to point to my-service:80.
• The third rule drops any time series that have the service name unwanted-service.

Common Relabeling Actions

Here are some common actions you can use in relabeling:

• replace: Replaces the value of a label with a new value.
• keep: Retains the time series if the condition is met.
• drop: Discards the time series if the condition is met.
• hashmod: Computes the hash and allows for sharding.
• labelmap: Maps labels based on a regular expression.

Conclusion

Relabeling is an essential feature in Prometheus that allows users to refine and manage their monitoring data effectively. By utilizing relabeling, you can ensure your metrics are accurate, relevant, and easy to work with, ultimately leading to better insights and decision-making.