Prometheus Tutorial

Introduction to Prometheus

Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud. Since its inception in 2012, many companies and organizations have adopted Prometheus, and the project has a very active developer and user community. It is now a standalone open-source project and maintained independently of any company. To emphasize this, and to clarify the project's governance structure, Prometheus joined the Cloud Native Computing Foundation in 2016 as the second hosted project after Kubernetes.

Installation

Prometheus is a single binary which you can run as a standalone service. The easiest way to get started with Prometheus is to download the latest release from the official website and extract it.

Running Prometheus

After extracting the files, you can run Prometheus directly from the command line:

Prometheus will start with a default configuration and store its data in memory. You can access the Prometheus web interface by navigating to http://localhost:9090 in your web browser.

Configuration

Prometheus uses a simple configuration file, prometheus.yml, to define what it monitors and how it monitors it. The default configuration file looks like this:

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']

This configuration specifies that Prometheus should scrape its own metrics every 15 seconds.

Monitoring Applications

To monitor your own applications, you need to expose metrics in a format that Prometheus understands. This is typically achieved by using a client library. Prometheus client libraries are available for many languages, including Go, Java, Python, and Ruby.

For example, to instrument a Python application using the prometheus_client library, you would do the following:

from prometheus_client import start_http_server, Summary
import random
import time

REQUEST_TIME = Summary('request_processing_seconds', 'Time spent processing request')

@REQUEST_TIME.time()
def process_request(t):
    time.sleep(t)

if __name__ == '__main__':
    start_http_server(8000)
    while True:
        process_request(random.random())

This code will start an HTTP server on port 8000 and expose a single metric, request_processing_seconds, which tracks the time spent processing requests.

PromQL - The Prometheus Query Language

Prometheus provides its own query language called PromQL to query the time series data. A basic example of a PromQL query is:

This query returns the current state of all targets (up or down). More complex queries can be built to aggregate and slice the data in various ways. For example:

This query calculates the per-second rate of HTTP requests over the last 5 minutes.

Alerting

Prometheus has an integrated Alertmanager that can handle alerts. You define alerting rules in your configuration file, and when an alert is triggered, the Alertmanager handles the notifications.

An example alerting rule configuration:

alerts:
  - alert: HighRequestLatency
    expr: job:request_latency_seconds:mean5m{job="myjob"} > 0.5
    for: 10m
    labels:
      severity: page
    annotations:
      summary: "High request latency"

This rule will trigger an alert if the mean request latency over 5 minutes exceeds 0.5 seconds for 10 minutes.

Conclusion

Prometheus is a powerful tool for monitoring and alerting in a cloud-native environment. By following this tutorial, you should have a basic understanding of how to install, configure, and use Prometheus to monitor your applications. The next steps would involve exploring more advanced features and integrations, such as using Grafana for visualization and configuring advanced alerting rules.