Cluster Health - Elasticsearch

Introduction

Monitoring the health of your Elasticsearch cluster is crucial for ensuring its reliability, performance, and availability. Elasticsearch provides various APIs and tools to monitor and manage cluster health effectively. This tutorial will guide you through the essential aspects of monitoring cluster health, interpreting health indicators, and taking actions to maintain optimal cluster performance.

Cluster Health API

The _cluster/health API provides a simple way to get an overview of the health status of your cluster. This API returns information about the cluster's status, number of nodes, number of data nodes, active primary and replica shards, and more.

Example Request

Example Response

{
  "cluster_name": "elasticsearch",
  "status": "green",
  "timed_out": false,
  "number_of_nodes": 3,
  "number_of_data_nodes": 2,
  "active_primary_shards": 5,
  "active_shards": 10,
  "relocating_shards": 0,
  "initializing_shards": 0,
  "unassigned_shards": 0,
  "delayed_unassigned_shards": 0,
  "number_of_pending_tasks": 0,
  "number_of_in_flight_fetch": 0,
  "task_max_waiting_in_queue_millis": 0,
  "active_shards_percent_as_number": 100.0
}
                

Interpreting Cluster Health Status

The cluster health status is indicated by three possible states:

• Green: All primary and replica shards are active.
• Yellow: All primary shards are active, but some replica shards are not allocated.
• Red: Some primary shards are not active, which means some data is unavailable.

Ensuring your cluster remains in the green state is ideal, but a yellow state is acceptable if the system can still handle the load. A red state requires immediate attention as it indicates data unavailability.

Monitoring Cluster Health

Regularly monitor your cluster health using the _cluster/health API or tools like Kibana. Kibana's monitoring features provide a visual representation of your cluster's health and performance metrics.

Kibana Monitoring

Kibana offers a monitoring plugin that visualizes cluster metrics such as node statistics, index statistics, and shard allocation. This tool is beneficial for proactively detecting and resolving issues.

Common Issues and Solutions

Unassigned Shards

Unassigned shards can occur due to node failures, disk space issues, or configuration errors. Use the _cluster/allocation/explain API to identify the cause and take corrective actions.

Example Request

Example Response

{
  "index": "my_index",
  "shard": 0,
  "primary": true,
  "current_state": "unassigned",
  "unassigned_info": {
    "reason": "NODE_LEFT",
    "at": "2021-05-24T14:42:25.048Z",
    "details": "node_left [node-1]",
    "last_allocation_status": "no_attempt"
  },
  "can_allocate": "no",
  "allocate_explanation": "cannot allocate because a previous copy of the primary shard existed on node [node-1] which is now left",
  "node_allocation_decisions": []
}
                

Conclusion

Maintaining the health of your Elasticsearch cluster is vital for ensuring data availability and system performance. By regularly monitoring cluster health, understanding health indicators, and addressing common issues, you can ensure your Elasticsearch cluster runs smoothly and efficiently.