Full Cluster Restart | Upgrading Elasticsearch

Introduction

Upgrading Elasticsearch is a crucial task for maintaining the stability, security, and performance of your cluster. One common upgrade method is a Full Cluster Restart. This process involves shutting down all nodes in the cluster, upgrading them, and then starting them up again. This tutorial will guide you through the full cluster restart process step-by-step.

Prerequisites

Before starting the upgrade process, ensure you have the following:

Backup of all your data
Administrative access to all Elasticsearch nodes
Knowledge of the current Elasticsearch version and the target version
Updated configuration files compatible with the new version

Step 1: Check Cluster Health

Before proceeding with the upgrade, it's essential to ensure that the cluster is in a healthy state. Use the following command to check the cluster health:

GET /_cluster/health

{
  "cluster_name": "your_cluster_name",
  "status": "green",
  "timed_out": false,
  "number_of_nodes": 3,
  "number_of_data_nodes": 3,
  "active_primary_shards": 10,
  "active_shards": 20,
  "relocating_shards": 0,
  "initializing_shards": 0,
  "unassigned_shards": 0,
  "delayed_unassigned_shards": 0,
  "number_of_pending_tasks": 0,
  "number_of_in_flight_fetch": 0,
  "task_max_waiting_in_queue_millis": 0,
  "active_shards_percent_as_number": 100.0
}

The status should be "green" before proceeding.

Step 2: Disable Shard Allocation

To prevent Elasticsearch from rebalancing shards while nodes are restarting, disable shard allocation:

PUT /_cluster/settings

{
  "persistent": {
    "cluster.routing.allocation.enable": "none"
  }
}

Step 3: Stop Elasticsearch on All Nodes

Stop the Elasticsearch service on all nodes. The command may vary based on your operating system:

sudo systemctl stop elasticsearch

or

sudo service elasticsearch stop

Step 4: Upgrade Elasticsearch

Upgrade Elasticsearch on all nodes. Follow the official Elasticsearch upgrade documentation for your specific version. For example, on Debian-based systems:

sudo apt-get update && sudo apt-get install elasticsearch

Step 5: Start Elasticsearch on All Nodes

Start the Elasticsearch service on all nodes after the upgrade:

sudo systemctl start elasticsearch

or

sudo service elasticsearch start

Step 6: Re-enable Shard Allocation

Once all nodes are up and running with the new version, re-enable shard allocation:

PUT /_cluster/settings

{
  "persistent": {
    "cluster.routing.allocation.enable": "all"
  }
}

Step 7: Verify Cluster Health

Finally, verify that the cluster is healthy and all nodes are running the upgraded version:

GET /_cluster/health

Check that the status is "green" and all nodes are accounted for.

GET /_nodes

Ensure that the version number is updated for all nodes.

Conclusion

Upgrading Elasticsearch using a Full Cluster Restart is a straightforward process, but it requires careful planning and execution. Following the steps outlined in this tutorial will help ensure a smooth upgrade with minimal downtime. Always remember to backup your data and check the cluster health before and after the upgrade.

Full Cluster Restart - Upgrading Elasticsearch