Cloudify High Availability Cluster Upgrade Guide

Overview

This guide explains how to upgrade a high-availability Cloudify Manager cluster to a new version.
Since the old version of Cloudify cluster was active-passive, the guide is the same for both this case and the Cloudify All-In-One case.
The best-practice to upgrade Cloudify is to deploy the new version on a new set of hosts, therefore a new host should be created for every existing Manager.

Upgrade to new hosts

  1. Create and download a snapshot from the existing Manager. Please refer to the Backup and restore guide for further instructions.
  2. Install a new Cloudify cluster. Please refer to the Cloudify cluster installation guide for further instructions.
  3. After the cluster and CLI installation is done, ssh into all Managers except one (best practice - 2 out of 3) and run cfy_manager stop. This would stop Cloudify services on these hosts.
  4. Copy the snapshot to the last Manager (the only active one).
  5. Ssh into this Manager, and run the following code:

    sudo /opt/patroni/bin/patronictl -c /etc/patroni.conf show-config | sudo tee /etc/patroni.conf.bak
    sudo /opt/patroni/bin/patronictl -c /etc/patroni.conf edit-config -s ttl=86400 -s retry_timeout=86400 --force
    if [[ -f /etc/haproxy/haproxy.cfg ]]; then
      sudo sed -i.bak -e 's/timeout client.*$/timeout client 1440m/' -e 's/timeout server.*$/timeout server 1440m/' /etc/haproxy/haproxy.cfg
      sudo service haproxy reload
    fi

    This will increase the Patroni and HAproxy timeouts, which is necessary for large snapshots to restore correctly.

  6. On the same Manager, upload the snapshot, and restore it. Please refer to the Backup and restore guide for further instructions.

  7. After the restore is done, revert the Patroni and HAproxy timeouts to their defaults:

    sudo /opt/patroni/bin/patronictl -c /etc/patroni.conf edit-config --apply /etc/patroni.conf.bak --force
    if [[ -f /etc/haproxy/haproxy.cfg ]]; then
      sudo mv /etc/haproxy/haproxy.cfg.bak /etc/haproxy/haproxy.cfg
      sudo service haproxy reload
    fi
  8. Connect to the inactive Managers, and on each one run cfy_manager start.

  9. Reinstall agents. Please refer to the Agents installation guide for further instructions.

  10. Delete old host/s.