Overview of Open Source Components in Cloudify

This section is to provide information about how the Cloudify architecture supports currently-implemented flows. Operational knowledge is assumed.

Cloudify Manager primarily is built with open-source components. The relationships between the components in the Cloudify Manager architecture is illustrated in the diagram below.

Cloudify components

Ports and Entry Points

Rather than specifying the ports in each component’s overview, ports are specified here so that you can easily review network requirements.

External Ports

By default, there are two external networks from which the Cloudify management environment is accessed:

Therefore, Cloudify requires only two entry points to its management environment:

Application Ports

The following ports are exposed for agent-manager communication:

The agents use the REST service to update the application’s model (for example, setting runtime-properties). Agents connect to RabbitMQ to receive tasks.

Local ports

The following additional ports are exposed on localhost, and used by the manager internally:

High Availability Ports

The following additional ports are used for communication between nodes in a Cloudify Manager cluster:

Nginx

Nginx is a high-performing Web server. In Cloudify Manager, it serves two purposes:

File Server

The file server served by Nginx, while tied to Nginx by default, is not logically bound to it. Although currently it is accessed directly in several occurences (via disk rather than via network), we will be working towards having it completely decoupled from the management environment so that it can be deployed anywhere.

Gunicorn and Flask

Gunicorn is a Web server gateway interface HTTP server. Flask is a Web framework.

Together, Gunicorn and Flask provide the Cloudify REST service. The REST service is written using Flask, and Gunicorn is the server. Nginx, is the proxy to that server. The Cloudify’s REST service is the integrator of all parts of the the Cloudify environment.

PostgreSQL

PostgreSQL is an object-relational database that can handle workloads ranging from small single-machine applications to large Internet-facing applications.

In Cloudify Manager, PostgreSQL serves two purposes:

Logstash

Logstash is a data handler. It can push/pull messages using several inputs, and apply filters and output to different outputs.

Logstash is used by Cloudify to pull log and event messages from RabbitMQ and index them in PostGresSQL.

RabbitMQ

RabbitMQ is a queue-based messaging platform.

RabbitMQ is used by Cloudify as a message queue for different purposes:

Riemann

Riemann is an event stream processor used primarily for monitoring.

Riemann is used within Cloudify as a policy-based decision maker. For more information on policies, see the policies section.

Celery

Celery is a distributed task queue.

The Cloudify management worker, the deployment-specific agents and the host agents are based on Celery.

Management Worker (or Agent)

Both the Workflow Executor and the Task Broker that appear in the diagram are part of the Cloudify Management Worker.

Note that all agents (the Management Worker, and agents deployed on application hosts) are using the same implementation.

InfluxDB and Grafana

InfluxDB is a time-series database.