Multi-cloud LocalStack Example

This example demonstrates deploying LocalStack application on a chosen infrastructure. LocalStack is a cloud service emulator that runs in a single container on your local environment. With LocalStack, you can run your AWS applications or Lambdas entirely on your local machine without connecting to a remote cloud provider! Whether you are testing complex CDK applications or Terraform configurations, or just beginning to learn about AWS services, LocalStack helps speed up and simplify your testing and development workflow.

The infrastructure, in the current example, can be one of the following:

• Openstack
• Amazon Web Services (AWS)
• AWS - Terraform
• AWS - Cloudformation
• Google Cloud Platform (GCP)
• Azure

The infrastructure deployment consists of:

• VM
• network
• all of the essential peripherals in each infrastructure (IP address, NIC, etc…).

the second deployment consists of the chosen infrastructure and LocalStack application deployment including all the prerequisites required for LocalStack container to run.

Prerequisites

This example expects the following prerequisites:

• A Cloudify Manager setup ready. This can be either a Cloudify Hosted service trial account, a Cloudify Premium Manager, or a Cloudify Community Manager.
• Access to the cloud infrastructure you select is required to demonstrate this example.

Cloudify CLI or Cloudify Management Console?

Cloudify allows for multiple user interfaces. Some users find the Cloudify Management Console (web based UI) more intuitive while others prefer the Cloudify CLI (Command Line Interface). This tutorial and all following ones will describe both methods.

• Using the Cloudify Management Console
• Using the Cloudify CLI

Cloudify Management Console

This section explains how to run the above described steps using the Cloudify Management Console. The Cloudify Management Console and Cloudify CLI can be used interchangeably for all Cloudify activities.

Create Secrets

To connect to an infrastructure, a set of credentials are required. Cloudify recommends storing such sensitive information in a Cloudify secret. Secrets are kept encrypted in a secure way and used in run-time by the system. Learn more about Cloudify secrets here.

In this example, an infrastructure provider is selected during blueprint install. To ensure the correct secrets are created, use the following table to import the secrets for the provider selected.

To store the access keys as secrets in the Cloudify Manager, login to the Cloudify Management Console and select the System Resources page. Scroll to the Secret Store Management widget and use the Create button to add the following new secrets:

Upload Plugins

Plugins are Cloudify’s extendable interfaces to services, cloud providers and automation tools. I.e., connecting to AWS requires the AWS plugin.

To upload the required plugins to your manager, select the Cloudify Catalog page, scroll to the Plugins Catalog widget and select the plugins you wish to upload.

For this example, upload the following plugins:

• Utilities
• Fabric
• The plugin that matches the infrastructure you’re going to use

Upload Blueprint

A blueprint is a general purpose model for describing systems, services or any orchestrated object topology. Blueprints are represented as descriptive code (yaml based files) and typically stored and managed as part of the source repository. The infrastructure blueprint is available here.

The flow required to setup a service consists of:

1. Upload the blueprint describing the service to the Cloudify Manager.
2. Create a deployment from the uploaded blueprint. This generates a model of the service topology in the Cloudify database and provides the “context” needed for running workflows.
3. Run the install workflow for the created deployment to apply the model to the infrastructure.

Let’s run these one by one.

To upload a blueprint to the Cloudify Manager, select the Local Blueprints page, and use the Upload button.

• Blueprint package: link
• Blueprint name: getting-started
• Blueprint YAML file: mc-localstack.yaml - choose localstack one from the drop down menu

Same can be done from Marketplace page, and use the Upload button.

• Select tab Other
• Select Multi-Cloud-LocalStack-Application-Server-Setup

Deploy & Install

Once the blueprint is uploaded, it will be displayed in the Blueprints widget. to deploy the blueprint click the Create deployment button next to the blueprint you wish to deploy. Specify a deployment name, update any inputs (such as the infrastructure region), and click Deploy & Install. Changing inputs is completely optional and the defaults are safe to use.

You will be directed to the Services page that can be found under Deployments on the left side bar, and will be able to track the progress of the execution.

The deployment you have created should be displayed in the deployments list in the Services page.

Validate

In this example we have setup a simple infrastructure. A virtual instance (VM) was created in the region specified in the Deployment inputs alongside a new network and various other resources.

• Go to your infrastructure (AWS, Azure, etc…) console and see the new instance and other resources that were created.
• Examine the Deployment page in the Cloudify Management Console for more information about your deployed nodes, topology, and view the installation logs.

To login to your new instance, you can look at the Deployment Outputs/Capabilities widget on the Deployment screen to find your instance public IP, SSH username, and SSH private key.

Teardown

To remove the deployment and destroy the orchestrated infrastructure resources, run the Uninstall workflow by clicking the Execute workflow menu next to the deployment, expanding Default workflows, and selecting Uninstall.

Cloudify CLI

Create Secrets

To enable Cloudify to connect to infrastructure, credentials are required. Cloudify recommends storing such sensitive information as a Cloudify secret. Secrets are encrypted in a secure way and used during run-time by the system. Learn more about Cloudify secrets here.

In this example, an infrastructure provider is selected during blueprint install. To ensure the correct secrets are created, use the following table to import the secrets for the provider selected.

Upload Plugins

Plugins are Cloudify’s extendable interfaces to services, cloud providers, and automation tools. For example, connecting to AWS requires the AWS plugin. You may upload specific plugins or, for simplicity, upload the plugin bundle containing all of the basic, pre-packaged, plugins.

To upload the default plugins bundle (this may take a few minutes depending on your internet speed):

cfy plugins bundle-upload

Tip: Read more about plugins and writing your own plugins.

Upload Blueprint and Deploy

A blueprint is a general purpose model for describing systems, services or any orchestrated object topology. Blueprints are represented as descriptive code (YAML-based files) and are typically stored and managed as part of the source code repository.

The infrastructure blueprint is available here.

Uploading a blueprint to Cloudify can be done by direct upload or by providing the link in the source code repository. The flow to do that is:

1. Upload the blueprint.
2. Create a deployment from the uploaded blueprint. This generates a model of the service topology in the Cloudify database and provides the “context” needed for running workflows.
3. Run the install workflow for the created deployment to apply the model to the infrastructure.

In order to perform this flow as a single unit, we will use the install command.

cfy install https://github.com/cloudify-community/blueprint-examples/releases/download/latest/getting-started.zip -n mc-localstack.yaml -i infra_name=<YOUR_INFRASTRUCTURE_NAME>

Replace YOUR_INFRASTRUCTURE_NAME with any of the following -

• openstack
• azure
• aws
• aws-terraform
• aws-cloudformation
• gcp

Validate

In this example we have setup a simple infrastructure. A virtual instance (VM) was created in the region specified in the Deployment inputs alongside a new network and various other resources.

• Go to your infrastructure console and see the new instance and other resources that were created.

• You can easily get a list of all deployed nodes by running:

  Listing nodes for deployment getting-started.mc-localstack...
  
  Nodes:
  +----------------+---------------------------+---------------------------+---------+----------------------------------+------------+----------------+---------------------+-----------------------------+------------+
  |       id       |       deployment_id       |        blueprint_id       | host_id |               type               | visibility |  tenant_name   | number_of_instances | planned_number_of_instances | created_by |
  +----------------+---------------------------+---------------------------+---------+----------------------------------+------------+----------------+---------------------+-----------------------------+------------+
  | infrastructure | getting-started.mc-localstack | getting-started.mc-localstack |         |     cloudify.nodes.Component     |   tenant   | default_tenant |          1          |              1              |   admin    |
  |   localstack   | getting-started.mc-localstack | getting-started.mc-localstack |         | cloudify.nodes.SoftwareComponent |   tenant   | default_tenant |          1          |              1              |   admin    |
  +----------------+---------------------------+---------------------------+---------+----------------------------------+------------+----------------+---------------------+-----------------------------+------------+
  
  Showing 2 of 2 nodes

Tip: To check out some more commands to use with the Cloudify Management Console, run cfy --help

To get the Outputs of our deployment run:

cfy deployment outputs getting-started.mc-localstack

The returned output would look like:

Retrieving outputs for deployment getting-started.mc-localstack...
 - "localstack_endpoint":
     Description: Endpoint of the LocalStack application
     Value: http://10.239.0.22:4566

Copy and paste the URL Value into your browser, you should see a simple web page {“status”: “running”}

An even easier way to review your deployment is through the Cloudify Management Console. Login to the console and browse to the Deployments page. Select the deployment (getting-started.mc-localstack) and explore the topology, inputs, outputs, nodes, and logs.

This is also a good time to examine the blueprint used in the example. The blueprint can be examined in the Cloudify Management Console, however in this case we will go to the Cloudify examples repository in Github and examine it there: mc-localstack.yaml

To start interacting with Localstack, please follow the steps below:

• Run install_lscli.sh on your Linux host ( CentOS 7.6 and Ubuntu 18.04). Script can be found under scripts/localstack in the blueprint package
  
• install_lscli.sh script will install AWS CLI and AWS CLI local
• AWS CLI local package provides the awslocal command, which is a thin wrapper around the aws command line interface for use with LocalStack.

Localstack installation verification and basic usage

• Once install_lscli.sh script executed and awslocal installed on your box you can check that LocalStack is up and running. Default port is 4566

  curl <IP address>:<localstack port>
  curl localhost:4566 - if cli installed on the same host where `LocalStack` is running
  curl <public IP address>:4566 - if installed on a separate host.
  
  Expected command output when localstack is running 
  {"status": "running"}

• Basic usage

Create s3 bucket, list s3 bucket, copy files to s3 bucket, copy files to s3 bucket. For more detailed info related to cli usage please refer here. See few examples below.

 awslocal --endpoint-url=http://<public IP address>:4566 s3 mb s3://mytestbucket
 awslocal --endpoint-url=http://<public IP address>:4566 s3 ls
 awslocal --endpoint-url=http://<public IP address>:4566 s3 cp test.txt s3://mytestbucket
 awslocal --endpoint-url=http://<public IP address>:4566 s3 ls s3://mytestbucket

Teardown

To remove the deployment and delete all resources, simply run the uninstall command:

cfy uninstall getting-started.mc-localstack