☰

Cloudify Plugin Development: Best Practices

Overview

Cloudify doesn’t impose too many requirements about how plugins should be developed. This, however, is a two-edged sword, because while there is a lot of flexibility on plugin development, there are certain design decisions that may make it easier to maintain and extend your plugins.

The purpose of this document is to outline what the Customer Success team perceives as “proper” plugin development.

Prerequisite Skills

Overall Structure

Interface First

The most important part of designing a plugin, is designing its TOSCA “view”. Even the most comprehensible plugin is almost entirely useless if customers can’t make proper use of it within blueprints (or TOSCA templates). Therefore, the first and foremost item to focus on should be the node types that are involved. The rationale:

Layered Approach

We propose the following layered approach for designing and implementing a Cloudify plugin:

Design layers

The Third-Party SDK

Third-Party SDKs can offload much of the forward-support and API compatibility challenges. Official SDKs usually release changes in conjunction with feature and service changes.

We prefer to not reinvent the wheel. Therefore, we promote the usage of third-party SDK’s for systems that we integrate with; however, certain rules should be followed:

The Context-Independent Code

(This is the most important part of the entire plugin’s code)

Here comes the implementation of the plugin’s functionality, optionally using third-party SDK’s. The most important design principle here is context independence, which means that the code makes absolutely no assumptions about the context in which it is being run. As a consequence:

The rationale behind this principle is that we want to be able to use this code from anywhere, not only within a Cloudify operation or workflow, thus:

This layer should be designed with reuse and extensibility in mind.

The Cloudify Integration Layer

This should be the simplest layer in the plugin. A good indication of a well-designed plugin is how small this layer is: the more “responsibility” included in this layer, the more likely it is that the design of the context-independent layer could be improved.

In this layer, ideally, we would only have the Cloudify operation (or workflow) functions, doing minimum amount of work and delegating to the lower layer for processing, and then properly handling return values as well as exceptions.

Testing

Plugin code must include tests, as follows:

Packaging

Writing setup.py

Coding

Referring to ctx

The ctx object is made available to operations in two ways:

The traditional approach was to import ctx as a threadlocal and use it:

from cloudify import ctx

...

@operation
my_operation(input1, input2, **kwargs):
  ctx.logger.info('Hello')

While this approach is straightforward when it comes to developing operations, it is cumbersome when considering writing unit tests. That’s because the ctx object needs to be placed as a threadlocal on the current thread, cleaned-up afterwards… leading to unnecessary boilerplate code.

The preferred approach is to avoid importing ctx altogether and just do this:

@operation
my_operation(input1, input2, ctx, **kwargs):
  ctx.logger.info('Hello')

Downloading Resources Using ctx.download_resource

The download_resource function may optionally receive a target_path argument. If it is not specified, the resource is downloaded into a brand new temporary directory, by preserving the original resource’s base name (see CFY-7629).

For example, the following code:

ctx.download_resource('resources/hello.html')

— will result in a random directory created inside the operating system’s temporary directory, and the file hello.html downloaded into it (for example: /tmp/tmp123456/hello.html).

In that case, it is important to remember to not only delete the temporary resource once you’re done with it, but to also delete its parent directory (/tmp/tmp123456 in the example above).

A preferred approach is to provide the target_path argument, and properly dispose of the resource when it’s not needed anymore. For example:

import tempfile
import os
...
# Create temporary file
temp_resource = tempfile.mkstemp()
try:
  ctx.download_resource('resources/hello.html', target_path=temp_resource)
  ...
  ...
finally:
  os.remove(temp_resource)

Exception Handling

Using causes with RecoverableError / NonRecoverableError

When raising one of Cloudify’s exceptions (RecoverableError or NonRecoverableError), as a result of an underlying exception, you should use the “causes” feature when creating the exception class. This ensures that important troubleshooting data is not lost.

For example:

import sys
from cloudify.utils import exception_to_error_cause
from cloudify.decorators import operation
from cloudify.exceptions import NonRecoverableError
...
@operation
def my_operation(ctx):
  ...
  try:
    ... some code ...
  except SomeUnderlyingException as ex:
    _, _, tb = sys.exc_info()
    raise NonRecoverableError('Failed to perform operation', causes=[exception_to_error_cause(ex, tb)])