Agent Packager
The Cloudify agent is basically a virtualenv in which a series of modules are installed and (optionally) to which a some configuration files are attached.
To use Cloudify with distributions other than the officially supported ones, an Agent Packager tool is provided to assist you to create agents for your distributions.
The purpose of the tool is to:
- Address the issues related to compiling module requirements on different distributions, by bridging the gap between user-compiled images, unfamiliar/minor distributions, and so on.
- Enable you to create your own Cloudify agents with your custom plugins.
- Make the agent creation process seamless in terms of there being a single configuration file and a single-line command.
- Enable you to override existing mandatory (and other) modules, by providing your own.
You can use the Cloudify agent packager to create an agent on the distribution on which you are running, that uses your distribution and compilers for modules that require compilation.
- You must use Python 2.7.x or Python 2.6.x to create an agent.
- Not all Cloudify plugins can run on Python 2.6.x. Only basic modules and plugins currently run on Python 2.6.x. To determine whether a plugin supports your Python version, refer to the documentation for the plugin you plan to use.
Creation Process
During the creation process, the agent-packager performs the following actions:
- Creates a virtualenv using your selected Python binary.
- Installs mandatory external modules into the virtualenv.
- Installs modules from a provided requirements.txt file.
- Installs mandatory and optional Cloudify plugins and modules into the virtualenv.
- Installs the
cloudify-agent
module into the virtualenv. - Installs any additional user-selected Cloudify plugins and Python modules into the virtualenv.
- Validates that all specified modules are installed.
- Generates an
included_plugins.py
file. The file is used by thecloudify-agent
module to automatically load all plugins specified in the file. - Creates a TAR file containing the virtualenv.
The tool creates a TAR file for use in Cloudify Linux-based environments. For other environments, a different type of agent might be required.
Installation
pip install cloudify-agent-packager
For development purposes:
pip install https://github.com/cloudify-cosmo/cloudify-agent-packager/archive/master.tar.gz
Usage
IMPORTANT NOTES:
- You must use this tool on the specific version of the distribution on which you intend your agent to run, as it might require compilation.
- You must have the required version of Python installed on your selected image.
- You must have the
tar
binary in your distribution (runwhich tar
to verify that you have TAR installed).
Creating the Agent Packager from the CLI
To create the agent packager from the CLI, do the following:
cfy-ap -h
Script to run Cloudify's Agent Packager via command line
Usage:
cfy-ap [--config=<path> --force --dryrun --no-validation -v]
cfy-ap --version
Options:
-h --help Show this screen
-c --config=<path> Path to config yaml (defaults to config.yaml)
-f --force Forces deletion and creation of venv and tar file.
-d --dryrun Lists the modules to be installed without actually installing them.
-n --no-validation Does not validate that all modules were installed correctly.
-v --verbose Verbose level logging
--version Displays current version
example:
cfy-ap -f -c my_config.yaml -v
Creating the Agent Packager from Python
To create the agent packager from Python, do the following:
import agent_packager.packager as cfyap
config = {} # dict containing the configuration as given in the yaml file.
cfyap.create(config=config,
config_file=None,
force=False,
dryrun=False,
no_validate=False,
verbose=True)
Using the tool from Python enables you to pass the configuration dictionary directly to the creation method, which enables automation of the agent creation process.
The cloudify-agent Module
See here.
Using the Agent
After creating the agent you can do one of the following:
Use the Agent on a Per-Node Basis
You can define the paths to the agent TAR file in a blueprint on a per-node basis. See the cloudify-agent documentation for more information.
Install Agents in Cloudify Manager during Bootstrap
You can provide URLs for agents that you want to provide during Cloudify Manager bootstrap.
Configuring the Tool
The YAML Configuration File
Following is an example configuration file.
distribution: Ubuntu
release: trusty
python_path: '/usr/bin/python'
requirements_file: path/to/my/requirements/file.txt
cloudify_agent_version: master
cloudify_agent_module: http://github.com/cloudify-cosmo/cloudify-agent/archive/master.tar.gz
core_modules:
cloudify_plugins_common: http://github.com/cloudify-cosmo/cloudify-plugins-common/archive/master.tar.gz
cloudify_rest_client: http://github.com/cloudify-cosmo/cloudify-rest-client/archive/master.tar.gz
core_plugins:
cloudify_script_plugin: http://github.com/cloudify-cosmo/cloudify-script-plugin/archive/master.tar.gz
cloudify_diamond_plugin: http://github.com/cloudify-cosmo/cloudify-diamond-plugin/archive/master.tar.gz
additional_modules:
- pyyaml==3.10
additional_plugins:
cloudify_fabric_plugin: http://github.com/cloudify-cosmo/cloudify-fabric-plugin/archive/master.tar.gz
output_tar: Ubuntu-trusty-agent.tar.gz
keep_virtualenv: true
Explanation of the Configuration YAML File
The distribution
and release
variables must correspond with the output generated when running:
python -c "import platform; print platform.dist()"
# e.g. ('Ubuntu', '14.04', 'trusty')
distribution
- The distribution for which the agent is intended. If this is omitted, the tool attempts to retrieve the distribution by itself. The distribution is then used to name the virtualenv (unless explicitly specified invenv
) and to name the output file (unless explicitly specified inoutput_tar
).release
- The release (e.g. precise, trusty) of thedistribution
for which the agent is intended. If this is omitted, the tool will attempt to retrieve the release by itself. The release is then used to name the virtualenv (unless explicitly specified invenv
) and to name the output file (if unless specified in `output_tar’).python_path
- Enables you to set the Python binary to be used when creatingvenv
. (Defaults to/usr/bin/python
).requirements_file
- Path to the requirements.txt file that contains the modules you want to be installed in the agent.cloudify_agent_version
- Specifies the version of thecloudify-agent
module to install (Not required ifcloudify_agent_module
is specified). Note that this can be used to create an agent for a specific Cloudify version.cloudify_agent_module
- Specifies the URL from which thecloudify-agent
module is to be installed. (Ignorescloudify_agent_version
, if specified).core_modules
- Adict
of core modules to install into the virtualenv. (If omitted or with a value offalse
, the module is installed as a part of thecloudify-agent
dependencies.) See a list of current core modules below.core_plugins
- Adict
of core plugins to install into the virtualenv. (If omitted or with a value offalse
, the module is installed as a part of thecloudify-agent
dependencies.) See a list of core plugins below. Ifexclude
is set (per module), it is not installed. Setexclude
with extra care.additional_modules
- Alist
of additional modules to install into the virtualenv. You can add any additional modules that are not Cloudify plugins here.additional_plugins
- Alist
of additional Cloudify plugins to install into the virtualenv.output_tar
- Path to the TAR file you want to create.keep_virtualenv
- Specifies whether to keep the virtualenv after creating the TAR file. Default isfalse
.
Agent Modules
Each agent contains a set of Python packages. These modules can be either simple Python libraries, or plugins.
Core External Modules:
These are modules, which are not developed by Cloudify, that are used by the agent.
- Celery (Mandatory)
Core Modules
These modules are developed by Cloudify and provide core functionality for the agent. The default agents provided with Cloudify come with these modules pre-installed.
- Cloudify REST Client (Mandatory)
- Cloudify Plugins Common (Mandatory)
Core Plugins
These plugins are developed by Cloudify and provide core functionality for the agent. The default agents provided with Cloudify come with these modules pre-installed.
- Cloudify Script Plugin (Optional)
- Cloudify Diamond Plugin (Optional)
The Cloudify Manager also runs an instance of an agent, which is called the cloudify_management_agent
.
This agent is responsible for starting all other agents, and therefore requires the following plugin.
To use the ZeroMQ proxy in
the script plugin, you must explicitly configure it in the additional_modules
section, as shown above.