Security

Security, in the context of a Cloudify Manager, means securing the communication with the manager and controlling who is allowed to use it to execute various operations.
Secured communication is achieved by using SSL, which allows clients to validate the authenticity of the manager as well as ensure the data sent to and from it is encrypted.
Controlling access to the manager and permissions to take actions is implemented via Flask-SecuREST - an open framework that is hooked to Cloudify’s request processing, to support user authentication and authorization.
More details on Cloudify’s SSL and Access Control implementation and configuration can be found below.

Cloudify’s security focuses on the REST service, since this is the first and only access point of clients to the Cloudify manager.
When security is enabled, all requests to the manager are authenticated and authorized before reaching their endpoint.
For example, when a Web-UI user attempts to upload a new blueprint, a request is sent to the REST service’s /blueprints endpoint. The request will only reach the endpoint if the user is logged in and is authorized to upload blueprints.
Similarly, a user that executes the CLI command cfy deployments list triggers a request to execute GET on /deployments, which will only be successful if it includes valid credentials, identifying an authorized user.
Requests generated by other HTTP clients (e.g. curl) must also include correct credentials.
If credentials are missing, invalid or represent an unauthorized user, the request fails with a “401: Unauthorized User” error.

Security configuration

Security configuration is done in the manager blueprint. The default settings are specified in manager-types.yaml, in this path:

  
node_types:
  ...
  cloudify.nodes.MyCloudifyManager:
    ...
    properties:
      ...
      security:
  

The default configuration can be overridden by specific manager blueprint yaml files. Each setting is described in detail in the following sections.

Setting Security On / Off

The first security setting is:

  
enabled: { get_input: security_enabled }
  

Security is enabled or disabled according to the input value of security_enabled. In order to activate security set the input value security_enabled to true.

SSL

Using SSL for client-server communication enhances security in two aspects:
1. Privacy - All communications between the client the server are encrypted
2. Trust - When a connection is established, the Cloudify manager presents a signed certificate to the client. The client can use that certificate to validate the authenticity of the manager.

Requests to the manager can be addressed to its public or private IP address.
By default, internal requests (i.e. requests sent from the manager itself or from agent hosts) are sent to the manager’s private IP address, while external requests (i.e. requests originating from other, external clients) should be sent to the manager’s public address.

Depending on your environment, set the routing of internal and external requests to the preferred address (see rest_host_internal_endpoint_type and rest_host_external_endpoint_type below).

SSL configuration

1. Enabling SSL
   SSL is enabled and disabled according to the input field ssl_enabled. By default, SSL is disabled. In order to enable SSL set ssl_enabled: true. When enabled, every request to the manager must be sent over https to port 443.
   
   
2. Setting internal and external request routing
   By default, internal requests are sent to the manager’s private IP address. To override this behavior, set rest_host_internal_endpoint_type: public_ip. On the contrary, external requests are expected to be sent to the manager’s public address. To override this, set rest_host_external_endpoint_type: private_ip
3. Required Files
   Upon bootstrap, the key pairs matching the internal and external addresses can be provided, by placing them in the manager blueprint’s directory (on the client machine), under ”/resources/ssl” :
   
   • external_rest_host.crt - the certificate presented to requests that address the manager’s public IP
     
   • external_rest_host.key - the matching key
     
   • internal_rest_host.crt - the certificate presented to requests that address the manager’s private IP
     
   • internal_rest_host.key - the matching key
     If the key pairs are not provided at bootstrap, self-signed certificates and matching keys will be created during bootstrap.

Manager Access Control

Cloudify’s Access Control is comprised of two steps - authentication and authorization. While authentication is mandatory when accessing a secured resource, authorization is optional. The sections below detail the access control components and their configuration.

Authentication

Userstore Drivers

While Cloudify does not manage user accounts, some authentication methods may require access to an external userstore - a repository of users - from which a user object is retrieved. In most cases, the userstore is a directory (e.g. ActiveDirectory) or a database. Userstore Drivers are classes that implement access to different userstore types. Cloudify assumes there is (at most) one userstore from which all users can be loaded.

Userstore driver configuration

Under “userstore_driver” a single userstore is set. Below is the default userstore configuration, set in manager-types.yaml:

  
  security:
    ...
    userstore_driver:
      implementation: flask_securest.userstores.simple:SimpleUserstore
      properties:
        userstore:
          users:
            - username: { get_input: admin_username }
              password: { get_input: admin_password }
              groups:
                - cfy_admins
          groups:
            - name: cfy_admins
              roles:
                - administrator
  

The userstore_driver configuration includes two keys:

• implementation - the fully qualified name of the module implementing a userstore driver, followed by “:” and the class name.
  In the above configuration, flask_securest.userstores.simple is the module, and SimpleUserstore is the class name.
  
• properties - a dictionary of arguments required to instantiate the implementing class. The arguments will be passed as kwargs to the class’ __init__ method.

Simple Userstore driver

The default configuration uses Flask-SecuREST’s simple userstore. In this implementation, the userstore is a simple data-structure defined in the manager blueprint and instantiated during bootstrap.

Configuring the Simple Userstore:

• implementation - The implementation field should point to flask_securest.userstores.simple:SimpleUserstore.
• properties - Holds the actual userstore to be used as demonstrated in the example above.

  Note

  The configuration above describes a userstore having a single user, that accepts its username and password from input values (admin_username and admin_password respectively). This implementation is good for demonstration purposes. Nevertheless, Configuring a “real” userstore driver (e.g. ActiveDirectory) is just as easy, as explained later.

File Userstore driver 3.3.1 FEATURE

The file based userstore provides the ability to define users and groups in an external file located on the manager host. This file can later be edited and will be reloaded by the Flask-SecuREST framework upon modification. A sample userstore file can be found here.

Configuring the File Userstore:

• implementation - The implementation field should point to flask_securest.userstores.file_userstore:FileUserstore.
• properties - userstore_file_path specifies the file’s remote target path.

  Important note

  Users running Cloudify version 3.3.0 are required to modify the rest-service’s create.sh to also copy the userstore file on bootstrap similarly to the roles_config.yaml file.

Authentication Providers

Authenticating a request can be done in different ways - matching passwords, binding to a directory service or delegating the login to an external service (oAuth). Each implementation of an authentication methods is called an Authentication Provider (AKA Authenticator). When processing a request, Cloudify calls the registered authenticators (one or more) in the order of definition in the manager-blueprint. If the first authenticator fails to authenticate the request - the second will be used, and so on, until successful.
If none of the authenticators succeeded, a “401: Unauthorized User” error is returned.

Authentication providers configuration

Under “authentication_providers” is a list of all authenticators in the order they should be executed. At least one authentication provider must be set. The default configuration set in manager-types.yaml uses two authenticators - “password” and “token”:

  
  security:
    ...
    authentication_providers:
      - name: password
        implementation: flask_securest.authentication_providers.password:PasswordAuthenticator
        properties:
          password_hash: plaintext
      - name: token
        implementation: flask_securest.authentication_providers.token:TokenAuthenticator
        properties:
          secret_key: my_secret
  

Each Authentication Provider configuration includes these keys:

• name - a unique name describing this authenticator. This name will appear in logs so it should be clear.
  
• implementation - the fully qualified name of a module implementing an authentication provider, followed by “:” and the class name.
  
• properties - a dictionary of arguments required to instantiate the authentication provider class. The arguments will be passed as kwargs to the class’ __init__ method.

According to the above configuration, Cloudify will use two methods to authenticate requests - matching passwords, or, if that fails, processing a token.
If none of the authenticators succeed - a “401: Unauthorized User” error is returned.

Password authentication
Cloudify’s first authenticator is Flask-SecuREST’s password authenticator, which uses basic HTTP authentication. The request is expected to include an “Authorization” header which contains a base64 encoded [username]:[password] value. Once decoded, the username is used for retrieving the user object from the userstore, in order to compare the given password to the stored one.

Token authentication
Cloudify’s second authenticator is Flask-SecuREST’s token authenticator. The request is expected to include a header named “Authentication-Token”, in which the username is stored. The token is signed and timed, meaning a secret key is required to open it, and it will expire some time after creation. Once opened, the username found is used to load the user object from the userstore.

It is possible to implement other authentication providers, including providers that do not require accessing a userstore directly (e.g. oAuth). This is explained later in this document.

Token Generation

In order to send a token with each request, the client must first obtain a token. Tokens can be generated by many systems, and they will work as long as a registered authentication provider knows how to process them. Cloudify exposes the /tokens endpoint to help the user obtain a token easily, even from external systems. To enable this feature a token generator must be set. Then any GET request to /tokens will call the token generator and return a token to the user.
Do note that the request to /tokens is in itself authenticated.

Configuring a Token Generator

Under “auth_token_generator” a single generator is set. Below is the default token generator set in manager-types.yaml:

  
  security:
    ...
    auth_token_generator:
      implementation: flask_securest.authentication_providers.token:TokenAuthenticator
      properties:
        secret_key: my_secret
        expires_in_seconds: 600
  

The auth_token_generator configuration includes two keys:

• implementation - the fully qualified name of the module implementing a token generator, followed by “:” and the class name.
  In the above configuration, flask_securest.authentication_providers.token is the module, and TokenAuthenticator is the class name.
  
• properties - a dictionary of arguments required to instantiate the implementing class. The arguments will be passed as kwargs to the class’ __init__ method. In the configuration shown above two properties are set:
  • secret_key - used to sign the token
  • expires_in_seconds - limits the lifetime of a token to 10 minutes. A token older than 10 minutes will be expired and fail the request.

    Secret key selection

    The secret key used by the token generator to sign the token must match the secret key used by the token authenticator to decrypt it.

Authorization

After authenticating the request’s user, authorization will take place, if an authorization provider is configured. An authorization provider is used to verify that the user has the permission to execute the requested method (e.g. GET) on the requested endpoint (e.g. /deployments).

Default authorization process

By default, Cloudify uses Flask-SecuREST’s Role-Based Authorization Provider to authorize users. Role-based authorization providers evaluate users permissions based on their assigned roles.
A user can have one or more roles, loaded by the Simple Role Loader; each role can give the user permissions to access some endpoints methods, and deny others.
See the configuration sections below for more details on roles and permission assignment.

Authorization provider configuration

Under “authorization_provider” a single provider is set. Below is the default authorization configuration, set in manager-types.yaml:

  
  security:
    ...
    authorization_provider:
      implementation: flask_securest.authorization_providers.role_based_authorization_provider:RoleBasedAuthorizationProvider
      properties:
        roles_config_file_path: '/opt/manager/roles_config.yaml'
        role_loader:
          implementation: flask_securest.authorization_providers.role_loaders.simple_role_loader:SimpleRoleLoader
  

The authorization_provider configuration includes two keys:

• implementation - the fully qualified name of the module implementing a authorization provider, followed by “:” and the class name.
  In the above configuration, flask_securest.authorization_providers.role_based_authorization_provider is the module, and RoleBasedAuthorizationProvider is the class name.
  
• properties - a dictionary of arguments required to instantiate the implementing class. The arguments will be passed as kwargs to the class’ __init__ method.
  In the configuration shown above two properties are set:
  • roles_config_file_path - this is the location - on the manager server - of the YAML file that maps roles to permission. Setting a different path requires modifying the REST creation script accordingly (found at ”/components/restservice/scripts/create.sh”, relative to the main manager blueprint file directory).
    
  • role_loader - this is the class that loads the roles of the acting user. By default, the Simple Role Loader is used, which loads roles from the simple userstore, inlined in manager-types.yaml.

Role assignment configuration

The Simple Role Loader load user roles from the simple userstore defined in manager-types.yaml. Here is a possible configuration:

  
  userstore_driver:
    implementation: flask_securest.userstores.simple:SimpleUserstore
    properties:
      userstore:
        users:
          - username: alice
            password: alice_password
            groups:
              - cfy_admins
          - username: bob
            password: bob_password
            groups:
              - cfy_deployers
          - username: clair
            password: clair_password
            roles:
              - viewer
          - username: dave
            password: dave_password
        groups:
          - name: cfy_admins
            roles:
              - administrator
          - name: cfy_deployers
              roles:
                - deployer
  

The above userstore configuration defines four users (alice, bob, clair and dave) and 2 groups (cfy_admins and cfy_deployers).

• alice - a member of the “cfy_admins” group, which is assigned the “administrator” role. This means alice has the “administrator” role.
• bob - a memeber of the “cfy_deployers” group, which is assigned the “deployer” role. bob therefore has this role.
• clair - not a member of any group, but is personally assigned the “viewer” role.
• dave - not a member of any group, and has no roles assigned to him directly. He there has no permissions at all.

Role permissions

Linking roles to permissions is done in a YAML file - by default this is /resources/rest/roles_config.yaml, relative to the main manager blueprint file directory.
The file contains dictionaries, in which the keys are role names and the values are permissions (also dicts). Permissions are divided to “allow” or “deny”, and specify endpoints and their HTTP methods. This is a possible (not default) configuration:

  

################################################################################
# The administrator role can access any endpoint, and call any method
################################################################################
administrator:
  allow:
    '*':
      - '*'
################################################################################
# The deployer role can access any endpoint, and call any method except DELETE
################################################################################
deployer:
  allow:
    '*':
      - '*'
  deny:
    '*':
      - DELETE
#############################################################################
# The viewer role can can access any endpoint, but only call the GET method
# it is also denied access to a specific blueprint
#############################################################################
viewer:
  allow:
    '*':
      - GET
  deny:
    '/api/v2/blueprints/blueprint_2':
      - '*'
  

The above configuration defines permissions of three roles:

• administrator - allowed to access any endpoint (the upper ‘*’) and execute any method (the inner ‘*’)
• deployer - allowed to access any endpoint and execute any method, but also denied the DELETE method on all endpoints.
  This practically means this role can execute any method on any endpoint except call DELETE.
• viewer - allowed to execute the GET method (and only that method) on all endpoints. Additionally, this role is denied access to blueprint_2 entirely! all the methods on this endpoint are denied.

Auditing

Security operations, such as authentication success or failure and user details, are audited in dedicated log file on the management server.
The default configuration is:

  
audit_log_file: /var/log/cloudify/rest-security-audit.log
audit_log_level: INFO
audit_log_file_size_MB: 100
audit_log_files_backup_count: 20
  

• audit_log_file - sets the full path to the auditing file on the manager
• audit_log_level - modifying the log level will produce less or more elaborate security auditing; valid values are: CRITICAL, ERROR, WARNING, INFO or DEBUG.
• audit_log_file_size_MB - limits the log file size. By default, the file is limited to 100 MB. When the file reaches that size, it will be renamed with the extension “.1” and a new log file will be created (older files will be renamed with the extension “.2”, “.3” and so forth).
• audit_log_files_backup_count - sets the maximum number of old log files to keep. By default this value is set to 20. That means that up to 20 old log files can be created, after which the oldest file will be removed.

Clients

Different clients can be used to send requests to the REST service, and all go through an authentication process. Here we review what each client requires in order to send a secured request.

CLI - cfy commands

credentials

Cfy commands automatically add the “Authorization” header to each request sent to the manager, containing the username and password of the current user. Setting the user credentials is done once, by exporting these environment variables:

• export CLOUDIFY_USERNAME=my_username
• export CLOUDIFY_PASSWORD=my_password

Client SSL configuration

Connecting to the manager over SSL
After bootstrap, and if SSL is enabled, requests sent to the manager must use the ‘https’ protocol and address port 443.
The bootstrapping client will already be set accordingly; Other clients that attempt to connect to the secured manager will need to specify port 443 when calling cfy use:

  
cfy use -t <manager-ip-address> --port 443
  

Verifying the manager’s certificate
By default, the CLI attempts to validate the manager’s certificate using public CAs (Certificate Authorities). The following environment variables can alter that behavior:

• LOCAL_REST_CERT_FILE - if a local (client-side) copy of the server’s certificate should be used for SSL cert validation, set it’s location on the client machine using this environment variable To enable this feature, set CLOUDIFY_SSL_CERT to the client’s local path to the manager’s certificate.
• CLOUDIFY_SSL_TRUST_ALL - to accept the manager’s certificate without validation, set this environment variable to True (or any non-empty value)

Web UI

Credentials for all requests sent from the Web UI are set at Login. Do note that this client only supports authentication by username and password. Other authentication methods (e.g. tokens) are currently not supported.

Cloudify’s REST client

Internally, Cloudify’s CLI actually uses Cloudify’s REST client, which supports the variety of security configurations discussed in this guide.
To use it, create an instance of CloudifyClient, initialized with values that match your configuration:

• host - The manager’s host name or IP address. Default: ‘localhost’
• port - The port to send requests to. Default: 80
• protocol - the protocol to send requests on. Default: ‘http’
• headers - headers to send with each request. Default: None
• cert - path to a client copy of the server’s certificate (e.g. to validate a self-signed certificate). Default: None
• trust_all - if set to True, SSL certificates will be trusted without validation; if False - they will be validated. Default: False

Here are two usage examples:

• Creating a REST client using credentials, to get a token from a secured manager without SSL

    
  from itsdangerous import base64_encode
  from cloudify_rest_client import CloudifyClient
  
  headers = {'Authorization': 'Basic ' + base64_encode('MY_USERNAME:MY_PASSWORD')}
  rest_client = CloudifyClient(host='1.2.3.4', headers=headers)
  token_value = client.tokens.get().value
    

• Creating a REST client using a token, to get the status of a secured manager with SSL

    
  from cloudify_rest_client import CloudifyClient
  
  headers = {'Authentication-Token': 'this_is_some_token_value'}
  rest_client = CloudifyClient(host='1.2.3.4', port=443, protocol='https', headers=headers)
  response = client.manager.get_status()
    

External clients

Other REST clients (e.g. cURL) must explicitly add the required header to each request.
For example:

• Get the server’s status, authenticate with username and password:
  

     curl -u 'MY_USERNAME':'MY_PASSWORD' <manager-ip-address>:<port>/api/v2/status  

• Get a token, authenticate with username and password:
  

     curl -u 'MY_USERNAME':'MY_PASSWORD' <manager-ip-address>:<port>/api/v2/tokens   

• Get all the blueprints, authenticate with a token:
  

     curl -H 'Authentication-Token:MY_TOKEN' <manager-ip-address>:<port>/api/v2/blueprints   

Advanced

Architecture

Cloudify’s security focuses on the REST service, marked in color in this diagram:
Clients communicate with the manager by sending http(s) requests to the REST service, which in turn processes these requests and communicates with internal management components (e.g. RabbitMQ, Elasticsearch).

Secured Request Flow

The above diagram illustrates the secured request flow:

1. The client sends a request to server_ip/endpoint
   
2. Nginx receives the request. If SSL is enabled, Nginx presents the server’s SSL certificate to the client.
   The client can then validate the certificate. If validation fails - the request fails, as the server cannot be trusted.
   If the certificate is validated successfully (or accepted without validation) - request processing continues.
   
3. Is the requested endpoint defined as “secured”?
   Generally, all endpoints are considered secured, which leads to step #4. The only exception is /version, which is open and does not require further access control checks.
   
4. Attempt to authenticate the request user - is this user valid? is it known to Cloudify?
   Authentication is performed by the registered authentication providers, as explained below.
   If the user is not authenticated - a “401: Unauthorized User” error is returned.
   
5. Attempt to authorize the request user - is the user allowed to access the endpoint and execute the requested method on it (e.g. GET, POST)?
   Authorization is performed by the registered authorization provider, as explained below. If a provider isn’t registered, this step will be skipped.
   If the user is not authorized - a “401: Unauthorized User” error is returned.
   Otherwise - congrats! the request can reach its endpoint!

Customize Security: Use Your Own Implementations

Implementing custom security components

As shown in previous sections, registering custom python implementations is fairly easy. To replace the default userstore, token generator, authenticators or authorization provider - simply set the path to the custom class and add the properties required to instantiate it.
Implementing security components is just as easy - simply follow the APIs explained below.

Userstore driver implementation

A userstore driver is a class that loads user details and returns them as a user object.
A valid userstore implementation is a:

• Python class
• Inherits from AbstractUserstore
• Implements get_user(self, identifier), which returns a user object containing a dictionary of user details read from the userstore. If a matching user is not found, get_user should return None.

Authentication provider implementation

An Authentication Provider is a class that performs authentication. It has access to the userstore instance (if a userstore was configured) and can therefore use it to get user details that are required to perform authentication (e.g. check if the user account is disabled).
A valid authentication provider implementation is a:

• Python class
• Inherits from AbstractAuthenticationProvider
• Implements authenticate(self, userstore=None), which returns a unique user identifier (e.g. username) if authentication was successful, and raises an exception if it failed.
  Exception messages should be informative but not expose confidential user or system details. For example: “Request authentication header is empty or missing” is OK, while “username jason attempted to use wrong password 123456” reveals too much information.

Token generator implementation

A token generator is a class that generates tokens and returns them. These tokens should of course be supported by one of the registered authenticators.
A valid token generator implementation is a:

• Python class
• Implements generate_auth_token(self), which returns a token.

Authorization provider implementation

An authorization provider is a class that performs the authorization logic, after the user authenticity has been verified. Authorization should evaluate if the acting user is allowed to execute the requested methods (e.g. POST) on the requested endpoint (e.g. /blueprints). A valid authorization provider is a:

• Python class
• Inherits from AbstractAuthorizationProvider
• Implements authorize(self), which returns true if the user is authorized, or false otherwise.

Packaging, Configuring and Installing Custom Implementations

In order to use custom security components, each component must be structured as a valid python package and installed on the REST service environment. Installation is performed via the plugins mechanism, which requires the package location be added to the plugins property of the rest_service node in manager-types.yaml:

  
node_types:
  ...
  manager.nodes.RestService:
    ...
    properties:
      ...
      plugins:
        oauth_authentication_provider:
          # see description below
          source: my-security-plugins/oauth-authentication-provider
          # see description below
          install_args: '--pre'

        mysql_userstore:
          # see description below
          source: https://github.com/my-org/mysql-userstore-driver/archive/master.zip
  

The plugins section is a dict that contains all plugins that should be installed. The keys of this dict are arbitrary names that represents the plugin’s function. Use meaningful names to make the configuration more readable. In the example shown above we use oauth_authentication_provider and mysql_userstore.

• source - can be any of the following:
  • A path to the package directory (a valid python package) relative to the main manager blueprint file directory (e.g. my-security-plugins/oauth-authentication-provider)
  • A URL to the package archive (e.g. https://github.com/my-org/mysql-userstore-driver/archive/master.zip)
  • A path/url to a Wagon package archive. 3.3.1 FEATURE
• install_args - optional additional arguments to the pip install command used to install your plugin.

  insalling *.wgn packages

  Wagon based rest plugin installation requires passing the appropriate install_args to the pip command. Normally, pip arguments for Wagon based packages would look like so:
  <plugin-name> -r <req_file> --use-wheel --no-index --find-links=wheels/ --pre

Examples

LDAP Userstore example

LDAPUserStore is an example of custom userstore driver that reads users from LDAP. The implementing class inherits from AbstractUserstore and implements the get_user method as required.

The properties to initialize this class should be specified in the manager blueprint as described earlier in Userstore driver configuration, e.g.

  
userstore_driver:
  implementation: flask_securest.userstores.examples.ldap_userstore:LDAPUserStore
  properties:
    admin_dn: cn=admin,dc=cloudify,dc=org
    admin_password: password
    directory_url: ldap://localhost:389
    root_dn: dc=cloudify,dc=org
    identifying_attribute: uid
    username_attribute: uid
    user_password_attribute: userPassword
    user_email_attribute: mail
    is_active_attribute: is_active
  

In order to use this custom userstore implementation, it must be installed on the REST service as describe in Packaging, Configuring and Installing Custom Implementations, e.g.:

  
node_types:
  ...
  manager.nodes.RestService:
    ...
    properties:
      ...
      plugins:
        ldap_userstore:
          source: ldap-userstore
  

  
my_manager_blueprint:
  my-manager-blueprint.yaml
  ldap-userstore:
    ldap_userstore:
      __init__.py
      simple_ldap_userstore.py
    README.md
    setup.py
  

 
sudo yum install python-devel openldap-devel gcc -y
  

LDAP authentication provider example: 3.3.1 FEATURE

The cloudify-ldap-plugin provides the ability to authenticate users against any LDAP endpoint. Configuring the ldap authentication driver:

  
authentication_providers:
  implementation: authentication.ldap_authentication_provider:LDAPAuthenticationProvider
  name: ldap_authentication_provider
  properties:
    'directory_url': ldap://x.x.x.x:389
  

  
node_types:
  ...
  manager.nodes.RestService:
    ...
    properties:
      ...
      plugins:
        ldap_authentication_provider:
          source: https://github.com/cloudify-cosmo/cloudify-ldap-plugin/archive/1.0.zip
          install_args: '--pre'
  

• No Wagon package - Userdata script should include sudo yum install python-devel openldap-devel gcc -y
  
• Using Wagon package - Userdata script should only include sudo yum openldap-devel -y