☰

Kubernetes Plugin

With the Cloudify Kubernetes Plugin you can define Kubernetes resources in your blueprints.

Plugin Requirements

Compatibility

Authentication

There are two authentication methods: token-based and config-based

Note: Kubernetes client certificates are based on the private IP Address of the cluster node. You must use token-based authentication to manage a remote cluster via the public IP address.

Token-Based Authentication

Cloudify Kubernetes Plugin v2.1.0+ includes support for token-based authentication.

Generate Authentication Token

To generate your authentication token, you must:

  1. Create a Service Account and Cluster Role Binding:

a. Create a sa-crb.yaml file on your Kubernetes Master. 

    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: examples-user
      namespace: default
    ---
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: ClusterRoleBinding
    metadata:
      name: examples-user
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: cluster-admin
    subjects:
    - kind: ServiceAccount
      name: examples-user
      namespace: default
b. Install the Service Account and Cluster Role Binding: 
    $ kubectl create -f sa-crb.yaml
    ...
3. Now extract the token: 
$ kubectl -n default describe secret $(kubectl -n default get secret | grep examples-user | awk '{print $1}') | grep 'token:' | awk '{print $2}'
eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6InJlZ3VsYXItdXNlci10b2tlbi1qeHhoNSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJyZWd1bGFyLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJiMGE3MzBiOC0yMTM5LTExZTgtODAxZC00MjAxMGEwYjBjMDQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGVmYXVsdDpyZWd1bGFyLXVzZXIifQ.m06FHyC8TbKZ1bcnxIV_JKpKrADIOYDN4BqEcTMR947fzzfTzU8QiVjYJQF4kCgAR1rC3dNYcQI8rtmwLJg3ttmAoFi_myi38Mb6JyW19vMjxUx3BK8xuiXhcReQyEt0X50koSminwQbqFqMNbtGtODqIyjfe-ePfbdbTV57n16YdtKrhpHuifkWhD26Vyskj1BWs7jmfzPmb8Q7ttKHEIsEgxjTjFxhRPMzp-UxeH1pLnd36tnfUxU9v-6dHCzJUIlYpu-IahhQmTvf5sK5eClT2h3bGJzMtDA2oji_0kFWJ0yemeJuOXX4fNNSeRo9lPPCQIlz1gBNPvSHQngwgQ
4. Copy this token and create a secret on your cloudify manager with it: 
$ cfy secrets create kubernetes_token -s eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6InJlZ3VsYXItdXNlci10b2tlbi1qeHhoNSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJyZWd1bGFyLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJiMGE3MzBiOC0yMTM5LTExZTgtODAxZC00MjAxMGEwYjBjMDQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGVmYXVsdDpyZWd1bGFyLXVzZXIifQ.m06FHyC8TbKZ1bcnxIV_JKpKrADIOYDN4BqEcTMR947fzzfTzU8QiVjYJQF4kCgAR1rC3dNYcQI8rtmwLJg3ttmAoFi_myi38Mb6JyW19vMjxUx3BK8xuiXhcReQyEt0X50koSminwQbqFqMNbtGtODqIyjfe-ePfbdbTV57n16YdtKrhpHuifkWhD26Vyskj1BWs7jmfzPmb8Q7ttKHEIsEgxjTjFxhRPMzp-UxeH1pLnd36tnfUxU9v-6dHCzJUIlYpu-IahhQmTvf5sK5eClT2h3bGJzMtDA2oji_0kFWJ0yemeJuOXX4fNNSeRo9lPPCQIlz1gBNPvSHQngwgQ
Secret `kubernetes_token` created

Reference Authentication Token in a Blueprint

The following is an example blueprint using token-based authentication:

tosca_definitions_version: cloudify_dsl_1_3

imports:
  - http://www.getcloudify.org/spec/cloudify/4.3/types.yaml
  - http://www.getcloudify.org/spec/kubernetes-plugin/2.3.1/plugin.yaml

inputs:

  kubernetes_master_configuration:
    default:
      host: { concat: [ 'https://', { get_secret: kubernetes_master_ip}, ':', { get_secret: kubernetes_master_port } ] }
      api_key: { get_secret: kubernetes_token }
      debug: false
      verify_ssl: false

  kubernetes_api_options:
    description: >
      kubernetes api options
    default: { get_input: kubernetes_master_configuration }

node_templates:

  kubernetes_master:
    type: cloudify.kubernetes.nodes.Master
    properties:
      configuration:
        api_options: { get_input: kubernetes_api_options }

  nginx_deployment:
    type: cloudify.kubernetes.resources.Deployment
    properties:
      definition:
        apiVersion: extensions/v1beta1
        kind: Deployment
        metadata:
          name: nginx-deployment
        spec:
          selector:
            matchLabels:
              app: nginx
          replicas: 2
          template:
            metadata:
              labels:
                app: nginx
            spec:
              containers:
              - name: nginx
                image: nginx:1.7.9
                ports:
                - containerPort: 80
      options:
        grace_period_seconds: 5
        propagation_policy: 'Foreground'
        namespace: 'default'
    relationships:
      - type: cloudify.kubernetes.relationships.managed_by_master
        target: kubernetes_master

Kube Config Authentication

Authentication with the Kubernetes Plugin is via a node that represents the Kubernetes master. The config should be a Kube Config style object.

One of four methods options can be used to provide the configuration:

Example:

  kubernetes_master:
    type: cloudify.kubernetes.nodes.Master
    properties:
      configuration:
        apiVersion: v1
        kind: Config
        preferences: {}
        current-context: kubernetes-admin@kubernetes
        clusters:
        - name: kubernetes
          cluster:
            certificate-authority-data: { get_input: kubernetes_certificate_authority_data }
            server: { concat: [ 'https://', { get_input: kubernetes_master_ip}, ':', { get_input: kubernetes_master_port } ] }
        contexts:
        - name: kubernetes-admin@kubernetes
          context:
            cluster: kubernetes
            user: kubernetes-admin
        users:
        - name: kubernetes-admin
          user:
            client-certificate-data: { get_input: kubernetes-admin_client_certificate_data }
            client-key-data:{ get_input: kubernetes-admin_client_key_data }

When you deploy Kubernetes Cluster with Cloudify Simple Kubernetes Blueprint or Cloudify Kubernetes Provider, secrets containing the configuration are created.

Release History

The information in this documentation is current for Cloudify Kubernetes Plugin version 2.3.1.

See releases.

Example

This example demonstrates demonstrates a basic node template usage.

  my_application:
    type: cloudify.kubernetes.resources.MultipleFileDefinedResources
    properties:
      files:
        - resource_path: resources/my_app_service.yaml
        - resource_path: resources/my_app_pod.yaml
    relationships:
      - type: cloudify.kubernetes.relationships.managed_by_master
        target: master

  master:
    type: cloudify.kubernetes.nodes.Master
    properties:
      configuration:
        file_content: { get_input: kubernetes_configuration_file_content }

Many more examples are available here.

Types

cloudify.kubernetes.nodes.Master

This node represents an existing Kubernetes master.

Properties:

cloudify.kubernetes.resources.FileDefinedResource

This is a Kubernetes resource, such as a pod, service, deployment, which is defined in a file.

Properties:

cloudify.kubernetes.resources.MultipleFileDefinedResources

Provide a list of files containing Kubernetes resources.

Properties:

cloudify.kubernetes.resources.BlueprintDefinedResource

This is the root type of all Kubernetes resource, such as a pod, service, deployment, which is defined in the blueprint inline.

Properties:

Derived resource types:

Check the plugin.yaml for latest additions.

Deletion Propagation

Some Kubernetes resources create other Kubernetes resources. If you delete them, the default behavior of the Kubernetes Python library is to orphan those resources. To prevent this, create a propagation policy:

  nginx_deployment:
    type: cloudify.kubernetes.resources.Deployment
    properties:
      ...
      options:
        grace_period_seconds: 5
        propagation_policy: 'Foreground'
        namespace: 'default'
    relationships:
      - type: cloudify.kubernetes.relationships.managed_by_master
        target: kubernetes_master

cloudify.kubernetes.resources.CustomBlueprintDefinedResource

This allows you to define a resource type that is supported by the Kubernetes API, but has not yet been incorporated into the Kubneretes plugin.

Properties:

Example:

The plugin can be easily extended by referencing create, read, and delete api mappings for these supported APIs.

This is an example of a custom blueprint defined resource:

node_types:

  cloudify.kubernetes.resources.PersistentVolumeClaim:
    derived_from: cloudify.kubernetes.resources.CustomBlueprintDefinedResource
    properties:
      api_mapping:
        default:
          create:
            api: CoreV1Api
            method: create_namespaced_persistent_volume_claim
            payload: V1PersistentVolumeClaim
          read:
            api: CoreV1Api
            method: read_namespaced_persistent_volume_claim
          delete:
            api: CoreV1Api
            method: delete_namespaced_persistent_volume_claim
            payload: V1DeleteOptions

Workflows

In addition to support for built-in workflows, the Kubernetes Plugin supports the following additional workflows:

update_resource_definition

Updates the resource definition of a cloudify.kubernetes.resources.BlueprintDefinedResource.

Parameters

Update Resource Definition Example

Let’s say that you created an nginx pod with the following blueprint resource definition:

  nginx:
    type: cloudify.kubernetes.resources.Pod
    properties:
      definition:
        apiVersion: v1
        kind: Pod
        metadata:
          name: nginx
        spec:
          containers:
          - name: nginx
            image: nginx:stable
    relationships:
      - type: cloudify.kubernetes.relationships.managed_by_master
        target: master

You specified “stable” as the version. Let’s say that you want to update the version.

You would do so like this:

cfy executions start update_resource_definition -d pod -vv -p resource_definition_changes="
{
  'spec': {
    'containers': [{
      'name': 'nginx',
      'image': 'nginx:latest',
    }],
  }
}" -p node_instance_id=nginx_9pqgdu

Further reading

The plugin is based on the Kubernetes Python Client library.