Deploying to Cloud#

Introduction#

OctaiPipe’s Cloud Deployment feature provides a powerful and efficient solution for managing deployment configurations through a simple command-line interface. It is built on top of Kubernetes, and allows users to abstract constellations of Kubernetes objects as discrete deployment units. The configuration, deployment, and management of these units is handled seamlessly, massively streamlining the deployment of infrastructure features.

Although any infrastrcuture feature can be deployed using this feature, the major use-case for OctaiPipe users is deploying octaisteps and models.

Users have the flexibility to use out-of-the-box deployment configuration templates for common use-cases, or creating custom templates to suit specific needs. This user guide provides information on using OctaiPipe’s Cloud Deployment feature, including instructions on both out-of-the-box and bespoke deployment types.

Requirements#

  • Connection to OpenVPN

  • Kubectl

Launching a Cloud Deployment#

The only requirement to launch a deployment is a config file detailing the type of deployment and its settings. See here for details about Deployment Configs.

Deployment Config#

Deployment Configs are yaml files which define everything necessary to create a deployment.

Configs are passed to the octaideploy to create the deployment(See FUNCTION).

A generic deployment config takes the following fields:

  • name string

    • Required

    • The name of the deployment.

    • Can be used to reference the deployment. Must be unique.

  • type string

    • Required

    • The type of cloud deployment being run

    • Corresponds to the type value in the cloud deployment class

  • template string

    • Path to deployment template (see above)

    • Defaults to the default template of the deployment in question.

    • Out-the-box ones can be found here src/octaipipe/configs/cloud_deployment

  • kubernetes_namespace string

    • The kubernetes namespace in which to deploy

    • If not passed, environment variables are checked for kubernetes_namespace. Otherwise defaults to default (effectively no namespace)

    • Note: we recommend setting this to ‘colab’; this makes sure the deployment has the necessary privileges to run properly. The ‘colab’ namespace is created as part of the OctaiPipe infrastructure installed during onboarding.

  • deployment_specs dict

    • Required

    • Used to replace placeholders in the deployment template. (see above)

    • key/values defined here take precedence over environment variables and default values when replacing placeholders.

All deployments take the above config fields. Out-the-box deployments can take more depending on specific configuration requirements. Details of this can be found in the section dedicted to out-the-box deployment types.

Example#

 1name: octaipipe-weight-forecast
 2type: stepDeployment
 3kubernetes_namespace: example-step
 4template: /path/to/bespoke/template.yml
 5
 6deployment_specs:
 7  schedule: "0 0 * * *"
 8  docker_image: octaipipe:latest
 9  container_name: octaipipe
10  commands:
11    - "octaistep --local-step-path ./etl/weight_pipeline_step.py  --config-path ./etl/weight_monthly.yml"
12    - "octaistep --local-step-path ./weight_forecasting/forecasting_step.py  --config-path ./weight_forecasting/configs/forecasting.yml"

The above will create deploy an octaistep process from an octaipipe image in a namespace called example-step to run daily at midnight (schedule). The process involves the two commands listed under commands.

Command: octaideploy()#

Cloud deployments are set up and controlled using the octaideploy function from the octaipipe.deployment.edge.octaideploy module.

To use the cloud deployment pattern, set the cloud parameter to True.

The octaideploy function takes the following parameters:

config_path (str): Points to a config for initializing a cloud deployment. If passed, the only valid compose_action is ‘start’ as all other action commands only run on previously deployed deployments.

deployment_id (str): Loads a previously deployed deployment. Any compose_action is valid with this parameter, and the action will be run on the existing deployment.

compose_action (str): The action to perform on the deployment. Valid actions are:

‘start’: Runs a deployment from a passed config file OR runs a pre-existing deployment from the manifest files held in the manifests column (loaded using deployment_id). ‘status’: Gets the status of a deployment by running kubectl get all on the manifests related to the deployment. ‘stop’: Stops the deployment from running. ‘restart’: Calls stop and then start. ‘down’: Stops a deployment and removes all associated objects (including database records, etc.). Running an action command without a config_path OR a deployment_id results in the action being applied to all cloud deployments. This is a drastic action; for example, running octaideploy(cloud=True, compose_action=’down’) would down all active deployments. As such, a y/n prompt is required to confirm the command. The exception to this is the status action command which will print to the terminal a JSON object containing deployment IDs and their desired status, e.g., {‘48c487f3’: ‘running’, ‘fa62df73’: ‘running’}.

Examples#

Start and configure a new deployment using the config path provided:

from octaipipe.deployment.edge.octaideploy import octaideploy

octaideploy(config_path='<path-to-config-file>', compose_action='start', cloud=True)

Start a pre-existing deployment:

from octaipipe.deployment.edge.octaideploy import octaideploy

octaideploy(deployment_id='<id-of-existing-deployment>', compose_action='start', cloud=True)

Stop a running deployment:

from octaipipe.deployment.edge.octaideploy import octaideploy

octaideploy(deployment_id='<id-of-existing-deployment>', compose_action='stop', cloud=True)

Clear all traces of a deployment:

from octaipipe.deployment.edge.octaideploy import octaideploy

octaideploy(deployment_id='<id-of-existing-deployment>', compose_action='down', cloud=True)

Restart an existing deployment:

from octaipipe.deployment.edge.octaideploy import octaideploy

octaideploy(deployment_id='<id-of-existing-deployment>', compose_action='restart', cloud=True)

Log the status of a deployment to the terminal:

from octaipipe.deployment.edge.octaideploy import octaideploy

octaideploy(deployment_id='<id-of-existing-deployment>', compose_action='status', cloud=True)

Down all cloud deployments (requires confirmation):

from octaipipe.deployment.edge.octaideploy import octaideploy

octaideploy(compose_action='down', cloud=True)

Return the ID and status of all cloud deployments:

from octaipipe.deployment.edge.octaideploy import octaideploy

octaideploy(compose_action='status', cloud=True)

Deployment Template#

Deployment templates define the kuberenetes manifests for each of the objects which make up the deployment.

Out-the-box deployments have pre-configured templates, you can see these at src/octaipipe/configs/cloud_deployment.

It is possible to create bespoke deployment templates. It is then necessary to refer to them in the Deployment Config with deployment spec called template which is the path to the bespoke template.

 1apiVersion: batch/v1
 2kind: CronJob
 3metadata:
 4  name: $(deployment_name)-cron
 5spec:
 6  schedule: $(schedule)
 7  concurrencyPolicy: $(concurrencyPolicy)
 8  successfulJobsHistoryLimit: 1
 9  failedJobsHistoryLimit: 2
10  jobTemplate:
11    spec:
12      backoffLimit: 2
13        template:
14          metadata:
15            labels:
16              name: $(deployment_name)
17          spec:
18            containers:
19            - name: $(container_name)
20              image: $(docker_image)
21              env: $(env_variables)
22              command: ["/bin/sh","-c"]
23              args: $(commands)
24            restartPolicy: Never
25---
26apiVersion: v1
27kind: Secret
28metadata:
29  name: $(deployment_name)-secrets
30type: Opaque
31stringData:
32  token: $(token)

If you are familiar with kubernetes this should be familiar, if not look here for more info on resource configuration in kubernetes: Managing Resources | Kubernetes

In the above template, two kubernetes objects are defined, a CronJob and a Secret. Each object should be separated by a triple-dash yaml separator (---).

Deployment templates have placeholder values, like so $(placeholder). These are used to configure deployment specific settings and are replaced at runtime.

$(placeholders)#

Placeholders defined by a dollar sign and parentheses are replaced at runtime by values from either:

  1. Deployment Specs

  • Defined in the Deployment Config:

An example of a cloud deployment config

  1. Environment Variables

  1. Default Values

  • Defined in the cloud deployment class:

Image of code showing default map

Order of precedence is as indicated in the list above i.e., if there is no matching deployment spec, environment variables are checked, if there is no matching environment variable, the default value is used.

Placeholders are case-sensitive.

Note

If there are any unmatched placeholders in a template, the deployment will error. Check traceback for the missing placeholder values.

Example of Template Configuration#

Taking the following template and config as yaml:

Template

 1apiVersion: v1
 2kind: Secret
 3metadata: $(secret_metadata)
 4type: Opaque
 5data: $(secret_data)
 6---
 7apiVersion: batch/v1
 8kind: Job
 9metadata: $(image_metadata)
10spec:
11  template:
12    spec:
13      containers: $(containers)

Config

 1name: example-config
 2type: generic
 3kubernetes_namespace: demo
 4
 5deployment_specs:
 6  labels:
 7    app: $(deployment_name)
 8    environment: production
 9
10  secret_metadata:
11    name: $(deployment_name)-secret
12    labels: $(labels)
13  secret_data:
14    username: YWRtaW4=
15    password: cGFzc3dvcmQ=
16
17  image_metadata:
18    name: $(deployment_name)-image
19    labels: $(labels)
20
21  containers:
22    - name: $(deployment_name)-app
23    image: app-image
24    command: $(command)
25    - name: $(deployment_name)-db
26    image: db-image
27
28  command: echo "hello world"

Output

 1apiVersion: v1
 2kind: Secret
 3metadata:
 4  name: example-config-secret
 5  labels:
 6    app: example-config
 7    environment: production
 8type: Opaque
 9data:
10  password: cGFzc3dvcmQ=
11  username: YWRtaW4=
12---
13apiVersion: batch/v1
14kind: Job
15metadata:
16  name: example-config-image
17  labels:
18    app: example-config
19    environment: production
20spec:
21  template:
22    spec:
23      containers:
24        - name: example-config-app
25        image: app-image
26        command: echo "hello world"
27        - name: example-config-db
28        image: db-image

Out-the-box deployment types#

Deployment type is selected by the type field in the deployment config. The current list of out-the-box deployment types is listed below:

  1. Generic Cloud Deployment

  2. Inference API Deployment

  3. InfluxDB Deployment

  4. Granfana Deployment

  5. Octaistep/Pipeline step Deployment

Creating Bespoke Deployments#

To create a bespoke deployment, template of kubernetes manifests which will create the objects required by the Deployment. You could either do this by editing an existing, default template or by creating an entirely new one.

In the template, replace any values you would like to be defined at run time with $(placeholders) see the placeholders section for more on this.

Your template can then be referred to in a Deployment Config in the template field.

Note

When creating a bespoke deployment, it is recommended to use the generic CloudDeployment deployment type. i.e. type: generic

Example run-through#

Below is a generic exmaple of a Dpeloyment configuration template.

Template ( /path/to/my-app-template.yml )

 1apiVersion: v1
 2kind: ConfigMap
 3metadata:
 4  name: $(deployment_name)-config
 5data:
 6  some-config-key: $(config_value)
 7---
 8apiVersion: networking.k8s.io/v1
 9kind: Ingress
10metadata:
11  name: $(deployment_name)-ingress
12  annotations:
13    nginx.ingress.kubernetes.io/rewrite-target: /
14spec:
15  rules:
16  - host: $(host)
17    http:
18      paths: $(http_paths)
19    service:
20      name: $(deployment_name)
21      port:
22        name: http
23        number: 80
24---
25apiVersion: v1
26kind: Service
27metadata:
28  name: $(deployment_name)
29  labels:
30    app: $(deployment_name)
31spec:
32  selector:
33    app: $(deployment_name)
34  ports:
35  - name: http
36    port: 80
37    targetPort: 8080
38---
39apiVersion: apps/v1
40kind: Deployment
41metadata:
42  name: $(deployment_name)
43spec:
44  replicas: 2
45  selector:
46    matchLabels:
47      app: $(deployment_name)
48  template:
49    metadata:
50      labels:
51        app: $(deployment_name)
52        env: $(environment)
53    spec:
54      containers:
55      - name: $(deployment_name)
56        image: $(deployment_name):latest
57        ports:
58        - containerPort: 8080
59        envFrom:
60        - configMapRef:
61          name: $(deployment_name)-config
62        env: $(env_variables)
63      volumeMounts:
64        - name: config-volume
65          mountPath: /etc/config
66      volumes:
67        - name: config-volume
68          configMap:
69            name: $(deployment_name)-config

To use this template, we need to create a config file.

Deployment Config ( my-app-config.yml )

 1name: my-app
 2type: generic
 3kubernetes_namespace: example-deployment
 4template: /path/to/my-app-template.yml # <path to template>
 5
 6deployment_specs:
 7  environment: "production"
 8
 9  http_paths:
10    - path: /$(app_name)
11      pathType: Prefix
12      pathRewrite: /
13      pathRewriteRedirect: Permanent
14
15  env_variables:
16    - name: MY_APP_ENV
17      value: $(environment)
18    - name: MOUNT_PATH
19      value: /etc/config

Note

You can also interact with the deployment via kubectl for more granular management capability.

Terminating a Cloud Deployment#

In order to terminate a deployment by its deployment id:

from octaipipe.deployment.edge.octaideploy import octaideploy
octaideploy(deployment_id='******', compose_action='down', cloud=True)

Note that you can find the deployment id by visiting the Deployment page on the front end: https://app.DEPLOYMENT.octaipipe.ai/app/deployments (replace DEPLOYMENT by the name of your OctaiPipe deployment in the URL)