Deploying to Cloud#

Introduction#

OctaiPipe’s Cloud Deployment feature provides a powerful and efficient solution for managing deployment configurations through a simple command-line interface. It is built on top of Kubernetes, and allows users to abstract constellations of Kubernetes objects as discrete deployment units. The configuration, deployment, and management of these units is handled seamlessly, massively streamlining the deployment of infrastructure features.

Although any infrastrcuture feature can be deployed using this feature, the major use-case for OctaiPipe users is deploying octaisteps and models.

Users have the flexibility to use out-of-the-box deployment configuration templates for common use-cases, or creating custom templates to suit specific needs. This user guide provides information on using OctaiPipe’s Cloud Deployment feature, including instructions on both out-of-the-box and bespoke deployment types.

Requirements#

  • Connection to OpenVPN

  • Kubectl

Launching a Cloud Deployment#

The only requirement to launch a deployment is a config file detailing the type of deployment and its settings. See here for details about Deployment Configs.

Deployment Config#

Deployment Configs are yaml files which define everything necessary to create a deployment.

Configs are passed to the octaideploy to create the deployment(See FUNCTION).

A generic deployment config takes the following fields:

  • name string

    • Required

    • The name of the deployment.

    • Can be used to reference the deployment. Must be unique.

  • type string

    • Required

    • The type of cloud deployment being run

    • Corresponds to the type value in the cloud deployment class

  • template string

    • Path to deployment template (see above)

    • Defaults to the default template of the deployment in question.

    • Out-the-box ones can be found here src/octaipipe/configs/cloud_deployment

  • kubernetes_namespace string

    • The kubernetes namespace in which to deploy

    • If not passed, environment variables are checked for kubernetes_namespace. Otherwise defaults to default (effectively no namespace)

    • Note: we recommend setting this to ‘colab’; this makes sure the deployment has the necessary privileges to run properly. The ‘colab’ namespace is created as part of the OctaiPipe infrastructure installed during onboarding.

  • deployment_specs dict

    • Required

    • Used to replace placeholders in the deployment template. (see above)

    • key/values defined here take precedence over environment variables and default values when replacing placeholders.

All deployments take the above config fields. Out-the-box deployments can take more depending on specific configuration requirements. Details of this can be found in the section dedicted to out-the-box deployment types.

Example#

 1name: octaipipe-weight-forecast
 2type: stepDeployment
 3kubernetes_namespace: example-step
 4template: /path/to/bespoke/template.yml
 5
 6deployment_specs:
 7  schedule: "0 0 * * *"
 8  docker_image: octaipipe:latest
 9  container_name: octaipipe
10  commands:
11    - "octaistep --local-step-path ./etl/weight_pipeline_step.py  --config-path ./etl/weight_monthly.yml"
12    - "octaistep --local-step-path ./weight_forecasting/forecasting_step.py  --config-path ./weight_forecasting/configs/forecasting.yml"

The above will create deploy an octaistep process from an octaipipe image in a namespace called example-step to run daily at midnight (schedule). The process involves the two commands listed under commands.

Command: octaideploy()#

Deployment Template#

Deployment templates define the kuberenetes manifests for each of the objects which make up the deployment.

Out-the-box deployments have pre-configured templates, you can see these at src/octaipipe/configs/cloud_deployment.

It is possible to create bespoke deployment templates. It is then necessary to refer to them in the Deployment Config with deployment spec called template which is the path to the bespoke template.

 1apiVersion: batch/v1
 2kind: CronJob
 3metadata:
 4  name: $(deployment_name)-cron
 5spec:
 6  schedule: $(schedule)
 7  concurrencyPolicy: $(concurrencyPolicy)
 8  successfulJobsHistoryLimit: 1
 9  failedJobsHistoryLimit: 2
10  jobTemplate:
11    spec:
12      backoffLimit: 2
13        template:
14          metadata:
15            labels:
16              name: $(deployment_name)
17          spec:
18            containers:
19            - name: $(container_name)
20              image: $(docker_image)
21              env: $(env_variables)
22              command: ["/bin/sh","-c"]
23              args: $(commands)
24            restartPolicy: Never
25---
26apiVersion: v1
27kind: Secret
28metadata:
29  name: $(deployment_name)-secrets
30type: Opaque
31stringData:
32  token: $(token)

If you are familiar with kubernetes this should be familiar, if not look here for more info on resource configuration in kubernetes: Managing Resources | Kubernetes

In the above template, two kubernetes objects are defined, a CronJob and a Secret. Each object should be separated by a triple-dash yaml separator (---).

Deployment templates have placeholder values, like so $(placeholder). These are used to configure deployment specific settings and are replaced at runtime.

$(placeholders)#

Placeholders defined by a dollar sign and parentheses are replaced at runtime by values from either:

  1. Deployment Specs

  • Defined in the Deployment Config:

An example of a cloud deployment config

  1. Environment Variables

  1. Default Values

  • Defined in the cloud deployment class:

Image of code showing default map

Order of precedence is as indicated in the list above i.e., if there is no matching deployment spec, environment variables are checked, if there is no matching environment variable, the default value is used.

Placeholders are case-sensitive.

Note

If there are any unmatched placeholders in a template, the deployment will error. Check traceback for the missing placeholder values.

Example of Template Configuration#

Taking the following template and config as yaml:

Template

 1apiVersion: v1
 2kind: Secret
 3metadata: $(secret_metadata)
 4type: Opaque
 5data: $(secret_data)
 6---
 7apiVersion: batch/v1
 8kind: Job
 9metadata: $(image_metadata)
10spec:
11  template:
12    spec:
13      containers: $(containers)

Config

 1name: example-config
 2type: generic
 3kubernetes_namespace: demo
 4
 5deployment_specs:
 6  labels:
 7    app: $(deployment_name)
 8    environment: production
 9
10  secret_metadata:
11    name: $(deployment_name)-secret
12    labels: $(labels)
13  secret_data:
14    username: YWRtaW4=
15    password: cGFzc3dvcmQ=
16
17  image_metadata:
18    name: $(deployment_name)-image
19    labels: $(labels)
20
21  containers:
22    - name: $(deployment_name)-app
23    image: app-image
24    command: $(command)
25    - name: $(deployment_name)-db
26    image: db-image
27
28  command: echo "hello world"

Output

 1apiVersion: v1
 2kind: Secret
 3metadata:
 4  name: example-config-secret
 5  labels:
 6    app: example-config
 7    environment: production
 8type: Opaque
 9data:
10  password: cGFzc3dvcmQ=
11  username: YWRtaW4=
12---
13apiVersion: batch/v1
14kind: Job
15metadata:
16  name: example-config-image
17  labels:
18    app: example-config
19    environment: production
20spec:
21  template:
22    spec:
23      containers:
24        - name: example-config-app
25        image: app-image
26        command: echo "hello world"
27        - name: example-config-db
28        image: db-image

Out-the-box deployment types#

Deployment type is selected by the type field in the deployment config. The current list of out-the-box deployment types is listed below:

  1. Generic Cloud Deployment

  2. Inference API Deployment

  3. InfluxDB Deployment

  4. Granfana Deployment

  5. Octaistep/Pipeline step Deployment

Creating Bespoke Deployments#

To create a bespoke deployment, template of kubernetes manifests which will create the objects required by the Deployment. You could either do this by editing an existing, default template or by creating an entirely new one.

In the template, replace any values you would like to be defined at run time with $(placeholders) see the placeholders section for more on this.

Your template can then be referred to in a Deployment Config in the template field.

Note

When creating a bespoke deployment, it is recommended to use the generic CloudDeployment deployment type. i.e. type: generic

Example run-through#

Below is a generic exmaple of a Dpeloyment configuration template.

Template ( /path/to/my-app-template.yml )

 1apiVersion: v1
 2kind: ConfigMap
 3metadata:
 4  name: $(deployment_name)-config
 5data:
 6  some-config-key: $(config_value)
 7---
 8apiVersion: networking.k8s.io/v1
 9kind: Ingress
10metadata:
11  name: $(deployment_name)-ingress
12  annotations:
13    nginx.ingress.kubernetes.io/rewrite-target: /
14spec:
15  rules:
16  - host: $(host)
17    http:
18      paths: $(http_paths)
19    service:
20      name: $(deployment_name)
21      port:
22        name: http
23        number: 80
24---
25apiVersion: v1
26kind: Service
27metadata:
28  name: $(deployment_name)
29  labels:
30    app: $(deployment_name)
31spec:
32  selector:
33    app: $(deployment_name)
34  ports:
35  - name: http
36    port: 80
37    targetPort: 8080
38---
39apiVersion: apps/v1
40kind: Deployment
41metadata:
42  name: $(deployment_name)
43spec:
44  replicas: 2
45  selector:
46    matchLabels:
47      app: $(deployment_name)
48  template:
49    metadata:
50      labels:
51        app: $(deployment_name)
52        env: $(environment)
53    spec:
54      containers:
55      - name: $(deployment_name)
56        image: $(deployment_name):latest
57        ports:
58        - containerPort: 8080
59        envFrom:
60        - configMapRef:
61          name: $(deployment_name)-config
62        env: $(env_variables)
63      volumeMounts:
64        - name: config-volume
65          mountPath: /etc/config
66      volumes:
67        - name: config-volume
68          configMap:
69            name: $(deployment_name)-config

To use this template, we need to create a config file.

Deployment Config ( my-app-config.yml )

 1name: my-app
 2type: generic
 3kubernetes_namespace: example-deployment
 4template: /path/to/my-app-template.yml # <path to template>
 5
 6deployment_specs:
 7  environment: "production"
 8
 9  http_paths:
10    - path: /$(app_name)
11      pathType: Prefix
12      pathRewrite: /
13      pathRewriteRedirect: Permanent
14
15  env_variables:
16    - name: MY_APP_ENV
17      value: $(environment)
18    - name: MOUNT_PATH
19      value: /etc/config

Note

You can also interact with the deployment via kubectl for more granular management capability.