Tutorial for Using MLOps global and local policies

Policy How-to

The policy is the key for Octaipipe towards MLOps. Anyone can use this tool to define and perform their MLOps tasks with Octaipipe. One may also take the policy as a decision-making process. Octaipipe can simply perform tasks such as monitoring, retraining, and so on via the policy definition. Octaipipe also allows users to define multiple policies within a single YAML file, giving maximum flexibility. In this tutorial, we would like to introduce the concept of policy and how to use this with two examples.

Generally speaking, there are three types of policy. The categorisation of these policies relies on where the actions act. These are - Global Policy: Global policy refers to policies that may affect many devices or cloud components. Global retraining is a typical Global policy. It monitors the drift on edge devices from the cloud deployed using Octaipipe. Retraining on the edge devices starts when the trigger rule is valid. Since this policy involves members such as edge devices and cloud components, we classify these policy types as a Global Policy. - Cloud Policy: As the name suggests, no edge device involves in this process. For example, one may create a monitoring process to watch the health condition of the Kubernetes cluster and send a notification when certain situations happen. Since these can execute within a K8S cluster, we can call this a Cloud Policy. - Local Policy: Unlike Global Policy, Local policy is a decision-making task that acts on the local or edge device. Usually, it is actioned on these devices. Once the trigger is valid, predefined actions will be executed sequentially.

A policy comes with three types of Octaipipe components.These are: - observations: Conceptually, observation refers to the type of source or said metric. You may also consider observations as one or more metrics that can be gathered during model production and used in automation decision-making. For example, this could be the dataset’s drift value or the model’s performance. Once configured, Octaipipe preserves this information for you. You may config multiple observations within a single policy. At this moment, Octaipipe has provided the following observations. - global_drift_percentage_observation - p_value_edge_observation - trigger: The trigger defines the criteria of the execution actions. To compose this trigger expression, one may use any Python assignment, arithmetic, logic or bitwise operators and observations. A proper trigger expression should result in a boolean output. Octaipipe accepts two forms of expression. - {expression}: One may use Python mathematical syntax, operators or observations to compose a single-line expression. When the result is true, it will trigger the policy to execute the actions. - {expresion 1} {operator} {expression 2}: One may use the comparison or assignment operators to perform actions {expression 1} and {expression 2} for this form of expression. Due to the limitation, only these outcomes are acceptable by the Octaipipe.

expression 1	expression 2	operator
boolean	boolean	==, >, >=, <, <=, !=,&, |, ^
numerical value	boolean	==, >, >=, <, <=, !=,&,
boolean	numerical value	==, >, >=, <, <=, !=,&,
numerical value	numerical	==, >, >=, <, <=, !=,&,

• actions: Actions refer to what tasks would be executed when a trigger is valid. Multiple actions can be defined in this section. The order of execution relies on the sequence of the definition in this section. One may need to give some essential supplement arguments for action if required. Octaipipe has provided the following activities for you to use.

  • global_retrain: Action to perform global retraining.

  • local_retraining: Action to perform local retraining.

  • send_notification: Action to send notification.

One may use these to achieve any MLops tasks you would like to have.

Running a global policy

In this section, we are walking through the complete process of creating, deploying and teardown a global policy. We will use the native Octaipipe global policy, global retraining, as an example. The global retraining can be viewed like the one shown below:

• Observations: proportion of devices that have experienced drift since last retraining

• Trigger: proportion of devices that sufferred drift

• Actions: global retraining / notification

Create a global policy

A policy definition YML file is essential for a global policy. One may use This file as the ference. There following fields are required to config a global policy #### General Concept To deploy a local policy, one has to provide these information for Octaipipe to deploy a global policy. - policies: One has to define observation, trigger and action within this section. It is also possible to deploy multiple policies. - env: This refers to the environment variables that the container or Kubernetes Pod uses to run the policy. - pipelines: One may define these steps here if it is required, e.g. inference step. - run_specs: like other pipeline steps, one has to assign run_specs. Octaipipe takes settings in this section to execute this policy. - name: This refers to the name of this deployment. It should always be “deployment_config” for edge deployment. - image_name: This refers to the container image that one would like to use in this policy - datasources: This refers to the credential used for influxDB, e.g. INFLUX_TOKEN or INFLUX_ORG. - device_ids: This section defines the device that one would like to deploy this global policy.

One would change these depending on your configurations.

Other supplement files

One has to provide other files if those are essential to deploy and run the policy. For example, a model inference definition YAML is required to run the global retrain. It is also necessary to provide a YAML file for global retraining. These files have to locate in ./configs.

deploy a global policy

Once configurations are ready, you only need to use new_deployment with the configuration file to deploy and run the policy. You may set cloud = True if this is a cloud policy.

import os
from octaipipe.deployment import deploy_to_edge

os.environ['OCTAIPIPE_DEBUG']='INFO'
deployment_global = deploy_to_edge('./case19.yml', cloud = False)
print(f'Deployment Id is : {deployment_global}')

Teardown a global policy

If you wnat to terminate a deployed global policy, all you need to do is to use down_deployment. The deployment id is essential to teardown a deployment. Octaipipe will do the reset of the tasks for you.

import os
from octaipipe.deployment import down_deployment

down_deployment(deployment_global)

Running a local policy

In this section, we are walking through the complete process of creating, deploying and teardown a local policy.

Like global policy, a policy definition YML file is essential for any local policy. Other files, such as inference configuration, may depend on the action you want to deploy. To explain, we would like to use the native Octaipipe local policy - local retraining through this tutorial. The local retraining can be viewed like the one shown below:

• Observations: local p value for data drift

• Trigger: drift p-value < certain numerical value

• Actions: local retraining / notification

Create a local policy

A policy definition YML file is essential for a local policy. A policy definition YAML file is necessary here. One may use This file as the reference. Some essential fields are required when config this file.

Required field

To deploy a local policy, similar to the global policy, one has to provide these information for Octaipipe to deploy a policy. - name: This refers to the name of this deployment. It should always be “deployment_config” for edge deployment. - policies: One has to define observation, trigger and action within this section. It is also possible to deploy multiple policies. - env: This refers to the environment variables that the container or Kubernetes Pod uses to run the policy. - pipelines: One may define these steps here if it is required, e.g. inference step. - run_specs: like other pipeline steps, one has to assign run_specs. Octaipipe takes settings in this section to execute this policy. - image_name: This refers to the container image that one would like to use in this policy - datasources: This refers to the credential used for influxDB, e.g. INFLUX_TOKEN or INFLUX_ORG. - device_ids: This section defines the device that one would like to deploy this local policy.

One would change these depending on your configurations.

Other supplement files

One has to provide other files required to properly deploy and run the policy. For example, a model inference definition YAML is require to run the local retrain. These files have to locate in ./configs.

deploy a local policy

Once configuration are ready, all you need to do is use deploy_to_edge with configuration file to deploy and run the policy

import os
from octaipipe.deployment import deploy_to_edge

os.environ['OCTAIPIPE_DEBUG']='INFO'
deployment_local = deploy_to_edge('./case20.yml', cloud = False)
print(f'Deployment Id is : {deployment_local}')

Teardown a local policy

Similar to the global policy, with the deployment-id, one may use down_deployment to teardown a local policy.

import os
from octaipipe.deployment import down_deployment
down_deployment(deployment_local)