Tutorial - Deploying Image based Inference to an Android Handset#

In this tutorial we will take you through the process of deploying an existing model to an Android handset for inference. Note that using OctaiPipe on Android requires integration of the OctaiPipe Android library into your own Android app, the integration process is detailed in the Android Model Inference Step section of the OctaiPipe documentation.

The yml configurations for both inference and deployment follow the same format as a regular Octaipipe configuration. Currently, we support Image based models on Android, with support for other data types coming soon.

Device registration and initialisation#

  • Make sure that your Portal url is set as the “OCTAICLIENT_ENDPOINT” environment variable.

  • Ensure that your Android Device has been registered in your Portal. Check the registration documentation for instructions.

  • Ensure that your Android App has been initialised with your device credentials. Refer to the Android Model Inference Step documentation for details of how to initialise an Android device using the initialiseDevice function.

Glance at registered devices#

By finding all devices we can take a look at the devices that have been registered in your Portal. Ensure that your device is registered and has device status as ‘Available’

[ ]:

import pandas as pd

from octaipipe import devices, models
import logging
import yaml

logging.basicConfig(level=logging.INFO, format='%(message)s')

all_devices = devices.find_all_devices()
all_devices = pd.DataFrame(all_devices)
all_devices.head(10)

Explore model record#

Since this tutorial focuses on inference deployment, we will be using an existing trained model. We assume here that an Image Inference model has already been trained called ‘model_octaipipe’. This argument needs to be replaced with your own model.

[ ]:

model = models.find_models_by_name('model_octaipipe')
pd.DataFrame(model)

Inference config#

[3]:

# Display inference config
# Note that you need to specify your own trained model where it says 'model_octaipipe'
with open("configs/android_inference_config.yml", 'r') as file:
    inference_config = yaml.safe_load(file)
print(yaml.dump(inference_config, sort_keys=False))

name: image_inference
model_specs:
  name: model_octaipipe
  type: keras
  version: '1'
input_data_specs:
  datastore_type: image
  settings:
    file_path: user_input
run_specs:
  mode: interactive
  only_last_batch: false

Deployment config#

[4]:

# Display deployment config
# Note that you need to specify your own device_id / device_ids  where it says 'your-device-id'
with open("configs/android_deployment.yml", 'r') as file:
    inference_config = yaml.safe_load(file)
print(yaml.dump(inference_config, sort_keys=False))

name: deployment_config
device_ids:
- your-device-id
env: {}
pipelines:
- image_inference:
    config_path: configs/android_inference_config.yml

Android Deployment#

Finally, we can call OctaidroidDeployment using our deployment configuration. This will create a deployment containing our model and configuration which will be pulled automatically from the Android Device next time it is online and the app is running. Logs will be displayed below confirming if the deployment was successful. Once deployed, the new model will be active when you next run inference from your device.

[ ]:

from octaipipe.deployment.android.octaidroid_deployment import OctaidroidDeployment

octaidroid_deployment = OctaidroidDeployment()
deployment_id = octaidroid_deployment.deploy(
    user_config_path='configs/android_deployment.yml',
    name='android_deployment',
    description='Image inference deployment'
)

We can stop a deployment as follows, this will deactivate the model on the device so that inference will no longer be runnable.

[ ]:

from octaipipe.deployment.interface import down_deployment
down_deployment(deployment_id)