Model Inference Step#
To run locally: model_inference
An ML lifecycle can be broken up into two main, distinct parts. The first is the training phase, in which the model is created or “trained” by running a specified subset of the dataset through the model. ML inference is the second phase, in which the model is deployed to make predictions on live data, giving actionable outputs.
The inference step follows this workflow which corresponds to sections in the inference config file:
Load an existing model:
model_specs
Load live test data:
input_data_specs
Perform model inference at intervals:
run_specs
Save live predictions:
output_data_specs
The following is an example of a config file together with descriptions of its parts.
Step config example#
1name: model_inference
2
3input_data_specs:
4 default:
5 - datastore_type: influxdb
6 settings:
7 query_type: dataframe
8 query_template_path: ./configs/data/inference_query.txt
9 query_config:
10 start: 5d
11 bucket: live-metrics
12 measurement: def-metrics
13 tags:
14 MY_TAG: value_0
15
16output_data_specs:
17 default:
18 - datastore_type: influxdb
19 settings:
20 bucket: "test-write"
21 measurement: "model-outputs"
22 - datastore_type: local
23 settings:
24 file_path: "./logs/results"
25
26model_specs:
27 name: Machine_RUL
28 type: ridge_reg
29 version: "2.0"
30
31run_specs:
32 prediction_period: 10s
33 save_results: True
34 onnx_pred: false
Input and Output Data Specs#
input_data_specs
and output_data_specs
follow a standard format for all the pipeline
steps; see Octaipipe Steps.
Model Specs#
25 model_specs:
26 name: Machine_RUL
27 type: ridge_reg
28 version: '2.0'
Specifications of the model to be loaded for inference. See Model Training Step.
Run Specs#
30 run_specs:
31 prediction_period: 10s
32 onnx_pred: false
prediction_period
: sleeping time interval between predictions.onnx_pred
: boolean of whether to use the onnx file of the trained model for inference. If false, the joblib model file is used.