OctaiPipe User Guide

Note

This project is under active development

Contents:

• OctaiPipe Getting Started Guide
  • Quick Links
    • Key Features
  • Not sure where to start?
  • Contents Overview
    • Quickstart
    • Installing OctaiPipe
    • Federated Learning
    • Data Loading and Writing Utilities
    • OctaiPipe Steps
    • OctaiPipe Models
    • Edge Device Integration
    • Running OctaiPipe Pipelines in Kubeflow
    • OctaiKube
    • Develop with OctaiPipe
    • Installing OctaiPipe from the Azure Marketplace
    • Cloud Deployment
    • Tutorials
    • Frequently Asked Questions
    • Release Notes
• Installing OctaiPipe
• Quickstart
• Federated Learning
  • Prerequisites
  • Steps and Results
  • Device Registration
  • FL Server Deployment
  • Define Pipeline Steps for FL
    • infrastructure
    • model_specs
    • run_specs
    • strategy
    • Custom PyTorch model
  • Deploy and perform FL
  • Summary Report and Detailed Experiment Info
  • Machine Learning Model
  • Programmer’s Guide
    • Custom PyTorch Model
    • Custom SGD Model
    • Federated XGBoost Implementation with OctaiPipe
    • FL Train Step
    • Strategy
• Data Loading and Writing Utilities
  • Interaction with the database
  • InfluxDB Configuration
  • SQL Configuration
  • MQTT Configuration
    • MQTT Data Loading
    • MQTT Data Writing
  • Interaction with API
  • API Configuration
  • Local data Configuration
  • Data Converters
    • Influx Flat Converter
• OctaiPipe Steps
  • Pipeline Steps
    • Preprocessing Step
    • Feature Engineering Step
    • Model Training Step
    • Model Evaluation Step
    • Model Inference Step
    • WASM Model Inference Step
    • Data (Concept) Drift Monitoring Step
    • Feature Selection Step
    • AutoML in OctaiPipe
    • Custom Pipeline Steps
  • Data Input and Output Specs
    • Input Data Specs
    • Output Data Specs
• OctaiPipe Models
  • Supported Models
    • Linear Regression
    • Ridge Regression
    • Elastic Net Regressor
    • Random Forest Regression
    • Tweedie Regressor
    • Quantile Regressor
    • Support Vector Regressor
    • Stochastic Gradient Descent Regressor
    • Bayesian Ridge Regressor
    • ARD Regressor
    • Gradient Boosting Regressor
    • Extreme Gradient Boosting Regressor
    • Skopt for LightGBM
    • LightGBM Regressor
    • Logistic Regression
    • k-Nearest Neighbors Classification
    • Support Vector Classification
    • Random Forest Classification
    • Extreme Gradient Boosting Classifier
    • LightGBM Classification
    • KMeans Clustering
    • Exponential Smoothing
    • Principal Component Analysis
  • Custom Models
  • Model Management
  • Model Versioning
• Model and Preprocessor Object Management
• Edge Device Integration
  • How-to Guide
    • Device Configuration
    • Register a Device
• Running OctaiPipe Pipelines in Kubeflow
  • Preparing Configuration files
    • Schema Definition
    • Example Config
  • Running a Pipeline
• Develop with OctaiPipe
  • Set up your environment to develop with OctaiPipe
  • Write/Load data to and from Influxdb using OctaiPipe modules
  • Build a machine learning model that predicts the RUL of assets in an IoT environment
  • Register the model to an Azure Storage container
  • Evaluate the performance of the model and save the metrics for model monitoring
  • Run inference on unseen data points
• Installing OctaiPipe from the Azure Marketplace
• User Interface
  • Deployments interface
  • Develop interface
  • Devices interface
  • Experiments interface
  • Models interface
• OctaiKube
  • Ad-Hoc Code running in KubeFlow
    • Modifying function to run in Kubeflow
  • OctaiKube Interface Functions
  • OctaiKube Adhoc Functions
  • OctaiKube Utils Functions
  • OctaiKube AutoML Functions
• Cloud Deployment
  • Cloud Deployment
    • Deploying to Cloud
  • Cloud Deployment Types
    • Step Deployment
    • Generic Cloud Deployment
    • Inference API Deployment
    • InfluxDB Deployment
    • Grafana Deployment
• Tutorials
  • End-to-end cloud training to edge deployment example - Remaining Useful Life
    • Register devices
    • Load C-MAPSS data and write to Influx on device for training
    • Preprocess data and train a model
    • Deploy model out to edge devices
    • Write live data to all devices and view inference
    • Remove deployment
    • Remove Influx instances
  • End-to-end cloud training to edge deployment example - classification
    • Register devices
    • Load C-MAPSS data and write to Influx on device for training
    • Preprocess data and train a model
    • Deploy model out to edge devices
    • Write live data to all devices and view inference
    • Remove deployment
    • Remove Influx instances
  • Example Custom Pipeline Steps
    • Parent class PipelineStep
    • Example Custom Steps
    • Custom step config
    • Custom data loading
    • Custom data preprocessing
    • Custom model evaluation
    • Custom periodic step
  • Remaining Useful Life Estimation
    • Objective:
    • Environment Setup
    • Running OctaiPipe Steps
  • Tutorial for Using MLOps global and local policies
    • Policy How-to
    • Running a global policy
    • Running a local policy
  • Federated Learning - Anomaly Detection
    • Register devices
    • Load Anomaly Detection data and split among devices
    • Set up FL config
    • Train base PyTorch FL model
    • Define custom model architecture
    • Train custom FL model
  • Tutorial - Running FL XGBoost with OctaiPipe
    • Step 1 - defining devices
    • Step 2 - Download and send data to devices
    • Step 3 - Running FL
  • Tutorial - Running ML Inference with OctaiOxide (OctaiPipe in WASM)
    • Benefits of using our WASM based image
    • Setup environment and devices
    • Load trained models to portal
    • CMAPSS Dataset
    • CMAPSS Model Deploy to your Device using Wasm
    • NASA Bearing Dataset
    • NASA Bearing neural network model deployment to device
• Infrastructure Cost Management
  • Azure
    • How to clean up Kubernetes resources
    • How to remove unattached disks
• Frequently Asked Questions
  • Who is OctaiPipe built for ?
  • How can I optimize cloud cost of running OctaiPipe?
• Release Notes
  • 2.2.10
  • 2.2.9
  • 2.2.8
  • 2.2.7
  • 2.2.6
  • 2.2.5
  • 2.2.2
    • Bug fixes
  • 2.2.1
    • Marketplace documentation
  • 2.2.0
    • Federated XGBoost
    • Initial support for Web Assembly
    • Improvements to Model monitoring and continuous learning
    • Support for self-service installation of OctaiPipe from the Azure marketplace
    • Continuous improvements to platform reliability, security and costs
  • 2.1.7
  • 2.1.6
  • 2.1.5
  • 2.1.4
  • 2.1.3
  • 2.1.2
  • 2.1.1
  • 2.1.0
    • Introduction of the OctaiPipe Edge Client
    • API endpoint security
    • Introduction of OctaiPipe core and lite
    • MQTT data loading and writing
    • Bug fixes
  • 2.0.3
  • 2.0.0
    • Web UI
    • Independent Security review
    • Azure and AWS Availability
    • Updated Examples and demo videos
  • 1.6.0
    • MLOps
    • Breaking Changes
  • 1.5.1
    • User Interface
    • Env Variables Definition
  • 1.4.0
    • Cloud Deployment
  • 1.3.0
    • Minor Changes and Bug Fixes
  • 1.2.10
    • Minor Changes and Bug Fixes
  • 1.2.2
    • Minor Changes and Bug Fixes
  • 1.2.1
    • Minor Changes and Bug Fixes
  • 1.2.0
    • Federated Learning
    • Minor Changes and Bug Fixes
  • 1.1.0
    • AutoML
    • New features and improvements
    • Minor Changes and Bug Fixes
  • 1.0.4
    • New features and improvements
  • 1.0.0
    • Core Pipeline Steps:
    • Edge
    • Cloud
    • OctaiClient Metadata