OctaiPipe Modules

The OctaiPipe Python library consists of 3 modules with increasing functionality. Each preceding module is a dependency for the next. The modules are:

• OctaiPipe Core

• OctaiPipe Lite

• OctaiPipe Full

Remember that when using OctaiPipe as a Data Scientist via an OctaiPipe Jupyter Notebook, you will always be working from an OctaiPipe Full Image. The question of which image module to use arises when deploying to the Edge, depending on your edge device resource constraints.

OctaiPipe Core

OctaiPipe Core is the foundational module containing essential functionality with a lean footprint.

Key features:

• Basic client interactions

• Device and Cloud authentication

• Data reading and writing

• Optional data loaders (SQL, InfluxDB, MQTT)

• Pipeline and custom step functionality.

• Federated EDA (FEDA)

Core Pipeline steps:

• Federated EDA

• Custom Step

Core Images

Image Name	Size
octaipipe_core-mqtt	439mb
octaipipe_core-mqtt-root	439mb
octaipipe_core-influxdb	480mb
octaipipe_core-influxdb-root	480mb
octaipipe_core-sql	487mb
octaipipe_core-sql-root	487mb
octaipipe_core-all_data_loaders	499mb
octaipipe_core-all_data_loaders-root	499mb

• The suffix after - indicates the data loaders available to the image.

• The suffix root indicates the image will be granted root permissions.

• Check OctaiPipe Images for more information on image notation.

Importing OctaiPipe Core Modules

OctaiPipe Full is a superset of Core and modules contained in core must be imported with the octaipipe_core prefix.

Examples of importing core classes from a Jupyter Notebook:

from octaipipe_core.data.influxdb.dataloader import InfluxDataLoader
from octaipipe_core.data.influxdb.datawriter import InfluxDataWriter
from octaipipe_core.data.mqtt.mqtt_client import MQTTClient
from octaipipe_core.pipeline.feda_step import FedaStep
from octaipipe_core.client.evaluation_client import EvaluationClient

OctaiPipe Lite

OctaiPipe Lite builds upon Core, adding support for non neural network based federated learning and model inference.

Key features:

• Federated XgBoost Training and Inference

• Scikit-learn SGD Classifier and Regressor FL Training and Inference

• K-Federated Learning (k-FED)

• Scikit-learn Preprocessing

Lite Images

Lite images are heavier in size with the addition of ML and FL libraries.

Image Name	Size
octaipipe_lite-mqtt	1.73gb
octaipipe_lite-mqtt-root	1.73gb
octaipipe_lite-influxdb	1.75gb
octaipipe_lite-influxdb-root	1.75gb
octaipipe_lite-sql	1.77gb
octaipipe_lite-sql-root	1.77gb
octaipipe_lite-all_data_loaders	1.77gb
octaipipe_lite-all_data_loaders-root	1.77gb

Importing and Utilizing OctaiPipe Lite

OctaiPip Lite modules can be imported with the octaipipe_lite prefix. Here are some examples:

from octaipipe_lite.model_classes.fl_aquarium.kmeans_model import KMeansClustering
from octaipipe_lite.model_classes.sgd_classifier import SGDClassification

OctaiPipe Full

OctaiPipe Full includes all OctaiPipe functionality. It’s designed for comprehensive machine learning, FL and MLOps workflows.

Key features:

• User authentication

• User Interface

• Deployment capabilities via OctaiDeploy

• Federated Learning deployment

• PyTorch-based models

• MLOps

• Monitoring and observability features

• Additional pipeline steps (clustering, feature engineering, model evaluation)

Full Images

These images include PyTorch, MLOps deployment and model management tools and access to all OctaiPipe supported models.

Image Name	Size
octaipipe-mqtt	2.89 GB
octaipipe-mqtt-root	2.89 GB
octaipipe-influxdb	2.89 GB
octaipipe-influxdb-root	2.89 GB
octaipipe-sql	2.89 GB
octaipipe-sql-root	2.89 GB
octaipipe-all_data_loaders	2.89 GB
octaipipe-all_data_loaders-root	2.89 GB

Importing and Utilizing OctaiPipe Full

To import classes or functions found from OctaiPipe Full, use the octaipipe prefix.

Feature Comparison

Feature	Core	Lite	Full
Data reading and writing	yes	yes	yes
Custom Steps	yes	yes	yes
Federated EDA (FEDA)	yes	yes	yes
Federated SKLearn	no	yes	yes
Federated XGBoost	no	yes	yes
SKLearn Inference	no	yes	yes
Federated XGBoost	no	yes	yes
k-FED	no	yes	yes
SKLearn Preprocessing	no	yes	yes
Neural Network FL Training	no	no	yes
Non-FL Sklearn models	no	no	yes
MLOps	no	no	yes
OctaiPipe User Interface	no	no	yes