OctaiPipe Modules#

The OctaiPipe Python library consists of 3 modules with increasing functionality. Each preceding module is a dependency for the next. The modules are:

  • OctaiPipe Core

  • OctaiPipe Lite

  • OctaiPipe Full

Remember that when using OctaiPipe as a Data Scientist via an OctaiPipe Jupyter Notebook, you will always be working from an OctaiPipe Full Image. The question of which image module to use arises when deploying to the Edge, depending on your edge device resource constraints.

OctaiPipe Core#

OctaiPipe Core is the foundational module containing essential functionality with a lean footprint.

Key features:#

Core Pipeline steps:#

Core Images#

Image Name

Size

octaipipe_core-mqtt

439mb

octaipipe_core-mqtt-root

439mb

octaipipe_core-influxdb

480mb

octaipipe_core-influxdb-root

480mb

octaipipe_core-sql

487mb

octaipipe_core-sql-root

487mb

octaipipe_core-all_data_loaders

499mb

octaipipe_core-all_data_loaders-root

499mb

  • The suffix after - indicates the data loaders available to the image.

  • The suffix root indicates the image will be granted root permissions.

  • Check OctaiPipe Images for more information on image notation.

Importing OctaiPipe Core Modules#

OctaiPipe Full is a superset of Core and modules contained in core must be imported with the octaipipe_core prefix.

Examples of importing core classes from a Jupyter Notebook:

from octaipipe_core.data.influxdb.dataloader import InfluxDataLoader
from octaipipe_core.data.influxdb.datawriter import InfluxDataWriter
from octaipipe_core.data.mqtt.mqtt_client import MQTTClient
from octaipipe_core.pipeline.feda_step import FedaStep
from octaipipe_core.client.evaluation_client import EvaluationClient

OctaiPipe Lite#

OctaiPipe Lite builds upon Core, adding support for non neural network based federated learning and model inference.

Key features:

Lite Images#

Lite images are heavier in size with the addition of ML and FL libraries.

Image Name

Size

octaipipe_lite-mqtt

1.73gb

octaipipe_lite-mqtt-root

1.73gb

octaipipe_lite-influxdb

1.75gb

octaipipe_lite-influxdb-root

1.75gb

octaipipe_lite-sql

1.77gb

octaipipe_lite-sql-root

1.77gb

octaipipe_lite-all_data_loaders

1.77gb

octaipipe_lite-all_data_loaders-root

1.77gb

Importing and Utilizing OctaiPipe Lite#

OctaiPip Lite modules can be imported with the octaipipe_lite prefix. Here are some examples:

from octaipipe_lite.model_classes.fl_aquarium.kmeans_model import KMeansClustering
from octaipipe_lite.model_classes.sgd_classifier import SGDClassification

OctaiPipe Full#

OctaiPipe Full includes all OctaiPipe functionality. It’s designed for comprehensive machine learning, FL and MLOps workflows.

Key features:

  • User authentication

  • User Interface

  • Deployment capabilities via OctaiDeploy

  • Federated Learning deployment

  • PyTorch-based models

  • MLOps

  • Monitoring and observability features

  • Additional pipeline steps (clustering, feature engineering, model evaluation)

Full Images#

These images include PyTorch, MLOps deployment and model management tools and access to all OctaiPipe supported models.

Image Name

Size

octaipipe-mqtt

2.89 GB

octaipipe-mqtt-root

2.89 GB

octaipipe-influxdb

2.89 GB

octaipipe-influxdb-root

2.89 GB

octaipipe-sql

2.89 GB

octaipipe-sql-root

2.89 GB

octaipipe-all_data_loaders

2.89 GB

octaipipe-all_data_loaders-root

2.89 GB

Importing and Utilizing OctaiPipe Full#

To import classes or functions found from OctaiPipe Full, use the octaipipe prefix.

Feature Comparison#

Feature

Core

Lite

Full

Data reading and writing

yes

yes

yes

Custom Steps

yes

yes

yes

Federated EDA (FEDA)

yes

yes

yes

Federated SKLearn

no

yes

yes

Federated XGBoost

no

yes

yes

SKLearn Inference

no

yes

yes

Federated XGBoost

no

yes

yes

k-FED

no

yes

yes

SKLearn Preprocessing

no

yes

yes

Neural Network FL Training

no

no

yes

Non-FL Sklearn models

no

no

yes

MLOps

no

no

yes

OctaiPipe User Interface

no

no

yes