Architecture

This guide introduces Beamstack’s ecosystem and explains how it fits in the ML lifecycle. Read the introduction guide to learn more about Beamstack.

Beamstack Ecosystem

The following diagram gives an overview of the Beamstack Ecosystem and how it relates to the wider Kubernetes and AI/ML landscapes.

How does Beamstack initilize components on a cluster?

Beamstack pulls components directly from the registry using helm and kustomize and installs them on the kubernetes cluster.

How does Beamstack deploy Beam YAML pipeline’s to the cluster?

Beamstack copies the Beam YAML pipeline file to the persistent volume, then the pipeline runners access the file from the persistent volume and create jobs.

Beamstack Components

See the following links for more information about each Beamstack component:

• Beamstack CLI can be used for deploying and monitoring AI/ML Pipelines

• Beamstack Custom Transforms can be used for data tranformation.

• Apache Beam YAML contains pipeline specifications

Next steps

• Follow Installing Beamstack to set up your environment and install Beamstack.