Skip to content

Architecture overview

SilentSwarm has one control-plane leader and multiple worker processes called fellows. The leader owns orchestration, registry state, APIs, streams, and the Ops UI. Fellows own training work, heartbeat reporting, log shipping, and tensor transport.

flowchart LR
    Operator[Operator / CI] --> CLI[swarm CLI]
    Operator --> UI[Ops UI]
    CLI --> Leader[Leader FastAPI]
    UI --> Leader
    Leader --> Registry[WorkerRegistry]
    Leader --> Jobs[JobManager]
    Leader --> Signals[SignalHub]
    Leader --> Metrics[Metrics and log streams]
    Signals -- WebSocket --> F0[Fellow 0]
    Signals -- WebSocket --> F1[Fellow 1]
    F0 -- activations --> F1
    F1 -- gradients --> F0
    F0 -- heartbeat/logs --> Leader
    F1 -- heartbeat/logs --> Leader

Design goals

Goal Current approach
Orchestration FastAPI leader with jobs, workers, signals, metrics, logs, and Ops UI.
Pipeline training Fellows execute assigned stages and exchange activation/gradient tensors.
Bandwidth research Compression boundaries compare no compression, fixed quantization, and learned bottlenecks.
Operational access Local CLI, Docker Compose frontend, remote bootstrap, and optional tunnel.
Observability Heartbeats, worker log streams, job metrics streams, and run artifacts.

Two planes

flowchart TB
    subgraph Control[Control plane]
        REST[REST /api/v1]
        WS["WebSocket /ws/signals/{fellow_id}"]
        SSE[SSE logs and metrics]
    end
    subgraph Data[Data plane]
        ZMQ[ZeroMQ PAIR link]
        Relay[HTTP relay fallback]
    end
    Leader[Leader :8080] --> Control
    Fellows[Fellows] --> Control
    Fellows --> Data

Layers in the code

The code is split by responsibility into three layers, with imports pointing down only:

flowchart TB
    App["Application<br/>leader, fellow, cli, bootstrap, shared"]
    Runtime["Runtime<br/>load a model, train a stage, generate, export"]
    subgraph Pipe["Mechanic: pipe"]
        Split["L1 split - sees models"]
        Wire["L2 wire - sees tensor frames"]
        Link["L3 link - sees bytes"]
    end
    App --> Runtime --> Pipe
    App -. torch-free parts .-> Pipe
  • Mechanic (pipe/): splitting a model into stages, framing the tensors that cross a stage boundary, and moving the bytes directly or through the leader's relay. It mirrors the standalone swarmpipe library and knows nothing about SilentSwarm.
  • Runtime (runtime/): what a GPU worker does with the mechanic - load a model, run a training loop, serve generation, export checkpoints and adapters. The only layer besides pipe/split/torch/ that imports PyTorch.
  • Application (leader/, fellow/, cli/, bootstrap/, shared/): who runs what, where and when - accounts, pools, jobs, placement, the web UI and the worker lifecycle. The leader never imports PyTorch; the fellow starts the runtime as a subprocess.

Much of the mechanic still lives under runtime/ and moves into pipe/ step by step; Contributing has the rules a change has to follow.

Pages