How SilentSwarm compares¶
The honest version. Every project on this page solves a real problem well, and for several of the questions people arrive with, one of them is a better answer than we are. This page is about where the line falls.
What this page does not do
It does not publish throughput comparisons against other projects. We have not run them on identical hardware, and a benchmark table assembled from other people's published numbers is marketing, not measurement. What we intend to publish about our own numbers, and how, is on the benchmarks page.
The short version¶
SilentSwarm is for a group of people who each have a GPU, want to run and train models that none of their machines could hold alone, and would rather not open a port or configure a VPN to do it.
Four things follow from that, and together they are the difference:
- Train and serve on the same topology. The pipeline that answers a request is the pipeline that runs a fine-tune. Most of the alternatives do one or the other well.
- A learned bottleneck at the stage boundary. Instead of shipping full activations between machines, a small trained compression layer sits at the seam. That is what makes pipeline parallelism over ordinary internet links arithmetically plausible rather than merely possible.
- NAT-friendly by default, with no network configuration. Workers connect out. When two of them can reach each other they talk directly; when they cannot, the coordinator relays. Nothing needs a forwarded port, and nothing needs a VPN - though it will use one if you have it.
- One command to add a machine. A
curlline that verifies its own source archive against a published checksum, builds its environment, and joins.
Where something else is the better answer¶
You have one machine, or several on one LAN, and you only want inference. Use Ollama, llama.cpp or vLLM. A single-node serving stack is faster, simpler, and better documented than any distributed system, and distributing a model that fits on one card costs you latency for nothing.
You want to run inference on a very large public model right now, for free, without operating anything. Petals is a public swarm you can join and use today. SilentSwarm is software you host; the pool is whoever you invite. That is a different trade: you get control and privacy, and you get the operational work that comes with them.
You have a rack, a scheduler, and a platform team. Ray is a general distributed-computing framework with a far larger surface than this, and GPUStack is built for managing a fleet of GPU servers as infrastructure. Neither is trying to solve "these six people's home machines are behind six different routers", which is the problem we started from.
You want the simplest possible way to use the GPUs already in your house. Exo is aimed squarely at that and is pleasant to start. If everything is on one network and inference is all you need, that simplicity is worth a lot.
Where SilentSwarm is the better answer¶
The machines are in different buildings, behind different routers, owned by different people. This is the case the transport was designed around: outbound connections, automatic direct-vs-relay selection per pair, and a relay that is part of the coordinator rather than a separate service to run.
You want to fine-tune, not just serve. Training across a pipeline split is the original purpose here, and the frozen-backbone plus learned-bottleneck path exists so that a group with mismatched consumer cards can train something without each of them needing the whole model in memory.
Who may use which model matters. Pools are the boundary: a pool's model is served only on that pool's own machines, membership is explicit, and the coordinator records timings rather than prompt text.
You need it to be yours. Self-hosted, source published, no account with us required to run it. The licence is free for noncommercial use - and it is not an OSI open-source licence, which we would rather say plainly than let you discover in a pull request.
What is not built yet¶
The comparison is only useful if this part is on the same page:
| Status | |
|---|---|
| Pipeline depth | Two stages today. Generalising to N stages is the top item on the roadmap. |
| Fault tolerance | Not yet. A worker that vanishes mid-job takes the job with it. Stage checkpoint and re-placement are planned. |
| Sandboxing | No. A worker runs as an ordinary user process with your permissions - not a container, not a VM. Run it on a machine where that is acceptable. |
| Paying contributors | No. Contribution is measured and shown; credits, published rates and payouts are roadmap, in that order. |
| Published benchmarks | Not yet. See benchmarks for what we intend to measure and how. |
If any of those four rows is a blocker for you today, one of the projects above is a better use of your afternoon, and we would rather you found that out here.