A 14B model on the GPUs your lab already owns
Many groups have a few workstations with 8 to 24 GB cards and no budget for one with 80 GB. SilentSwarm splits a model across the machines you have, so together they serve and fine-tune models none of them could hold alone - on hardware you run yourselves.
Buying capacity for a company? SilentSwarm for compute buyers
Why it fits a research group
The models worth studying outgrew the cards most labs own. Renting the difference by the hour works until the data may not leave the institute, or the grant runs out before the experiments do.
- Models bigger than any one card. Our benchmark model, Qwen3-14B, runs across four older GPUs of 4 and 6 GB. None of them could hold it alone.
- Fine-tune, not only chat. LoRA fine-tuning runs on the same split as serving, with checkpoints along the way and an exported model at the end. A thesis project can train on its own data without a cloud account.
- Data that stays put. The coordinator is a small server you run, and it needs no GPU. Prompts and training data stay on machines you run, the coordinator included.
- Numbers you can cite and rerun. Every benchmark we publish names the hardware, the commit and the script, so you can repeat it on your own pool.
Measured on two home PCs
Qwen3-14B in 4-bit, split across two ordinary PCs on one home internet connection: two GTX 1050 Ti cards with 4 GB each in one, two GTX 1660 SUPER cards with 6 GB each in the other. Measured on 2026-10-19 through the public chat API.
Small samples on old cards: these numbers show the shape, not a guarantee. The benchmark page has the method, the raw results and the limits, including a minute of waiting before the first token of a 1,500-token prompt.
Where it fits, and where something else is better
Each of these tools solves a real problem well. If a model fits on one of your machines, a single-machine tool will be faster than any distributed one.
| Tool | Built for | Choose it when |
|---|---|---|
| SilentSwarm | Serving and LoRA fine-tuning across mismatched machines, in one room or in different buildings | The model does not fit one card, and you want to train it, not only chat with it |
| Ollama, llama.cpp | Running a compressed model on one machine; llama.cpp can also spread one over a local network | The model fits one machine, or chatting on one local network is all you need |
| vLLM | Fast serving for many users on data centre GPUs | You have modern cards with enough memory and serve a lot of traffic |
| exo | Pooling the devices on one home network for chat | Everything is on one network and you only need answers, not training |
| Petals | A public swarm that serves very large models | Your prompts may run on strangers' machines, and you want to operate nothing |
| Cloud GPUs | Renting large cards by the hour | There is budget, the deadline is close, and the data may leave the institute |
The longer version, including the cases where we are the wrong choice: how SilentSwarm compares.
Three steps to a first pool
Written for someone who has used Docker and a terminal before. Each step links to its page in the docs.
- Run the coordinator. One small always-on machine runs it with Docker - a virtual machine or the department server is enough. It needs no GPU. Deployment
- Add each workstation with one command. It installs into one directory in the user's home, needs no administrator rights and connects out, so no port has to be opened. Adding machines
- Load a model, or start a fine-tune. From the web interface, the command line or the OpenAI-compatible API. Serving and training
What exists, and what does not yet
SilentSwarm is an open beta. We would rather a pilot starts with the gaps named than finds them halfway through a semester.
Available today
- Serving through an OpenAI-compatible API, across one machine or several.
- LoRA fine-tuning split across machines, with checkpoints and exported models.
- Published benchmarks with hardware, commit and scripts, on the benchmark page.
- Free use for noncommercial research. The licence names educational institutions and public research organisations as permitted users.
Not yet
- Training a large model from scratch. The split is built for fine-tuning.
- Fast long prompts on old cards. On the cards above, a long prompt waits about a minute before the answer starts.
- Every model family. The split is tested on the Llama, Qwen and GPT-2 families; others may need an adapter.
- Support with a response time. We help pilots ourselves, as fast as we can.
Questions labs ask
- Can a 14B model run on cards with 4 to 8 GB?
- Yes, split across enough of them. Our benchmark runs Qwen3-14B in 4-bit across four older GPUs with 4 and 6 GB in two PCs, and none of them could hold it alone. Each card holds a share of the model sized to its memory.
- How fast is it?
- On those two home PCs, 3.3 tokens per second for one user and 11.6 in total for eight users at once. These are small samples on old cards; the benchmark page has the method and the limits.
- Is it free for universities?
- Yes. SilentSwarm is licensed under the PolyForm Noncommercial licence, which names educational institutions and public research organisations as permitted users, however the work is funded.
- Do our prompts and data leave the institute?
- Not when you run it yourselves. The coordinator is a small server of yours, and prompts and training data stay on machines you run, the coordinator included.
- How does it compare to llama.cpp, exo or Petals?
- If the model fits on one machine, llama.cpp or Ollama on that machine will be faster. exo pools the devices on one home network for chat, and Petals serves models from a public swarm of other people's machines. SilentSwarm is for a model that fits no single card, when you want to fine-tune it as well as serve it, on machines that may sit in different rooms or buildings.
- What do we need to start?
- One small always-on machine for the coordinator, which needs no GPU, and the workstations whose GPUs will share the model. Each joins with one command, without administrator rights.
Running a lab? Tell us about your machines.
Which GPUs you have, which models you want to run or train, and what the work is for. Groups that start early get our help setting up their first pool. We answer every message ourselves.
We use your details only to answer you - see the privacy policy. SilentSwarm is source-available under the PolyForm Noncommercial licence; commercial use needs a licence from NexPatch AI, which you can ask about in the same message.