GPUStack
A GPU cluster manager and model-serving control plane for heterogeneous infrastructure.
By GPUStack · reviewed 2026-08-07
Deep profile checked 2026-08-16 · GPUStack latest installation, requirements, and release documentation
What GPUStack does
GPUStack discovers GPU workers, schedules model instances, and exposes managed inference endpoints across one or more machines. It sits above engines such as vLLM and SGLang, making it better suited to shared infrastructure than a single-user desktop runtime.
Pricing model
Free and open source; enterprise offerings available
Position in the stack
Where GPUStack fits
GPUStack is the infrastructure control plane in a self-hosted AI stack. It discovers GPU workers, schedules model instances, selects or integrates serving engines, and provides managed endpoints and operational visibility. User-facing chat, RAG, and agent applications normally sit above it.
Typical deployment flow
A practical sequence for evaluating GPUStack before making it part of a permanent stack.
- STEP 1
Create the control plane
Install the GPUStack server and define authentication, persistent state, networking, and administrative access.
- STEP 2
Register workers
Attach supported Linux GPU machines and confirm drivers, accelerator discovery, network reachability, and available memory.
- STEP 3
Deploy model services
Choose a compatible serving engine, schedule replicas, expose endpoints, and monitor resource and service health.
Best for
- Pooling GPUs across multiple servers
- Operating shared model endpoints
- Managing heterogeneous accelerators and serving engines
Not the right layer for
- Running models directly on a Mac worker
- A personal desktop chat application
- Building RAG and agent workflows by itself
System fit
Requirements and hardware notes
Application requirements are separate from the memory needed by the selected model and context window.
Check your hardwareMinimum baseline
- Modern Linux worker nodes on AMD64 or ARM64
- Docker and the accelerator vendor's driver/container runtime
- Bidirectional server-worker networking and required port ranges
Recommended baseline
- Separate CPU server and Linux GPU workers
- NTP on every node
- External database and planned backup for larger installations
Hardware-specific notes
- The server can run without a GPU.
- GPU workers are Linux-only in the current quickstart guidance.
- NVIDIA, AMD, Ascend, Hygon, MetaX and other accelerators have different backend requirements.
Installation and deployment paths
Choose one path that matches the number of users and the level of operations you can maintain.
Single Docker server
Best for initial control-plane evaluation
Docker and persistent server data
Server plus workers
Best for heterogeneous GPU inventory
Linux GPU workers, drivers, container toolkits, and network ports
Kubernetes with Helm
Best for teams already operating Kubernetes
Supported Kubernetes and Helm versions plus NFD
Data boundary
What stays local and what may leave
- Local data
- Model placement, worker state, credentials, and deployment metadata are stored in the GPUStack control plane and configured database.
- Network dependencies
- Workers need registry and model-source access unless images and weights are mirrored for an air-gapped deployment.
- Accounts and telemetry
- Restrict the administration UI, rotate the initial password, and protect worker and inference ports.
License checkpoints
- GPUStack's license is separate from vLLM, SGLang, drivers, container images, and models.
- Vendor accelerator runtimes can impose additional redistribution terms.
Operational checkpoints
- Open only the documented server, worker, inference, and distributed-serving ports.
- Monitor driver and backend compatibility before upgrading workers.
- Use placement failures as scheduling evidence, not proof that a model can never run on the hardware.
Capabilities
Capabilities refer to the tool's application layer. Hardware and model support can still depend on a connected inference engine.
Deployment and compatibility
- Deployment
- Self-hostedCluster
- Platforms
- Linux · Docker · Kubernetes
- Hardware backends
- NVIDIA GPU · AMD GPU · Huawei Ascend · Hygon DCU · Moore Threads · Iluvatar · MetaX · Cambricon · T-Head
- Install methods
- Installer · Docker · Kubernetes
Models and integrations
- Model formats
- Depends on selected serving engine
- Common integrations
- vLLMSGLangTensorRT-LLMOpenAI-compatible clients
Strengths
- Central GPU and model control plane
- Multi-node scheduling and observability
- Support for several accelerator vendors
Limitations
- Worker support and model compatibility depend on the underlying engine
- Adds infrastructure overhead for a single workstation
Planning checklist
Before you choose GPUStack
Answer these questions with the exact models, hardware, users, and data you expect to operate.
- Are all intended worker accelerators and drivers supported?
- Can model files and container images be distributed reliably to workers?
- How should models be placed across different GPU memory sizes?
- Which application layer will provide RAG, agents, and end-user access?
GPUStack FAQ
Does GPUStack replace vLLM or SGLang?
No. GPUStack can manage model services that use serving engines such as vLLM or SGLang. It solves a broader scheduling and operations problem.
Is GPUStack useful for one GPU?
It can be used, but the control-plane overhead is usually justified when there are multiple models, users, GPUs, or worker machines to manage.
Can a Mac be a GPUStack worker?
Current worker guidance is centered on supported Linux environments and accelerators. Check the current official compatibility documentation before planning a Mac worker.
Official sources
Use these links to confirm current compatibility and installation requirements.
Related tools
A model-serving platform for deploying language, embedding, reranking, image, and audio models.
A Kubernetes-native inference platform for standardizing scalable predictive and generative model services.
A portable C/C++ inference engine that underpins much of the GGUF local-model ecosystem.
Comparisons featuring GPUStack
Use a reviewed comparison when the choice is between two adjacent tools.
Execution evidence
Known working recipes using GPUStack
Recipes connect hardware, a model artifact, tools, settings, verification, and a reportable result.
No verified recipe is linked to this record yet.
Compatibility estimates remain available in the planner. A recipe appears here only after its exact stack and verification protocol are documented.