Tool Category
GPU Infrastructure
Schedule GPUs, manage workers, and operate model services across machines or clusters.
Open source
GPUStack
GPU InfraGPUStack · Apache-2.0
A GPU cluster manager and model-serving control plane for heterogeneous infrastructure.
Self-hostedClusterLinuxDocker
Best for
Pooling GPUs across multiple servers
Open source
Xinference
ServingXorbits · Apache-2.0
A model-serving platform for deploying language, embedding, reranking, image, and audio models.
Desktop / localSelf-hostedmacOSWindows
Best for
Serving several model types from one control plane
Open source
KServe
GPU InfraKServe Project · Apache-2.0
A Kubernetes-native inference platform for standardizing scalable predictive and generative model services.
ClusterLinuxKubernetes
Best for
Standardized model services on Kubernetes