GPU Infrastructure
Schedule GPUs, manage workers, and operate model services across machines or clusters.
GPUStack
GPU InfraGPUStack · Apache-2.0
A GPU cluster manager and model-serving control plane for heterogeneous infrastructure.
Pooling GPUs across multiple servers
LiteLLM
ServingBerriAI · MIT for the open-source project; enterprise features use separate terms
A self-hosted LLM gateway and Python SDK that normalizes many hosted and local model providers behind OpenAI-compatible interfaces.
One API across several model providers
Xinference
ServingXorbits · Apache-2.0
A model-serving platform for deploying language, embedding, reranking, image, and audio models.
Serving several model types from one control plane
KServe
GPU InfraKServe Project · Apache-2.0
A Kubernetes-native inference platform for standardizing scalable predictive and generative model services.
Standardized model services on Kubernetes