SkyPilot - Run AI Workloads on Any Cloud Cheatsheet
SkyPilot runs AI and batch workloads on any cloud or Kubernetes cluster from a single YAML definition. Its core value for LLM work is economic: GPU availability and pricing vary wildly across providers and regions, and SkyPilot automatically finds the cheapest available capacity that meets your requirements, provisions it, syncs your code, runs the job, and tears everything down. It also handles spot-instance preemption with automatic recovery, which makes cheap interruptible GPUs practical for long training runs.
Installation
| Method | Command |
|---|
| pip | pip install "skypilot[all]" |
| Specific clouds | pip install "skypilot[aws,gcp,kubernetes]" |
| Check credentials | sky check |
| Verify | sky --version |
sky check reports which clouds you have working credentials for — SkyPilot only considers those.
Defining a Task
# train.yaml
resources:
accelerators: A100:8
use_spot: true
any_of:
- cloud: aws
- cloud: gcp
- cloud: kubernetes
num_nodes: 1
workdir: .
setup: |
pip install -r requirements.txt
run: |
torchrun --nproc_per_node=8 train.py --config configs/7b.yaml
| Field | Purpose |
|---|
resources.accelerators | GPU type and count (A100:8, H100:4) |
use_spot | Use interruptible instances |
any_of / ordered | Candidate clouds/regions |
workdir | Local dir synced to the cluster |
setup | Run once per cluster |
run | The actual job |
num_nodes | Multi-node training |
Core Commands
| Command | Description |
|---|
sky launch -c mycluster train.yaml | Provision and run |
sky exec mycluster train.yaml | Run again on an existing cluster |
sky status | List clusters and costs |
sky logs mycluster | Stream job logs |
sky queue mycluster | Show the job queue |
sky down mycluster | Terminate (stop paying) |
sky stop mycluster | Stop but keep the disk |
Managed Jobs (Spot with Recovery)
# Long training run on spot instances, auto-recovered on preemption
sky jobs launch -n llama-sft train.yaml
| Command | Purpose |
|---|
sky jobs launch | Submit a managed job |
sky jobs queue | List managed jobs |
sky jobs logs -n NAME | Tail logs |
sky jobs cancel -n NAME | Cancel |
Managed jobs are the feature that makes spot GPUs usable for training: on preemption SkyPilot re-provisions elsewhere and resumes from your checkpoint, so a multi-day run survives interruptions.
Cost Optimization
| Mechanism | Effect |
|---|
| Cross-cloud search | Picks the cheapest matching option |
use_spot: true | Often 60–90% cheaper |
sky show-gpus | Current prices/availability by cloud |
| Autostop | sky autostop -i 10 mycluster terminates idle clusters |
sky down | The most important habit — idle GPUs are the main waste |
# What A100 capacity exists and at what price?
sky show-gpus A100
Serving Models
# serve.yaml
service:
readiness_probe: /v1/models
replicas: 2
resources:
accelerators: A100:1
ports: 8000
run: |
vllm serve meta-llama/Llama-3.1-8B-Instruct --port 8000
sky serve up -n llm-api serve.yaml
sky serve status
SkyServe adds replicas, health checks, and autoscaling across clouds — useful for serving with vLLM on the cheapest available GPUs.
Kubernetes Backend
| Capability | Note |
|---|
| Existing clusters | cloud: kubernetes targets your own K8s |
| Mixed | Combine on-prem K8s with cloud burst |
| GPU scheduling | Uses standard K8s GPU resources |
| No cloud credentials needed | For pure on-prem use |
SkyPilot vs Alternatives
| Aspect | SkyPilot | Ray | Cloud-native (SageMaker/Vertex) |
|---|
| Multi-cloud | Core feature | Possible, manual | Single vendor |
| Spot recovery | Built-in | Manual | Varies |
| Lock-in | None (plain YAML) | Framework | High |
| Best for | Cheapest GPUs anywhere | Distributed Python apps | Deep single-cloud integration |
Pairs with training stacks like Axolotl, TorchTitan, and serving via vLLM.
Resources
