Skip to content
1337skills 1337skills

py-spy Commands

py-spy is a sampling profiler for Python programs written in Rust. It can profile running Python processes without modifying code or restarting them, producing flame graphs and real-time top-like views with minimal overhead.

Installation

Linux/Ubuntu

# Install via pip
pip install py-spy

# Install via cargo (Rust)
cargo install py-spy

# Install via package manager (Arch Linux)
pacman -S py-spy

# Verify installation
py-spy --version

# py-spy requires root to attach to processes you don't own
# Or set ptrace permissions:
echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope

py-spy record — Generate Profiles

# Record a flame graph SVG from a script
py-spy record -o profile.svg -- python my_script.py

# Record from a running process by PID
sudo py-spy record -o profile.svg --pid 1234

# Record for a specific duration (seconds)
sudo py-spy record -o profile.svg --pid 1234 --duration 60

# Set sampling rate (default 100 Hz)
py-spy record -o profile.svg --rate 200 -- python my_script.py

# Record only idle threads (waiting/sleeping)
py-spy record -o profile.svg --idle --pid 1234

# Record including native C/C++ extension frames
py-spy record -o profile.svg --native -- python my_script.py

# Record subprocesses (follow forks)
py-spy record -o profile.svg --subprocesses -- python my_script.py

# Record only GIL-holding threads
py-spy record -o profile.svg --gil -- python my_script.py

# Record with function line numbers
py-spy record -o profile.svg --function -- python my_script.py

# Non-blocking mode (may miss some samples)
py-spy record -o profile.svg --nonblocking --pid 1234

Output Formats

# SVG flame graph (default)
py-spy record -o profile.svg --format flamegraph -- python my_script.py

# speedscope format (view at https://speedscope.app)
py-spy record -o profile.speedscope.json --format speedscope -- python my_script.py

# Raw sample output
py-spy record -o profile.raw --format raw -- python my_script.py

# Chrome trace format (view in chrome://tracing)
py-spy record -o profile.json --format chrometrace -- python my_script.py

# pprof format (compatible with Go pprof tools)
py-spy record -o profile.pb.gz --format pprof -- python my_script.py

py-spy top — Live Profiling

# Live top-like view of a running process
sudo py-spy top --pid 1234

# Live view of a script
py-spy top -- python my_script.py

# Include native frames in top view
sudo py-spy top --native --pid 1234

# Show idle threads
sudo py-spy top --idle --pid 1234

# Include subprocesses
py-spy top --subprocesses -- python my_script.py

# Custom sampling rate
sudo py-spy top --rate 50 --pid 1234

py-spy dump — Thread Dump

# Dump current stack traces of all threads
sudo py-spy dump --pid 1234

# Dump with local variables
sudo py-spy dump --locals --pid 1234

# Dump in JSON format
sudo py-spy dump --json --pid 1234

# Dump including native frames
sudo py-spy dump --native --pid 1234

Profiling Specific Scenarios

Web Applications

# Profile a Django application
py-spy record -o django_profile.svg -- python manage.py runserver

# Profile a Flask app
py-spy record -o flask_profile.svg -- python app.py

# Profile a Gunicorn worker (attach to specific worker PID)
sudo py-spy record -o gunicorn_profile.svg --pid $(pgrep -f 'gunicorn.*worker')

# Profile uvicorn (ASGI)
py-spy record -o uvicorn_profile.svg --subprocesses -- uvicorn main:app --workers 4

Data Processing

# Profile a pandas/numpy workload with native extensions
py-spy record -o data_profile.svg --native -- python etl_pipeline.py

# Profile with subprocesses (multiprocessing)
py-spy record -o multiproc_profile.svg --subprocesses -- python parallel_job.py

Long-Running Services

# Attach to a running daemon for 5 minutes
sudo py-spy record -o daemon_profile.svg --pid 1234 --duration 300

# Periodic profiling via cron (30-second snapshots)
# Add to crontab: */5 * * * * sudo py-spy record -o /tmp/profile_$(date +\%s).svg --pid $(cat /var/run/myapp.pid) --duration 30

Docker and Containers

# Profile a Python process in a Docker container
# 1. Find the container PID on the host
CONTAINER_PID=$(docker inspect --format '{{.State.Pid}}' my_container)

# 2. Attach py-spy to the container's PID namespace
sudo py-spy record -o container_profile.svg --pid $CONTAINER_PID

# Or run py-spy inside the container
docker exec -it my_container py-spy top --pid 1

# Docker run with SYS_PTRACE capability for py-spy
docker run --cap-add SYS_PTRACE my_image

Kubernetes

# Profile a Python pod
# 1. Find the node and PID
kubectl exec -it my-pod -- pgrep -f python

# 2. Use kubectl exec with py-spy installed in the container
kubectl exec -it my-pod -- py-spy record -o /tmp/profile.svg --pid 1 --duration 30

# 3. Copy the profile out
kubectl cp my-pod:/tmp/profile.svg ./profile.svg

# For ephemeral debug containers
kubectl debug -it my-pod --image=py-spy-debug --target=my-container -- py-spy top --pid 1

Interpreting Output

Flame Graph (SVG)

# Reading the flame graph:
# - X-axis: proportion of total CPU time (wider = more CPU)
# - Y-axis: stack depth (bottom = entry point, top = leaf function)
# - Each box: a function call
# - Click to zoom, Ctrl+F to search
# - Color is random (not meaningful by default)

Top View Columns

# Column explanations in py-spy top:
# %Own    — CPU time in this function only (self time)
# %Total  — CPU time in this function + callees
# OwnTime — Absolute self time
# TotalTime — Absolute total time
# Function — Module and function name
# Line    — Source file and line number

Combining with Other Tools

# Generate folded stacks for Brendan Gregg's FlameGraph tools
py-spy record -o profile.raw --format raw -- python my_script.py
# Raw format is already folded stack compatible

# Convert to speedscope for interactive analysis
py-spy record -o profile.speedscope.json --format speedscope -- python my_script.py
# Open at https://speedscope.app

# Convert pprof output for use with go tool pprof
py-spy record -o profile.pb.gz --format pprof -- python my_script.py
go tool pprof -http=:8080 profile.pb.gz

Troubleshooting

# Permission denied — need root or ptrace access
sudo py-spy top --pid 1234
# Or: echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope

# "Error: Failed to find python version" — specify Python path
sudo py-spy record --pid 1234 --python-program /usr/bin/python3

# Missing symbols in native mode — install debug packages
sudo apt install python3-dbg

# Process not found in container — use host PID
docker inspect --format '{{.State.Pid}}' my_container

Quick Reference

CommandPurpose
py-spy recordGenerate flame graph or profile file
py-spy topLive top-like CPU profiler view
py-spy dumpOne-shot thread stack dump
--pid PIDAttach to running process
--nativeInclude C/C++ extension frames
--subprocessesFollow child processes
--gilOnly profile GIL-holding threads
--rate NSet sampling frequency (Hz)
--formatOutput format (flamegraph, speedscope, raw, chrometrace, pprof)